#Vuoi sapere come si crea un testo come questo? #Vai su http://webmarketing.html.it/articoli/leggi/84/il-file-robotstxt-escludere-i-motori-indesiderati/ # #Per onestà ammetto che questo file robots.txt è stato copiato paro paro dal sito HTML.it #Adesso non venitemi a dire che sono un copione scroccone perchè non è vero ;) # #Creato il 03/05/2006 per Pino La Mura #Riccardo Romagnoli - www.romadesign.it #Aggiunto il 03 Maggio 2006 User-agent: webmirror Disallow: / #Aggiunto il 03 Maggio 2006 User-agent: webcopy Disallow: / #Aggiunto il 03 Maggio 2006 User-agent: WinHTTrack Disallow: / #Aggiunto il 03 Maggio 2006 User-agent: website extractor Disallow: / #Aggiunto il 03 Maggio 2006 User-agent: SuperBot Disallow: / #Aggiunto il 03 Maggio 2006 User-agent: SuperBot/2.6 Disallow: / #Aggiunto il 03 Maggio 2006 User-agent: WebReaper Disallow: / User-agent: Googlebot-Image Disallow: / User-agent: Black Hole Disallow: / User-agent: Titan Disallow: / User-agent: NetMechanic Disallow: / User-agent: CherryPicker Disallow: / User-agent: EmailCollector Disallow: / User-agent: DISCo Pump 3.1 Disallow: / User-agent: NetAttache Disallow: / User-agent: NetAttache Light 1.1 Disallow: / User-agent: EmailSiphon Disallow: / User-agent: WebBandit Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: Crescent Disallow: / User-agent: SiteSnagger Disallow: / User-agent: ProWebWalker Disallow: / User-agent: Alexibot Disallow: / User-agent: Teleport Disallow: / #User-agent: TeleportPro #Disallow: / #User-agent: Teleport Pro/1.29.1718 #Disallow: / #User-agent: Teleport Pro/1.29.1632 #Disallow: / #User-agent: Teleport Pro/1.29.1590 #Disallow: / #User-agent: Teleport Pro/1.29 #Disallow: / User-agent: Wget Disallow: / User-agent: MIIxpc Disallow: / User-agent: Telesoft Disallow: / User-agent: Website Quester Disallow: / User-agent: WebCopier Disallow: / User-agent: NetAnts Disallow: / User-agent: Mister PiX Disallow: / User-agent: WebAuto Disallow: / User-agent: TheNomad Disallow: / User-agent: turingos Disallow: / User-agent: spanner Disallow: / User-agent: InfoNaviRobot Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: Bullseye/1.0 Disallow: / User-agent: URLy Warning Disallow: / User-agent: Wget/1.5.3 Disallow: / User-agent: LinkWalker Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: Mata Hari Disallow: / User-agent: LexiBot Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Web Image Collector Disallow: / User-agent: JennyBot Disallow: / User-agent: MIIxpc/4.2 Disallow: / User-agent: BuiltBotTough Disallow: / User-agent: ProPowerBot/2.14 Disallow: / User-agent: BackDoorBot/1.0 Disallow: / User-agent: toCrawl/UrlDispatcher Disallow: / User-agent: WebEnhancer Disallow: / User-agent: TightTwatBot Disallow: / User-agent: Szukacz/1.4 Disallow: / User-agent: QueryN Metasearch Disallow: / User-agent: Openfind data gathere Disallow: / User-agent: Xenu's Disallow: / User-agent: Zeus Disallow: / User-agent: RepoMonkey Bait & Tackle/v1.01 Disallow: / User-agent: Kenjin Spider Disallow: / User-agent: Cegbfeieh Disallow: /