# ...but allow the googlebot user agent to crawl nearly anywhere: User-agent: Googlebot/2.X (+http://www.googlebot.com/bot.html) Disallow: /local/ Disallow: /cgi-bin/ Disallow: /~chem215/lectures/ Disallow: /~chem215/local/ Disallow: /~cmura/restricted/ Disallow: /~cmura/JOURNALS_and_LIT/ Disallow: /~adcock/restricted/ # prevent robots from indexing index.html and various directories... User-agent: * Disallow: /local/ Disallow: /cgi-bin/ Disallow: /~chem215/lectures/ Disallow: /~chem215/local/ Disallow: /~cmura/restricted/ Disallow: /~cmura/JOURNALS_and_LIT/ Disallow: /~cmura/Science/Mura_IntroMD_May2007.pdf Disallow: /~adcock/restricted/ Disallow: /index.html