############################### # # User-agent: * # # list folders robots are not allowed to index # Disallow: /log/ Disallow: /stats/ Disallow: /cgi-bin/ Disallow: /sxst/ Disallow: /sxst 2008/ Disallow: /SXST CS3/ Disallow: /sxt/ # # list specific files robots are not allowed to index # # # ###############################