User-agent: * Disallow: /upload/ Disallow: /*.pdf$ Disallow: /*.swf$ Disallow: /*.csv$ Disallow: /*.xls$ Disallow: /*.doc$