A few months back I put together a robots.txt handler for EPiServer CMS 7. Technology moves quickly and since then EPiServer have released CMS version 7.5

My generous ex-colleague Marcin (https://github.com/wma83) has forked and updated the package to work with version 7.5. This involved a few minor changes to handle the changes in how sites and site settings are defined.

Its on the Nuget feed already – http://nuget.episerver.com/ – search for POSSIBLE.RobotsTxtHandler version 2.0.1

The source code is available at https://github.com/markeverard/POSSIBLE.RobotsTxtHandler