Mark Everard

Hello, I'm Mark – a PhD physicist turned developer / architect.

A Robots.txt handler for EPiServer 7

with 2 comments

David Knipe put together a really nice module for EPiServer CMS 6R2 called EPiRobots which allowed Site Administrators to manage and update a site’s robots.txt content. Unfortunately the module doesn’t work with CMS 7 :(

I asked Dave about his plans for a version compatible with CMS 7 and there is apparently some work/ideas in the pipeline. However that news did not satisfy my IMMEDIATE requirement! Dave also mentioned that I could ‘do whatever I wanted with the source code’. So I’ve forked it!

I’ve upgraded the forked version to work against CMS 7. The main change was to remove an initialisation piece that added an explicit handler for the Robots.txt path through the EPiServer UrlRewriting API (which has been overhauled for CMS 7). The robots.txt content is now served from a standard MVC controller with a route being configured in an EPiServer initialization module. I’ve also rebuilt the Admin user interface using MVC and explicitly included the views as content within the NuGet package rather than serving them from the .dll. This is just my personal preference. Leveraging NuGet and the simplicity of the solution wins for me! It also means you can change the view look and feel if you so wish.

The source code is available at https://github.com/markeverard/POSSIBLE.RobotsTxtHandler

A package (now renamed from EPiRobots) will be available in the EPiServer Nuget Feed (as soon as its been approved) - http://nuget.episerver.com/ - search for POSSIBLE.RobotsTxtHandler

Written by Mark

November 1st, 2013 at 11:59 am

Posted in ASP.NET,C#,EPiServer

  • http://dmytroduk.com/ Dmytro Duk

    Looks good!
    As I can see it can be easily converted into an add-on. It does not require customizing in Visual Studio. Web.config update can be removed if module.config contains the list of module assemblies.
    What about releasing it as an add-on?

  • Vladimir Levchuk

    Hi Mark,

    Thank you for you effort!

    I tried to use your nuget package on EPiServer 7.0 (with assembly versions 7.0.*.1) and get the exception listed below ( solution is proposed at https://github.com/markeverard/POSSIBLE.RobotsTxtHandler/pull/1 ).

    Unable to load one or more of the requested types. The following information may be a subset of the Type/LoaderException information present – inspect with debugger for complete view.
    Check assemblies [POSSIBLE.RobotsTxtHandler, Version=1.0.3.0, Culture=neutral, PublicKeyToken=null] and/or types [POSSIBLE.RobotsTxtHandler.IRobotsContentService,
    POSSIBLE.RobotsTxtHandler.RobotsTxtData,
    POSSIBLE.RobotsTxtHandler.RobotsContentService,
    POSSIBLE.RobotsTxtHandler.UI.Controllers.RobotsTxtController,
    POSSIBLE.RobotsTxtHandler.UI.Models.RobotsTxtViewModel,
    POSSIBLE.RobotsTxtHandler.UI.Controllers.RobotsTxtAdminController,
    f__AnonymousType0`2,
    f__AnonymousType1`2]. Information from LoaderExceptions property [Could not load file or assembly 'EPiServer.Framework, Version=7.0.859.16, Culture=neutral, PublicKeyToken=8fe83dea738b45b7' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040),
    Could not load file or assembly 'EPiServer.Framework, Version=7.0.859.16, Culture=neutral, PublicKeyToken=8fe83dea738b45b7' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)].