Custom error pages replaced by IIS?

Development Machine W7, Production IIS7 Windows 2008

Just had a sticky problem crop up in production. Custom error pages that used to work no longer work, I don’t know when but sometime our custom ASP.NET error pages have been replaced by stock IIS error pages.

How we use custom error pages

If a website user or search bot attempts to access a product page and the product no longer exists, we send them to a custom error page. The page suggests they try another product or search for alternatives. This page sends back a http 404 status so that the search engines know to drop the accessed URL from the index. It also causes our Google Mini to drop the page from its database too. If the user navigates to another url that has never had content they get a default 404 page returned.

Lets have a look

So you have a great little custom error page like this;

Protected Sub Page_Load1(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

        'Ensure a 404 not found sent for SEO purposes

        Response.StatusCode = 404

        Response.TrySkipIisCustomErrors = True

    End Sub

However even browsing direct to the page you only get,

This is not your crafted custom page, instead this is the IIS7 page kicking in. IIS has put itself in the pipeline to the end user, replacing its own page with in place of the custom page.

IIS custom page configuration

The configuration for this is found in the site Error Pages section;

If you go into this menu you see all the custom error pages set up in IIS, now select the edit feature settings link on the right hand side;

This is where the behaviour is set and currently this is set to show detail errors for users local to the machine but custom errors for people elsewhere. Note this is not ASP.NET, although the idea is the same, this is above ASP.NET, it is the handling IIS does for these pages.

You do not want normal users to see the detailed errors as it can give away important information about the configuration of your server thus the setting shown is fine and looking at it one would guess it would show our custom pages as we saw by the returned IIS 404 page this is not the case.

So why do we not see our custom pages? Well back in the page code behind shot earlier is the key, you must set

Response.TrySkipIisCustomErrors = True

This tells IIS to get its nose out of what we’re up to and let us get on with it. Trying to browse the custom error page, now we get our custom error page retuned as would be expected in the first case.

Lesson learnt

The real nasty bit of this is that our development machines do not exhibit this behaviour, everything looks fine there. As Rick Strahl says it is another good case for using the staging servers with the production environment so that you test everything.

Other links:

Rick Strahl's Web Log has an almost identical post here

What to expect from IIS7 custom error modulethis link was the one that taught me what I needed to solve the problem.

I missed something important about Windows Media Services WMS in Windows 2008. A lot of the features that were previously only available in Windows 2003 datacenter or Windows 2003 Enterprise editions are standard in Windows 2008! Yipee and about time!

The inability to write custom authentication plugins except for the advanced windows versions prevented an old project from actually being viable. Now custom authentication plugins will work for me.

The new 2008 version is reportedly much more efficient and up to some serious traffic and has some other very relevant to features to some work I cant talk about;

More...

Project complete

Finally there has been a window of opportunity to get the CMS->Web application complete.

Read my write up of what  I’ve been up to with it here.

It uses WCF, LINQ to objects and a smart client windows forms application to publish item information from the catalogue management system in Quark Xpress format Oracle binary fields to a SQL server driven ASP.NET website.

This is version one and as always there are already some ideas on how it could be used to improve the experience of the site visitors. Already I’ve changed it to hot link product  codes and “see index” to a search shortcut.

To be honest, I’m just glad that finally I have put this one to bed. From Delphi to .NET it has been many years sitting on floppy disks on the shelf but finally it is born to generate some more turnover that in the current environment has to be a good thing. It should also be a spring board to offering even richer information on the site!

Read about in part 1.

You have configured your Google Mini, got it integrated with you site. What you find now is that your results are getting skewed by irrelevant content on your site. This is what I’ve just found.

Exclude unwanted page sections

The result set was upset by the “customers who bought this also bought…” and the site page header and footer. This turned out very simple to resolve. There is a HTML tag that can be used to stop parts of the page from getting indexed. The definition of these are found in this document, excluding Unwanted Text from the Index.
Here are the examples pulled from that documentation for brevity;

<!--googleoff: anchor--><A href=sharks_rugby.html>shark </A> <!--googleon: anchor-->
<!--googleoff: snippet-->Come to the fair!<!--googleon: snippet-->
<!--googleoff: all-->Come to the fair!<!--googleon: all-->

You surround the control or section of the page you do not want to participate in the results with one of the three HTML comment tags shown above. This will not affect the rendering of you page but does mean something to the Google search appliance.

Index: The words between the tags are ignored by Google, they are treated as if they don’t occur on the page at all.

anchor: text in the html anchor tag to another page will not cause that destination page to appear as a result due to the link on this page.

Snippet: the search result will not use the text between the tags in the auto generated snippet that is included in the results.

all: Turns on all the attributes. Text between the tags is not indexed, followed to another linked-to page, or used for a snippet.

To solve my problem googleoff was applied to;

• “Customers who bought this bought” control reference
• Product category breadcrumb on the product pages
• master page header and footers

This has resulted in “contact us” not returning every page in the site any more, as it used to be linked from every page through the site master pages and made the snippets much more relevant from search results.

Resulting in much richer results. Caution should be applied to avoid excluding too much of your content from Google as you can’t predict what and why someone is searching on your site. Excluding too much content may hinder them finding what they require or prevent them ever getting what they need.

Check the documentation for other controls you have available to control the indexing of pages (the crawl).