Enabling client caching of dynamic content

In my last post, I was explaining how I wanted to return pictures from my ASP.NET MVC based application. In short, this is what I did:

[ReplaceMissingPicture(Picture = "~/Content/Images/nopicture.png", MimeType = "image/png")]
public ActionResult Picture(int id)
{
    var item = Service.GetItem(id);

    if (item.Picture != null)
    {
        var picture = item.Picture.ToArray();
        var mime = item.PictureMime;

        return File(picture, mime);
    }
    else
    {
        return null;
    }
}

Please see my last post for more information about what you see here.

This is all nice, but there’s a problem. Every time the client requests our “Picture” action method, the picture is returned with a HTTP status code 200. This is what you normally want, but my concern is that this happens every time. If this were a static file hosted by the web server, the web server would be smart enough to return an HTTP status code 304, meaning that the requested resource didn’t change since the last time it was requested.

How does this work? That’s relatively simple. When the resource is first requested, and the picture is returned, the web application should add a header to the HTTP response named “Etag”. This header contains a string that represents the “version” of the resource. This can really be anything.

When the client requests the same resource again, and the client has a cached copy of the resource, it sends a request containing an extra header named “If-None-Match”, meaning that if the resource still matches this version, it should return HTTP 304 and no content.

So how do we implement this?

For the version  information, I chose to place a timestamp column in the table containing the picture, and an associated property in the LINQ to SQL business object. The timestamp column type is not related to dates and times. It’s just a value that you can check for if you want to know if the record has changed in any way since you last read it. Which is perfect for this example.

So the first thing we need to do is send the version to the client. In order to keep this testable, I decided to implement it in a way that closely resembles this answer to my question on Stack Overflow. First of all, we need an interface that we can call from our Action method that deals with adding the tag and checking for it. This should do fine:

public interface ITagService
{
    string GetRequestTag();
    void SetResponseTag(string value);
}

When we add a property of this interface to our controller and initialize it in the constructor, we can decide what kind of implementation we use at runtime (whether that is the application hosted in IIS, or during testing). So we can use a mock for testing, and we can use the following class in IIS:

public class HttpTagService : ITagService
{
    public string GetRequestTag()
    {
        return HttpContext.Current.Request.Headers["If-None-Match"];
    }

    public void SetResponseTag(string value)
    {
        HttpContext.Current.Response.AppendHeader("ETag", value);
    }
}

So now our Picture action method looks like this:

[ReplaceMissingPicture(Picture = "~/Content/Images/nopicture.png", MimeType = "image/png")]
public ActionResult Picture(int id)
{
    var item = Service.GetItem(id);

    var responseTag = item.Version != null ? 
        Convert.ToBase64String(item.Version.ToArray()) :
        string.Empty;

    if (item.Picture != null)
    {
        var picture = item.Picture.ToArray();
        var mime = item.PictureMime;

        TagService.SetResponseTag(responseTag);

        return File(picture, mime);
    }
    else
    {
        return null;
    }
}

A simple ToBase64String() from our version column (which is mapped to a Binary object) should do just fine.

Before we go into how we’ll check for the tag (which is pretty trivial) we also need to know how to return the actual HTTP 304 status. The following needs to happen:

• Response.SuppressContent must be set to true.
• Response.StatusCode needs to be set to 304.
• Response.StatusDescription needs a description.
• Response needs a header “Content-Length” set to “0”.

This last one is important, as it will allow the client to keep the connection open for the next request while not expecting any more output from the current request.

In order to keep this testable, I chose to create a custom ActionResult called NotModifiedResult. Here’s the implementation:

public class NotModifiedResult : ActionResult
{
    public override void ExecuteResult(ControllerContext context)
    {
        var response = context.HttpContext.Response;

        response.SuppressContent = true;
        response.StatusCode = 304;
        response.StatusDescription = "Not Modified";
        response.AddHeader("Content-Length", "0");
    }
}

This is exactly what we need, and ASP.NET MVC makes it easy again to keep this all neat and tidy. Our tests can simply check for the type of result returned, and ExecuteResult() never needs to be executed in a test environment.

So finally, our Picture action method looks like this:

[ReplaceMissingPicture(Picture = "~/Content/Images/nopicture.png", MimeType = "image/png")]
public ActionResult Picture(int id)
{
    var item = Service.GetItem(id);

    var requestTag = TagService.GetRequestTag() ?? string.Empty;
    var responseTag = item.Version != null ? 
        Convert.ToBase64String(item.Version.ToArray()) :
        string.Empty;

    if (responseTag == requestTag)
    {
        return new NotModifiedResult();
    }

    if (item.Picture != null)
    {
        var picture = item.Picture.ToArray();
        var mime = item.PictureMime;

        TagService.SetResponseTag(responseTag);

        return File(picture, mime);
    }
    else
    {
        return null;
    }
}

I’m sure this doesn’t need any explanation.

Two more things I’d like to mention. First of all, I decided to turn the Picture column into a delay loaded column by setting its Delay Loaded property to true in the LINQ to SQL designer. This way, when the table is queried, the column is not selected by default. Only when you access the picture column a new query is started specifically for the picture. This way, when you don’t need the picture (like in a table overview of the business objects) it’s not queried for and retrieved from the database. Also, this means that our action method requires two queries: one for the Item and one for the Picture column. Except that in the case of a client cached picture, the second query doesn’t need to be executed.

Second, I want you to know that this technique here doesn’t just apply to pictures, it applies to any sort of content that can be accessed as a resource. If the client can cache it, you can enable the web application to support that caching method.

Now all I need to do, is apply the same to my ReplaceMissingPicture filter that I talked about last time.

About returning missing pictures.

While working on my pet project, I wanted to return pictures stored in the database. This is actually pretty much child’s play in ASP.NET MVC, and is well documented elsewhere.

Practically, it boils down to this:

public ActionResult Picture(int id)
{
    var item = Service.GetItem(id);

    if (item.Picture != null)
    {
        var picture = item.Picture.ToArray();
        var mime = item.PictureMime;

        return File(picture, mime);
    }
    else
    {
        return null;
    }
}

In this particular code, Service is a my data service returning an item (which is a business object in my application) and Picture is a Binary property, linked to a varbinary(MAX) column by LINQ to SQL.

All you need here is a byte array, and a mime type string, and calling File() will return a FileContentResult that takes care of everything else. Like I said, child’s play.

But there’s a couple of things missing. First of all, what happens if the picture can’t be found? What do we return? Do we throw an exception? Well, we certainly throw an exception in the Service when the item can’t be found, but nothing will point to this Action when the item doesn’t exist, so the only way the user would get this exception is by explicitly demanding this Action (and in that case, I don’t care if they get a 404).

But if the item exists, yet there is no picture, then what do we return? The sane thing to return is another picture, that visually says “Sorry, but there’s no picture.”. But returning it from the Action means we need to look it up, use Server.MapPath() to get to the file and return that instead. That means that we’ll need to call Server.MapPath() from inside the Action, and that’s bad for testability.

So what I decided to do was create an Action Filter, that would respond to null being returned from the Action method and replace the content with the missing picture. So here goes:

public class ReplaceMissingPictureAttribute : ActionFilterAttribute
{
    public string Picture { get; set; }
    public string MimeType { get; set; }

    public override void OnActionExecuted(ActionExecutedContext filterContext)
    {
        if (filterContext.Result is EmptyResult)
        {
            filterContext.Result = new FilePathResult(filterContext.RequestContext.HttpContext.Server.MapPath(Picture), MimeType);
        }
        base.OnActionExecuted(filterContext);
    }
}

And we apply it like so:

[ReplaceMissingPicture(Picture = "~/Content/Images/nopicture.png", MimeType = "image/png")]
public ActionResult Picture(int id)
{
    var item = Service.GetItem(id);

    if (item.Picture != null)
    {
        var picture = item.Picture.ToArray();
        var mime = item.PictureMime;

        return File(picture, mime);
    }
    else
    {
        return null;
    }
}

So we’re still pointing to the right content to return in case null is returned, but the filter takes care of looking up the right content instead of the Action. So when we test for this, we can simply test for null being returned.

Next time I’ll discuss how I managed to make this client cacheable.

HandleError and HTTP response codes.

After asking this question on Stack Overflow, I went out and used HandleError for my application exception handling. It works, as long as you turn on the customErrors configuration element. Then, after applying the HandleError attribute twice on my controller base class, I was able to get the exceptions NotFoundException and NoAccessException (my own, the names speak for themselves) turned into a nice rendering of the NotFound and NoAccess views.

Then I read up on HTTP status codes, and figured that HTTP code 500 is perhaps not the most appropriate for a situation in which an object with a certain Id cannot be found. In the future, for example, I might conclude that if the user wants to delete one of his objects, that object wouldn't be really deleted in the database but instead just flagged as such. In that case, my model should throw a DeletedException instead of a NotFoundException, so that I could inform the user that the object once existed but is no longer there. Also, I would want to return HTTP code 410 ("Gone") along with it, so that search engines can remove the entry from their indexes.

HandleError doesn't allow you to do that. But no worry, since I derived a new one from it, temporarily called MyHandleError (perhaps to be renamed HandleErrorWithStatus in the future) that would call the base OnException, and set the StatusCode in the Reponse after the correct View has been selected. Here’s the code to do it:

[SuppressMessage("Microsoft.Performance", "CA1813:AvoidUnsealedAttributes",
Justification = "This attribute is AllowMultiple = true and users might want to override behavior.")]
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = true, AllowMultiple = true)]
public class MyHandleErrorAttribute : HandleErrorAttribute
{
    private int statusCode = 500;

    public int StatusCode
    {
        get
        {
            return statusCode;
        }
        set
        {
            statusCode = value;
        }
    }

    public override void OnException(ExceptionContext filterContext)
    {
        if (filterContext == null)
        {
            throw new ArgumentNullException("filterContext");
        }

        // If custom errors are disabled, we need to let the normal ASP.NET exception handler
        // execute so that the user can see useful debugging information.
        if (filterContext.ExceptionHandled || !filterContext.HttpContext.IsCustomErrorEnabled)
        {
            return;
        }

        if (!ExceptionType.IsInstanceOfType(filterContext.Exception))
        {
            return;
        }

        if (new HttpException(null, filterContext.Exception).GetHttpCode() != 500)
        {
            return;
        }

        base.OnException(filterContext);

        filterContext.HttpContext.Response.StatusCode = StatusCode;
    }

}

Intel X25-M

I took the jump and installed one of these suckers into my XPS1330M. Something I should've done a long time ago. If your boot time takes longer than 10s, you should check the event log to see what kept it so long (like a missing UPEK FingerPrint Reader driver, for example)

Win7, ASP.NET MVC, IIS 7, SQL Server 2008

While setting up my development machine for the coming weeks, I needed to make a number of tough decisions. The first decision was whether to go with SQL Server 2005 or SQL Server 2008. I figured it was not a bad thing to go with the lastest version for new development. I remember there was an issue with Visual Studio 2008 integrating with SQL Server 2008, but that seemed to be fixed in Visual Studio 2008 SP1.

The next decision was whether to go with SQL Server Express or Standard. I chose the latter, simply because it's a different environment and if I'm going to test my app with a "real" server I might as well start to use it in development as well.

The final decision to make was whether to use the development web server included with VS2008 or go with IIS7. Again, I chose the latter for the aforementioned reasons: if I'm going to deploy to IIS7, I might as well debug it in IIS7 as well. That will tell me about the kind of problems I might run into before I go into "production".

Seems that a number of those problems already found me while trying to use the simplest ASP.NET MVC project possible: File>Create New. Compile. Run. Fail.

The problem was that it was impossible to register with the new website. ASP.NET MVC out of the box uses the ASP.NET MembershipProvider to offer signup/login functionality. Which is cool. But of course, it won't work out of the box when you're using the SQL Server 2008 Standard edition.

The first issue is that the Standard edition doesn't support attached databases. So creating a new database file (.mdf) won't work. That also means that no such database is ready when you're running for the first time. You'll have to make it yourself. That one is easy: ASP.NET comes with a nice tool that does this for you on any database of your choosing. I knew that! So I opened SQL Server Management Studio, created a new database (I called it "aspnetdb". Aptly. Very aptly.) and used the aspnet_regsql tool to create the required schema in there.

Then, I changed the connection string in Web.config to use this database. I kept the integrated security on (anyone still using "mixed mode security" should be dragged onto the street and shot). So then it worked. But only when using the development webserver. Because that uses my personal account to connect to SQL Server, and I happen to be database administrator (at least, that's what SQL Server tells me) so it can connect and do all of its stuff just fine.

When you use IIS 7, it uses the identity of the app pool of your web app to connect to SQL Server. That's also an easy guess. The problem is that in Windows 7, there's some weirdness about what user account is used exactly to that purpose. SQL Server will report a failed login for "IIS APPPOOL\DefaultAppPool". But I couldn't find such a user. While I was googling, I came across a number of sites advising me to switch it back to using Network Service for the DefaultAppPool, but I knew there had to be a better way. The weirdness ended when I took a leap of faith: I added a login named IIS APPPOOL\DefaultAppPool without checking the name. That worked.

After that, I made it dbo of the aspnetdb database (which is probably too much, but it'll do for now) and then everything went smoothly. This is exactly the sort of problems I was hoping to encounter before actually deploying. So my plan worked!

Here I found an excellect blogpost on how to make a bootable USB drive to install Windows 7 RC from. Excellent! Worked like a charm!

About Hyper-V, VPN, the next few weeks

In the coming months, I'll be at home enjoying my "vacation" and using it to learn some cool technologies, and hopefully I'll find some time to write interesting things about the process. One of the first things on my tasklist is setting up a number of VM's on a server running Hyper-V. I need a development server that will run the latest build of my project as well as the source repository. Another server will run the "release" build that will, hopefully, one day be public.
I already have a server, a nice Dell PowerEDGE with 4 cores humming happily with 12GB of RAM and a raid system. Nothing too fancy. On it, I have the Windows Server 2008 installed with the Hyper-V role.
One of the first issues I had was setting up SSTP correctly. I did this a couple of months ago, and it worked fine but now that I'm on Windows 7 I didn't quite remember all the hoops I had to jump through the first time around. Luckily there's the Interweb, and I found some info about what that issue might be.
The problem is of course that I'm a cheap bastard and that I'm using a self-signed certificate (aren't we all?) and that the client doesn't necessarily approve of this. I read this post, and this post as well. I figured out that it wasn't enough to import the server certificate into any old certificate store. No. It had to be the root store on the Computer account. And also, registering the certificate itself wasn't enough. It had to be the root certificate that issued the server certificate. So I had to open up the certificate, switch to the Certification Path tab, click on the topmost certificate, click "View Certificate", in there switch to the "Details" tab, and click Copy to File in order to export the signing certificate. Then, I had to import it into the Trusted Root Certification Authorities store of the Local Computer account. And then it worked.
Also note that for Windows 7 RC, there's an update called RTAS for Windows 7, that will enable you to install (yes, it won't install it, but the option will be there) the Hyper-V management console in Windows 7. After installing the update, use the "Turn Windows Features On or Off" in the Control Panel to install the Hyper-V maangement console. And use the excellent HVRemote to make it all work, of course.

Update: now that Windows 7 has gone RTM, the link to RTAS for Windows 7 is no longer valid. Here's the real deal.

MCPD EAD Certified!

This week I passed 70-549, concluding my series of exams to earn the MCPD Enterprise Application Developer certification. I would like to thank Brian to point me to the Transcender preparation exams. They are really a lot harder than the exams themselves, and the answers try to teach you something at the same time.

MCPD here we come!

Today I passed MCP exam 70-528, so I've earned the credential "MCTS .NET 2.0: Web Applications". This is one step closer to MCPD. While preparing for the exam, I used Transcender's study guide and preparation tests, and it really made a difference. For the last exam, 70-536, I used the ones included with the exam guide, and then bought into MeasureUP's preparation tests, but they were very frustrating. So if you're busy getting ready for certification, you should really check out those Transcender's tests.

The Trashcan

Some people, when starting a new project decide they need a "base library" of some sorts to put all the neat little tricks in that somehow were "left" out of the .NET Framework. They need some wrapper around some functions that do extra checking or they feel generic collections lack a couple of features or something like that.

First of all, I don't have anything against that, since I also think a nice function here and there can make the body of a lot of code more readable, more elegant. The truth is however, that this little base library quickly becomes the defacto container for everything that was deemed "unworthy of finding a true place".

There's a place for everything, and some of that stuff has some place elsewhere, but somehow the developer couldn't be bothered (or couldn't figure out) where to really put it. Perhaps because there wasn't enough time to think about it, or perhaps because that developer was lazy, considering that it might be a good idea to do this at "refactoring time" (which probably equals to "will never happen").

So this little library turns into a "trashcan" of stuff that doesn't belong there, stuff that has nothing to do with the other stuff. Sometimes that stuff might be shared among two different projects in the solution. Two is usually enough to decide that this thing needs to be "reusable". So after a while every single project in the solution has a reference to the trashcan, and every single project becomes really "dependent" on it. Without it, that project will not build. And then comes the time that someone wants to flex their "code reuse" muscles and decides that some part of the application is "reusable" within another app. But of course, the trashcan has to be shared as well, even though 90% of the trashcan is totally useless or even dangerous to that other application.

So fear the trashcan, people.