Subverison

A scary message to receive when you try to check a file out of subversion:

 svn: Checksum mismatch while reading representation:
    expected:  ad6cf2fede66a068b6d750d76be1f1ea
      actual:  289b6ed30dbabb94bf194c876b707a75

or:

 svnadmin: Checksum mismatch while reading representation:
    expected:  ad6cf2fede66a068b6d750d76be1f1ea
      actual:  289b6ed30dbabb94bf194c876b707a75

Oh, no! How am I going to recover this file? OK, if I recover it, how messed up is my repository and am I going to be able to recover the repository? These were the thoughts that went through my mind within the blink of an eye when I saw that message.

Well, it turns out that recovering the file was easy. I simply had to go to a checked out directory somewhere where that file was checked out. I had such a directory in theory, but that directory had that specific file removed from it in the working image. No worries, simply do an svn update in that directory and voila, the file was restored. But wasn't your repository corrupted, how could it restore that file? That's easy, an inherent feature of svn is that the file is cached locally, this means that you have a local backup of every checked out file!

So now on to the hard part, how do I fix my repository so that this file is no longer corrupt in it?

It turns out that svn has a really nifty tool called svnadmin which allows you to dump every single (uncorrupted) transaction that has been made from your current repository. This same tool allows you to then load any or all of these into another (potentially new) repository. You can even modify these transactions before reloading them to create a repository with a revised history if you wish to. This is all fairly well documented in the Subversion Manual, you might use svndumpfilter along the way.

So this meant that if I could dump all the transactions before the corrupted one and all the transactions after it, all I would need to do is recreate the corrupted transaction in order to recreate a completely new uncorrupted version of my repository! ...This is what I did.

So now you might ask, how do you go about recreating the corrupted transaction? Well this is the tricky/potentially really hard part. Even if you can't, it's nice to know that you've gotten this far, you could just rebuild the pieces that you can of the transaction and go from there.

One way to try and recreate the corrupted transaction is to try and tweak the actual repository until you can get svnadmin to dump the corrupted revision. I found references to others who had done this. The basic idea is to tweak the checksum in the corrupted repository so that it will not flag any errors (seems like an option to svnadmin to dump the transaction regardless of errors would be nice?) and thus perform the dump. Once dumped, you can potentially fix the transaction. The other references mention that you can use a zeroed out checksum as a wild card, but I found it simpler to actually just insert the actual checksum from my error message above.

In my case the data for one file was corrupted. Since I had a non-corrupted version of the file I used khexedit to splice in the new uncorrupted data. I was lucky: since the file length hadn't changed. If not, I would probably have had to tweak some of the content-length values also.

One tip to note: while cutting and pasting with khexedit, it makes life a lot easier if you use the bookmarks feature of this remarkable editor. This is especially needed to select large regions of bytes. Since binary files can often be much larger than text files, the range of stuff to select can be much larger than anything that you have ever had to select before, making traditional means of selecting things very difficult. When I started to select the file data (a jpg), I selected the bottom and began scrolling up while holding the mouse key down. I soon noticed that despite the obvious scrolling, the scroll bar did not seem to be moving! This was because the amount I was scrolling by was insignificant compared to the size of the file I was editing. I was not prepared to continue scrolling for an hour, so another method was needed: bookmarks! Simply bookmark up the end, and then go to the beginning. Start your selection with the shift key and press the end bookmark's shortcut (in my case alt-2) to highlight (select) the entire section, a life saver!

That's it, I now had a fixed transaction and could recreated my entire repository from the dumps! I was a happy camper, but I can't help but think that the svn tools could go one step further and provide a tool to operate on a single dumped transaction, maybe call it svndumpedit. Give me options to add in files, deletes and changes to a transaction. Hmm, this just gave me another idea, this could probably be done by creating a junk repository and performing the exact svn commands to it in one transaction that you need and then dumping that transaction to get a dump to use in your original repository. As you can see, with a little creativity there are options available to help you fix an svn repository.

One thing I got out of this was the lesson to dump all my transactions as a backup method. I had backups of the repository, but they were not old enough to have a version with the uncorrupted transaction in it.