Monday 9 January 2012

SVN 1.7 New Feature Remote Dumps - svnrdump

I recently wanted to move a project from one svn repo to another, but I also wanted to keep the version history.

This can be done with the svnadmin commands but it can sometimes be tricky so my svn admin declined my request and said I should just check in the latest version!


SVN 1.7 to the rescue!

There is a new feature in SVN 1.7 called svnrdump. This is almost the same as svnadmin dump, although not quite.

The main difference is that you don't need to be an admin (or be on the server) to use it.

The other difference is a pain in the arse as we will see below, but basically svnrdump dumps in the diff format 3 that can't be read by svndumpfilter!

Apparently that was a design choice, but in my mind it was a bad one!


So how did I move the project?

In order to do this process you will need svn 1.7 and uberSVN (or install SVN normally in Apache) installed locally.

I used Cygwin as well to do the piping to svndumpfilter as I wasn’t sure how to do it in Windows.

This example assumes that I have one repo at https://svn.mycompany.net/repos/oldrepo that contains a project that I want to move to this repo https://svn.mycompany.net/repos/newrepo

It also assumes that you have installed svn locally and created a repo called test.

svnrdump dump https://svn.mycompany.net/repos/oldrepo > oldrepo.dmp
cat oldrepo.dmp | svnrdump load http://localhost/test
svnadmin dump http://localhost/test > dump.full
cat dump.full |svndumpfilter include trunk > dump.trunk
cat dump.trunk | svnrdump load https://svn.mycompany.net/repos/newrepo/trunk/clients/ --username myUsername
svn mv https://svn.mycompany.net/repos/newrepo/trunk/clients/trunk https://svn.mycompany.net/repos/newrepo/trunk/clients/oldproject --username myUsername -m "Rename imported trunk to oldproject"

Basically we use the new feature in 1.7 to remote dump the repo. This can take a long time.
We then load it in our local repo.
We then dump it using svnadmin from our local repo. The reason for this is that svnrdump doesn't provide a dump that can be filtered with svndumpfilter.
We then filter it to just include trunk (or you could just specify a certain directory).
We then import this filtered dump into the new repo.
This gave me a dir called trunk so I renamed it to oldproject.

In order for svnrdump load to work you need to have the “pre-revprop-change” pre-commit hook present (it can be empty).

You can do this locally in your own installation whilst testing by creating a file called pre-revprop-change.bat (or pre-revprop-change.sh if you are on Linux - don't forget to chmod +x it!) in your hooks directory, but I still needed my admins to do it for me on my company server. Fortunately they were obliging for this part.


Something to note that can happen if you are testing and the load is aborted.

You can get errors like this:
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
Failed to get lock on destination repos, currently held by 'GH-T624-09:e4464587-ae24-48be-9220-e4d0ecdba17a'
svnrdump: E000022: Couldn't get lock on destination repos after 10 attempts

To work out why you can't get a lock, you need to list all the properties on revision 0.
svn proplist --revprop -r0 http://192.168.56.128:9880/test
Unversioned properties on revision 0:
svn:date
svn:rdump-lock

Its fairly clear here what the lock is. So lets remove it!
svn propdel --revprop -r0 svn:rdump-lock http://192.168.56.128:9880/test
property 'svn:rdump-lock' deleted from repository revision 0

8 comments:

  1. Thank you! Your post just help me out - with regard to the svn:rdump-lock file...

    ReplyDelete
  2. Thanks a lot, this was a day saver !!!

    ReplyDelete
  3. Not here:
    1. svn proplist --revprop -r0 http://localhost/svn
    Unversioned properties on revision 0:
    svn:date
    svn:rdump-lock

    2.svn propdel --revprop -r0 svn:rdump-lock http://localhost/svn
    property 'svn:rdump-lock' deleted from repository revision 0

    BUT 3. svn proplist --revprop -r0 http://localhost/svn
    Unversioned properties on revision 0:
    svn:date
    svn:rdump-lock

    Any ideas??

    ReplyDelete
  4. @bloo:
    Your /path/to/repo/hooks/pre-revprop-change script must exist and return 0 (success)

    ReplyDelete
  5. Hi, while trying
    cat oldrepo.dmp | svnrdump load http://localhost/test
    it gives me an error

    svnrdump: E165001: Commit blocked by pre-commit hook (exit code 1) with output:
    The length of the 'log' message is less than 20 characters
    cat: write error: No space left on device.

    Looks like the dump file is with some hooks. How can I ignore while using svnrdump load command? Anyidea?

    ReplyDelete
  6. I haven't come across this before, so just guessing:
    Do you have any commit hooks on your new repo?
    Also, that cat error looks suspicious.
    How big is your dumped repo and is your local disk big enough?

    ReplyDelete
  7. Yeah Disk size is 150 GB free and dump is 20 GB.
    One thing let me mention here, I created a local repository on my system. Say on my C:\ drive as Temp_DIR
    But I am not able to use it with svnrdump as local IP, such as
    svnrdump load http://x.x.x.x/c$/Temp_DIR
    So, I created a "test" repository on my SVN Client only and using the dump command as
    cat oldrepo.dmp | svnrdump load http://subversion.mycompany.com/../test

    Can you please tell me what is correct step here?

    ReplyDelete
    Replies
    1. I think I just loaded dump locally using format
      file:///C:/Test_DIR which works as http for svrdump

      Delete