In case you missed the announcement on the kvm mailing , sheepdog is a new third party open source project around kvm providing distributed storage management features. Sheepdog provides high availability to kvm guests by providing block level storage volumes to virtual machines similar to Amazon EBS (Elastic Block Storage). In fact one of the items on the sheepdog project todo list is to support the Amazon EBS API. Sheepdog is designed to scale to hundreds of nodes. You can think of this technique as striping your virtual disk data across multiple nodes similar to what raid does. The project is still very early in its’ development cycle but already provides basic functionality.

Sheepdog Requirements

Before giving sheepdog a test run, there are some important bits of information that you need to know. Firstly, sheepdog currently requires the nodes to run btrs filesystems due to atomic write requirements. Secondly, due to the locking mechanism you won’t be able to perform live migration of your guests, at least not yet. One of the todo list items of the project is to remove this limitation. Additionally, the cluster management part of sheepdog requires some java libraries, namely Jgroups, commons-daemon, commons-logging, commons-cli and log4j1.2. Other software requirements for running sheepdog are sun JDK 6 and libssl. From the hardware side, you’d need at least 3 x86_64 machines so there’s some hardware requirements if you want to test sheepdog at home.

Architecture

The sheepdog architecture consists of three components: disk I/O manager, cluster manager and client (patched qemu/kvm block driver). The client side divides the VM image into fixed size objects of 4MB (by default) across storage nodes in a redundant fashion.

Sheepdog uses consistent hashing in determining which hosts to store objects on. Consistent hashing was first used in 1997 to minimize the impact of system failures in large web applications. It did this by distributing requests among a changing population of web servers. There’s a really good blog article on understanding consistent hashing by Tom Kleinpeter that can be found here. If you’re not familiar with consistent hashing you should give it a read.

Additional Resources

Original mailing list announcement and discussion
http://www.mail-archive.com/kvm@vger.kernel.org/msg24008.html

Instructions for installing sheepdog
http://www.osrg.net/sheepdog/usage.html

Documentation on architecture of sheepdog
http://www.osrg.net/sheepdog/design.html#architecture