When evaluating storage options for a Microsoft Hyper-V guest machine, there are several options available these days. Solutions like iSCSI and Fibre Channel present block storage directly to virtual machines via Virtual Switches and Virtual SANs. While offering physical server-like performance, these solutions require significant hardware, infrastructure and the skill-sets to manage them. Two popular options that don’t require extravagant disk subsystems are pass-through disks and VHD/VHDX, however. Both offer the ability to attach the disk to the guest in the virtual machine settings, so there’s little configuration in the virtual machine itself. Let’s take a quick look at these two options.
Hyper-V Pass-Through Disks
For those not familiar, pass-through disks are disks present on the Hyper-V server that could either be local to the hypervisor or they could be LUNS mapped to the hypervisor via iSCSI or Fibre Channel. In this configuration, the disks are reserved on the hypervisor to enable exclusive access to the disk by the VM. This is done by initializing the pass-through disk in Disk Manager on the hypervisor, and then placing the disk in an Offline state.
You can see what this looks like in both diskpart and disk manager.
Microsoft DiskPart version 6.3.9600
Copyright (C) 1999-2013 Microsoft Corporation.
On computer: HV01
DISKPART> list disk
Disk ### Status Size Free Dyn Gpt
-------- ------------- ------- ------- --- ---
Disk 0 Reserved 3725 GB 0 B *
Disk 1 Reserved 3726 GB 0 B *
Disk 2 Reserved 3726 GB 0 B *
Disk 3 Reserved 3726 GB 0 B *
Disk 4 Offline 1863 GB 1024 KB
Disk 5 Online 55 GB 0 B
Disk 6 Online 894 GB 0 B *
Disk 7 Online 279 GB 0 B *
Disk 8 Reserved 3725 GB 0 B *
DISKPART> select disk 4
Disk 4 is now the selected disk.
DISKPART> detail disk
LIO-ORG IBLOCK SCSI Disk Device
Disk ID: 675CD493
Type : FIBRE
Status : Offline (Policy)
Path : 0
Target : 0
LUN ID : 4
Location Path : UNAVAILABLE
Current Read-only State : No
Read-only : No
Boot Disk : No
Pagefile Disk : No
Hibernation File Disk : No
Crashdump Disk : No
Clustered Disk : No
There are no volumes.
Virtual machine configuration for pass-through disks is straightforward as well. Once the disk is offlined in the hypervisor, simply open the settings for the virtual machine and click on the storage controller and add a hard disk. Select the radio button for, “Physical hard disk:” and choose the appropriate disk from the drop-down. Again, this MUST be an initialized disk, that has been placed Offline in Disk Management on the hypervisor.
Now that you’ve seen a bit about pass-through disks, let’s talk about the pros and cons of this type of storage.
- Performance. This is the most oft cited reason for using pass-through disks. Proponents like to talk about the advantage of not virtualizing the disk, and the near-physical performance of pass-through disks.
And, that’s about it. Performance is really the only reason you’ll hear for using pass-through disks. And while performance is a great reason to select a particular configuration, many experts would argue that the advantage of pass-through disks over VHDs even in older versions of Hyper-V (2008 and 2008R2) was small. Typically, numbers such as 15%-20% are thrown around. With the improvements in Hyper-V 2012 and 2012R2 and the VHDX format, this advantage shrinks. Use fixed rather than dynamic VHDXs, and the advantage shrinks further, to “virtually” nothing (pun intended).
- Not portable. Pass-through disks are not easily moved. Rather than being able to copy or migrate a virtual disk to new storage, a pass-through disk must be physically moved. Given the myriad of server and storage controller configurations, this is not typically an easy affair.
- Uses the entire disk. Since the whole disk is reserved for the virtual machine, no other virtual machines can use the disk.
- Not recommended for OS installations. OS installations can be problematic on pass-through disks since the VM configuration files must be located on another disk.
- No host-level backups. Backups must occur at the guest level rather than the host level, since the VM has exclusive access to the disk. As a result, backup and recovery becomes significantly more cumbersome.
- Difficult live-migrations. Live migrations require storage attached to a virtual machine to migrate along with the VM. Hyper-V clusters can be configured with pass-through disks, but it requires special considerations, and it’s not an optimal or recommended configuration.
- Cannot take snapshots. Snapshots are a super-handy tool and an important advantage of using virtual machines over physical servers. Losing this ability is a huge con.
- Cannot be dynamically expanded. Although dynamic disks are generally not recommended for production scenarios, they do have their use cases. Pass-through disks do not offer this functionality.
Clearly, there are numerous drawbacks to using pass-through disks. Now let’s take a look at the alternative in this discussion – VHD/VHDX. This is Microsoft’s implementation of virtual disks and is now the preferred method for storage in Hyper-V. Generally, there’s no reason to use VHDs over VHDXs with modern hypervisors and VMs (there are a few environment-specific reasons beyond the scope of this article to use VHD). VHDX supports much larger disks (64TB vs. 2TB) and is considerably more resilient to corruption, especially after a crash or power loss. VHDX also offers online resizing, allowing you to grow or shrink a virtual disk while the VM is running.
Looking at the pros and cons of VHDX, it’s basically the reverse of pass-through disks. Like pass-through disks, VHDXs can be stored on local disks on the hypervisor or SAN LUNs attached to the hypervisor. And given there’s at most a few percent advantage in performance of a pass-through disk over a fixed-size VHDX, it’s no wonder that Microsoft pushes VHDX as the preferred storage method for VM storage.
So far so good. Now this is where is one tiny wrinkle rears its head. If you’re following along and creating a VHDX while reading this, you’re likely still waiting for the previous step to complete, especially if the virtual disk is a large one, and you’re not using enterprise SSD storage. If you’re like us, and creating VHDXs on a small mirror or large, but relatively slow RAID arrays (i.e. SATA RAID-5 or RAID-6), this process can take a while. For something like a 4TB VHDX on slower disks, this can really take a while.
So we have to ask, “what’s happening during the VHDX creation that takes so dang long?” Well, as it turns out, during this process, the entire space required to store the VHDX is zeroed out on the disk. This is a conscious decision by Microsoft, as there are security implications in not doing so. As Hyper-V Program Manager Ben Armstrong explains here, VHDX creation could be nearly instantaneous. If the zeroing is not done, however, data may be recovered from the underlying disk(s). This would be a huge security no-no, of course, so Microsoft has no choice but to opt for safe route.
But what about for new disks on which data has never been stored? Clearly, there’s no security risk there. Many IT Pros would prefer the ability to decide for themselves whether or not a quick VHDX creation is appropriate. After all, we make decisions that have major security implications every day in our jobs. Microsoft has provided a tool to do just this for VHDs in the past. Unfortunately, Microsoft did not release such a tool for VHDX files.
What about an option within Hyper-V? Maybe a feature request for the next version? Not likely. As Mr. Armstrong notes, “the problem is that we would be providing a “do this in an insecure fashion if you know what you are doing checkbox” which would need a heck of a lot of text to try and explain to people why you do not want to do it – and then most people would not read the text anyway.”
Enter VhdxTool, from the good folks over at Systola. They’ve picked up where Microsoft has left off and provided a tool to create and resize VHDXs nearly instantaneously, even with multi-terabyte disks. They explicitly state this software is to be used at your own risk – it should only be used on new disks that contain no data, and not on disks that may contain data, especially when that VHDX may be accessed by end-users.
So how fast is it? We tested VhdxTool on four 4TB drives containing 4TB VHDXs, with pretty astounding results.
vhdxtool.exe create --FilePath C:\ClusterStorage\CSV_BU01\BU01.vhdx --DiskSize 3700GB --BlockSize 64 --LogicalSectorSize 4096
Successfully completed in 00:00:01.4481751.
vhdxtool.exe create --FilePath C:\ClusterStorage\CSV_BU02\BU02.vhdx --DiskSize 3700GB --BlockSize 64 --LogicalSectorSize 4096
Successfully completed in 00:00:01.4663183.
vhdxtool.exe create --FilePath C:\ClusterStorage\CSV_BU03\BU03.vhdx --DiskSize 3700GB --BlockSize 64 --LogicalSectorSize 4096
Successfully completed in 00:00:01.5300855.
vhdxtool.exe create --FilePath C:\ClusterStorage\CSV_BU04\BU04.vhdx --DiskSize 3700GB --BlockSize 64 --LogicalSectorSize 4096
Successfully completed in 00:00:01.1984083.
So in a little over one second, a 4TB VHDX was created to attach to our VM. Not bad. Additionally, there are also a number of command line options to accommodate any scenario. These are well documented on Systola’s site, but allow you to create, extend, convert, upgrade or view VHDXs.
Due to it’s many advantages, using VHDXs in place of pass-through disks is clearly the way forward in Hyper-V. If you’ve ever had reservations about using large VHDX files due to long creation times, Systola has provided an indispensable tool that gives IT Pros the option to fast-create VHDXs. Just remember, use good discretion when doing so and be sure to keep your data safe.