30 x IOPS with Provisioning Services RAM cache

I’m currently working with a client to determine the potential scope of a virtual desktop deployment, and performing an infrastructure readiness assessment. While some additional compute power is required, this requirement is easy to fix and won’t break the budget. But the storage requirements are another story; not capacity, there is plenty of GB available, the problem is the IOPS. A XenApp published desktop (with anti-virus) can require 20 IOPS per user, so with 600 users 12,000 IOPS are required. With no budget for additional storage it seemed this project was doomed.

With a genuine “ah huh” moment, I decided to research the Citrix Provisioning Services (PVS) “Cache in device RAM with overflow on hard disk” vDisk cache option. We normally deploy PVS with the “Cache on device hard drive” option, but I had been meaning to test “Cache in device RAM with overflow on hard disk” for a while. Now was the time.

I was tasked with creating a proof of concept to show how Citrix Provisioning Services with “Cache in device RAM with overflow on hard disk” could provide the IOPS this project requires. A basic PVS 7.1 farm with two streaming servers was created. The existing XenApp 6 server template was deployed and the PVS 7.1 hotfix 002 target device software was installed. This hotfix is mandatory as a bug existed in the original PVS 7.1 target device software which rendered “Cache in device RAM with overflow on hard disk” useless. Soon we had a PVS vDisk ready to stream and the testing could begin.

Set PVS vDisk to Cache in device RAM with overflow on hard disk. Note that only 1 GB RAM was allocated for this test to make it easy to fill the RAM cache.

The PVS RAM cache is located in nonpaged memory. The vdiskdif.vhdx file is the disk overflow file for when the RAM cache is full. This image shows the VM after a reboot; the nonpaged memory allocation is only 120 MB and the vdiskdif.vhdx cache file is the minimum 4 MB.

A 650 MB file was copied into C:\ resulting in the RAM cache growing as indicated by the nonpaged memory growing to 762 MB. The vdiskdif.vhdx cache file is unchanged at 4 MB.

Another 650 MB file was copied to C:\. This time the RAM cache reaches maximum capacity and the vdiskdif.vhdx cache file grows to 624 MB.

Now let’s test IOPS IOMeter was configured to emulate a virtual desktop workload, that is, 90% writes, 80% random and 4K blocks. A 1GB workload was chosen. This is based off Jim Moyle (aka Mr IOPS) blog

Firstly we set the vDisk to Cache on device hard drive no RAM cache

And we get a whopping 1005 IOPS 🙂

Now we set the vDisk back to Cache in device RAM with overflow on hard disk

Whoa, now we’re cooking with 36,885 IOPS!!

And again, this time with a 2GB workload configured in IOMeter. Still a very respectable 24,913 IOPS. You can see that the disk cache file is 1.9 GB. When RAM cache is exhausted the least recently used block of data is written to disk which frees RAM and provides optimal performance – reference http://support.citrix.com/proddocs/topic/provisioning-7/pvs-technology-overview-write-cache-intro.html

No SANs were harmed during this testing; IOMeter tests were restricted to 30 second durations.

I believe the results speak for themselves. There are two final items to tick off before we move forward with this solution.

1. Measure the actual IOPS hitting the disk when the overflow to disk is occurring
2. Decide on the RAM cache size. Will 1 GB do the job, or do we need 5 GB per XenApp VM?

I’m impressed. If you’ve got the RAM or have budget for additional RAM, and your virtual desktop can be non-persistent VDI or XenApp hosted shared desktops – you should consider the Provisioning Services “Cache in device RAM with overflow on hard disk” vDisk cache option.