One of the interesting new features in vSPhere is vStorage Thin Provisioning. I’m not going to explain what Thin Provisioning is all about. For that you can consult several knowledgeable blogs (for example this entry on Virtual Geek) and/or books (for example Scott Lowe’s excellent Mastering VMware vSphere 4.0).
The problem I have with Thin Provisioning, is that there are a lot of existing guests and templates out there that were created with Thick VMDKs in the past. The only documented way I could find to convert these Thick VMDKs to Thin VMDKs was to use svMotion from the vSphere client.
Now that is not a solution I want to (or can) use in an automated environment.I could of course ’emulate’ the svMotion process in a PowerShell script with the RelocateVM method, but that is in my opinion not really an ideal solution.
Luckily there is the VirtualDiskManager with some handy methods like the CopyVirtualDisk_Task method.
This method combined with the ReconfigVM_Task method ultimately gave me a working solution. With the following script I succeeded in converting a Thick VMDK into a Thin VMDK in place. My “in place” statement is of course relative, you have to have enough free space in the datastore to create a Thin version of the VMDK besides the Tick version of the VMDK. Once the reconfiguration is done you can safely remove the Thick VMDK file(s).
The guest has to be powered off for the following script to work !
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
$vmName = "guest-name" $vCenter = "vCenter-hostname" $esxAccount = "ESX-account" $esxPasswd = "ESX-password" $vcHost = Connect-VIServer -Server $vCenter $vmImpl = Get-VM $vmName if($vmImpl.PowerState -ne "PoweredOff"){ Write-Host "Guest must be powered off to use this script !" -ForegroundColor red exit } $vm = $vmImpl | Get-View $esxName = (Get-View $vm.Runtime.Host).Name # For Virtual Disk Manager we need to connect to the ESX server $esxHost = Connect-VIServer -Server $esxName -User $esxAccount -Password $esxPasswd $vDiskMgr = Get-View -Id (Get-View ServiceInstance).Content.VirtualDiskManager $firstHD = $true $vm.Config.Hardware.Device | where {$_.GetType().Name -eq "VirtualDisk"} | % { if(!$_.Backing.ThinProvisioned){ $dev = $_ $srcName = $dev.Backing.FileName $dstName = $srcName.Replace("/","/thin_") $esx = Get-View $vm.Runtime.Host $srcDC = Get-Datacenter | Get-View $spec = New-Object VMware.Vim.VirtualDiskSpec $controller = $vm.Config.Hardware.Device | where {$_.Key -eq $dev.ControllerKey} switch($controller.GetType().Name){ "VirtualBusLogicController" {$adapter = "busLogic"} "VirtualIDEController" {$adapter = "ide"} "VirtualLsiLogicController" {$adapter = "lsiLogic"} "VirtualLsiLogicSASController" {$adapter = "lsiLogic"} "ParaVirtualSCSIController" {$adapter = ""} "Default"{ Write-Host "Unknown controller type" exit } } $spec.adapterType = $adapter $spec.diskType = "thin" $taskMoRef = $vDiskMgr.CopyVirtualDisk_Task($srcName, $srcDC.MoRef, $dstName, $srcDC.MoRef, $spec, $false) $task = Get-View $taskMoRef while("running","queued" -contains $task.Info.State){ $task.UpdateViewData("Info") } if($firstHD){ $specHD = New-Object VMware.Vim.VirtualMachineConfigSpec $firstHD = $false } $deviceMod = New-Object VMware.Vim.VirtualDeviceConfigSpec $deviceMod.device = $dev $deviceMod.device.Backing.FileName = $dstName $deviceMod.Operation = "edit" $specHD.deviceChange += $deviceMod } } Disconnect-VIServer -Server $esxHost -Confirm:$false # For the reconfiguration of the VM we connect to the vCenter Connect-VIServer -Server $vCenter $taskMoRef = $vm.ReconfigVM_Task($specHD) $task = Get-View $taskMoRef while("running","queued" -contains $task.Info.State){ $task.UpdateViewData("Info") } |
Annotation:
Line 3-4: provide an ESX account and password with root access on the ESX server
Line 8-11: the guest must be powered off for this script to work
Line 16: the VirtualDiskManager methods can only be used when connected to an ESX
Line 20: this loops through all devices and finds the Thick VMDKs
Line 24: the backing filename of the Thin VMDK is the same as the backing filename of the Thick VMDK with a “thin_” prefix
Line 29-39: this Switch construct provides the correct adapter name. Take note that the adapter names are case-sensitive
Line 41: the diskType property allows to specify what VMDK format the copy should have
Line 43: the CopyVirtualDisk_Task method creates a Thin copy of the Thick VMDK
Line 49-59: this construct creates the specification for the ReconfigVM_Task method
Line 64: the new Thin VMDKs are now connected (as backing file(s)) to the VM
Note that the script doesn’t remove the Thick VMDK files from the VM’s directory. This is a safety precaution should something go wrong with the conversion.
Note that this procedure was tested and tried in my environment and seems to work correctly. If you are going to use this script, first try it out in your test environment and make sure the guests with the Thin VMDKs are working correctly !
Ruben
@antize
I have created the script that you were looking for. It was specifically designed for LabManager but can be used in any ESX/VM/vmware environment in general.
We achieved huge savings using it in all our centers worldwide.
The script is called makeThin
https://vmutils.blogspot.com/2011/06/automatic-thinning-of-virtual-disks.html
LucD
Hi Ruben, thanks for the link. Impressive script and documentation.
antize
Hi Luc,
This solution works for VMs that vcenter knows about, however is there any way via API to search through datastore directories looking for VMDKs that are thick?
We have a Lab Manager system that has a ton of folders created with VMs on the datastores but most of which are not deployed on vcenter. It would be great to recurse through the datastores directly looking for VMDKs that are thick provisioned via API. Is this possible?
LucD
@antize That is possible.
We make a function of the script and then use the Get-Datastore, Get-VM and Get-Harddisk cmdlets to find guests that have “Thick” hard disks.
For those guests we call the function, which I called Set-ThinDisk.
In the sample all datastores are examined, but you can use the -Name parameter on the Get-Datastore or a Where-Object construction to select specific datastores.
Let me know how it works out ?
function Set-ThinDisk{
param($vmName,$vCenter,$esxUser,$esxPassword)
$vcHost = Connect-VIServer -Server $vCenter
$vmImpl = Get-VM $vmName
if($vmImpl.PowerState -ne "PoweredOff"){
Write-Host "Guest must be powered off to use this script !" -ForegroundColor red
exit
}
$vm = $vmImpl | Get-View
$esxName = (Get-View $vm.Runtime.Host).Name
# For Virtual Disk Manager we need to connect to the ESX server
$esxHost = Connect-VIServer -Server $esxName -User $esxAccount -Password $esxPasswd
$vDiskMgr = Get-View -Id (Get-View ServiceInstance).Content.VirtualDiskManager
$firstHD = $true
$vm.Config.Hardware.Device | where {$_.GetType().Name -eq "VirtualDisk"} | % {
if(!$_.Backing.ThinProvisioned){
$dev = $_
$srcName = $dev.Backing.FileName
$dstName = $srcName.Replace("/","/thin_")
$esx = Get-View $vm.Runtime.Host
$srcDC = Get-Datacenter | Get-View
$spec = New-Object VMware.Vim.VirtualDiskSpec
$controller = $vm.Config.Hardware.Device | where {$_.Key -eq $dev.ControllerKey}
switch($controller.GetType().Name){
"VirtualBusLogicController" {$adapter = "busLogic"}
"VirtualIDEController" {$adapter = "ide"}
"VirtualLsiLogicController" {$adapter = "lsiLogic"}
"VirtualLsiLogicSASController" {$adapter = "lsiLogic"}
"ParaVirtualSCSIController" {$adapter = ""}
"Default"{
Write-Host "Unknown controller type"
exit
}
}
$spec.adapterType = $adapter
$spec.diskType = "thin"
$taskMoRef = $vDiskMgr.CopyVirtualDisk_Task($srcName, $srcDC.MoRef, $dstName, $srcDC.MoRef, $spec, $false)
$task = Get-View $taskMoRef
while("running","queued" -contains $task.Info.State){
$task.UpdateViewData("Info")
}
if($firstHD){
$specHD = New-Object VMware.Vim.VirtualMachineConfigSpec
$firstHD = $false
}
$deviceMod = New-Object VMware.Vim.VirtualDeviceConfigSpec
$deviceMod.device = $dev
$deviceMod.device.Backing.FileName = $dstName
$deviceMod.Operation = "edit"
$specHD.deviceChange += $deviceMod
}
}
Disconnect-VIServer -Server $esxHost -Confirm:$false
# For the reconfiguration of the VM we connect to the vCenter
Connect-VIServer -Server $vCenter
$taskMoRef = $vm.ReconfigVM_Task($specHD)
$task = Get-View $taskMoRef
while("running","queued" -contains $task.Info.State){
$task.UpdateViewData("Info")
}
}
$vmName = "guest-name"
$vCenter = "vCenter-hostname"
$esxAccount = "ESX-account"
$esxPasswd = "ESX-password"
$serverMode = (Get-PowerCLIConfiguration )."Default Server Mode"
Set-PowerCLIConfiguration -DefaultVIServerMode "Single"
foreach($ds in Get-Datastore){
foreach($vm in (Get-VM -Datastore $ds)){
if(Get-HardDisk -VM $vm | where{$_.StorageFormat -eq "Thick"}){
Set-ThinDisk $vm $vCenter $esxAccount $esxPasswd
}
}
}
Set-PowerCLIConfiguration -DefaultVIServerMode $serverMode
Alasdair
Hi Luc,
Did you ever have any luck with this? I’ve had a trawl through hte new version of the toolkit, but it doesn’t look like VMware have added a thin disk option.
Cheers….
Steve Ochry
@LucD
I was provided a workaround in C#. Essentially, the workaround is to create a propertyCollector and retrieve the properties on the httpNfcLeaseMor object. Looping through the properties, picking out the ones you need. Code pasted below in C#, for what it is worth.
private void RetrieveHttpNfcLeaseProperties(ManagedObjectReference httpNfcLeaseMor,
out HttpNfcLeaseInfo httpNfcLeaseInfo,
out HttpNfcLeaseState httpNfcLeaseState,
out int httpNfcLeaseInitializeProgress,
out LocalizedMethodFault httpNfcLeaseError)
{
httpNfcLeaseInfo = null;
httpNfcLeaseState = HttpNfcLeaseState.error;
httpNfcLeaseInitializeProgress = 0;
httpNfcLeaseError = null;
PropertyCollector propertyCollector =
new PropertyCollector(vimClient, vimClient.ServiceContent.PropertyCollector);
ObjectContent[] objectContent = propertyCollector.RetrieveProperties(
new[] {ViewBase.GetPropertyFilterSpec(httpNfcLeaseMor)});
if ((objectContent != null) && (objectContent.Length > 0))
{
foreach (DynamicProperty property in objectContent[0].PropSet)
{
switch (property.Name)
{
case “info”:
httpNfcLeaseInfo = (HttpNfcLeaseInfo) property.Val;
break;
case “state”:
string state;
if (property.Val is XmlNode[])
{
state = ((XmlNode[]) property.Val)[1].Value;
}
else
{
state = property.Val.ToString();
}
httpNfcLeaseState = (HttpNfcLeaseState)Enum.Parse(typeof(HttpNfcLeaseState), state);
break;
case “initializeProgress”:
httpNfcLeaseInitializeProgress = (int) property.Val;
break;
case “error”:
httpNfcLeaseError = (LocalizedMethodFault) property.Val;
break;
}
}
}
}
LucD
Thanks for sharing that Steve.
I’ll have a look if I can package that solution in a PowerShell script.
Steve Ochry
@LucD
I am bumping into this very issue on the C# side of things. Did you ever get anywhere with a work around?
LucD
Hi Steve, no I’m afraid I didn’t find a workaround for this yet 🙁
But it’s still on my list of things to investigate further.
Aladair Carnie
Thanks for the update.
Aladair Carnie
Hi Luc,
Any luck getting this to work?
Al…….
LucD
I encountered a problem while testing this. The HttpNfcLease MoRef the ImportVApp method returns has a problem.
One of the PowerCLI developers confirmed that there is a “…bug in xml deserialisation of NfcLeaseState object.”
I’m still looking if there is a way around the problem.
Aladair Carnie
This would be useful for restoring VMs without the need of third party systems or using the DR Appliance. The only issue I’ve had is the network adapter which defaults to a standard vmxnet adapter, but as long as it’s removed and replaced with a vmxnet3 card before powering it on (which can easily be scripted), the VM boots up and picks up it’s original IP info and works perfectly.
Aladair Carnie
Hi Luc,
Using the powershell toolkit, is there a way of importing an OVF and specifying the disk type as Thin? I’ve had a dig around in the kit and can only find the vApp cmdlet in the main toolkit. I assume that it will import an OVF, but there is no option for disk provisioning, however I did notive that the OVF Tool has those options.
Any guidence will as always be much appreciated.
Ali………
LucD
Very intriguing idea.
A quick look at the VirtualApp methods in the SDK makes me think it should be possible.
Let me play with the concept in my lab. Might even produce a new post when I get it to work.
Luc.