Home > VMware > VMware VMotion CPU problem after vSphere upgrade

VMware VMotion CPU problem after vSphere upgrade

September 15th, 2009 Leave a comment Go to comments

I wanted to install some DRAC 5 cards into each of my Dell PowerEdge 2900 VMware ESX hosts, so I decided to VMotion off the Virtual Machines onto other hosts in the cluster before shutting down the node. The first Virtual Machine migrated without a problem but when I tried a second I received an error stating the there was a CPU incompatibility between the host and the Virtual Machine.

Here is the error:

vmotioncpuidmaskerror.png


Host CPU is incompatible with the Virtual Machine's requirements at CPUID level 0x1 register 'ecx'.
Host bits: 0000:0000:0000:0000:0010:0010:0000:0001
Required: 1000:0000:0000:000x:xxx0:0x1x:xxx0:x001
Mismatch detected for these features:
*General incompatibilities; refer to KB article 1993 for possible solutions.

The only recent change was the upgrade to vSphere 4. The error detailed an incompatibility issue between the host and Virtual Machine CPU. The host hardware hadn't changed so I decided to investigate the Virtual Machine. After a quick look at the affected Virtual Machine's .vmx configuration file I noticed what appeared to be a CPU ID Mask. I checked the .vmx configuration against a couple of other Virtual Machines that were still able to VMotion and sure enough the CPU ID Mask was not present in these Virtual Machines.

The 5 lines in the non VMotion'ing Virtual Machine's .vmx file were:

cpuid.1.ecx = "R----R----R--R-0-----------H-R--"

cpuid.1.ecx.amd = "R---------------------------R---"
cpuid.80000001.ecx.amd = "------------------RR-RR---------"
cpuid.80000001.edx = "----R---------------------------"
cpuid.80000001.edx.amd = "--------------------------------"

I shut the Virtual Machine in question down and removed the 5 lines above from the .vmx file. I powered on the Virtual Machine and tried to VMotion it. I still received the same error as above. I shut down the Virtual Machine again, verified that the CPU ID Mask was not present in the .vmx and then powered it back on. After the second full shutdown I could successfully VMotion the Virtual Machine again.

NOTE: I have noticed that if you edit a .vmx file before shutting a Virtual Machine down it appears that the changes are overwritten because the running configuration is saved over the top of the existing configuration in the .vmx file itself. The easiest way to make a change to a .vmx is to shut the Virtual Machine down before editing it.

The CPU ID Mask can also be modified from the vSphere client by:

1) Shutdown the Virtual Machine
2) Right-click the Virtual Machine and select "Edit Settings..."
3) Select the "Options" tab
4) Click on "CPUID Mask"
5) Click on the "Advanced..." button

This issue affected 3 of 8 Virtual Machines in my 3 node vSphere cluster and I'm unsure why it only affected certain Virtual Machines. There seems to be nothing in common (that I could think of anyway) between the 3 affected Virtual Machines that I believe could have attributed to this problem.

EDIT (29/10/09): It seems VMware are now aware of the problem and have issued a workaround that requires you to use the vSphere client to remove the CPU ID mask. KB1011294

Categories: VMware Tags: , ,
  1. Bryan
    September 30th, 2009 at 18:30 | #1

    I normally don't post on sites when I'm looking for answers, but this was SUCH a headache and your instructions fixed the error. Now I can Vmotion and enable FT. Thank you SOO much!

  2. Bilby
    February 4th, 2010 at 05:14 | #2

    The upgraded virtual machines may have some CPU masks applied which are causing the migration difficulties.

    Note: This issue has been resolved in VMware vCenter Server 4.0 Update 1. For more information, see VMware vCenter Server Update 1 Release Notes.

    To ensure that VMotion is successful:
    Power down the virtual machine.
    Click the link to Edit Settings of the virtual machine.
    Click the Options tab.
    Select CPUID Mask under Advanced.
    Click Advanced.
    Click Reset All to Default.
    Click OK.
    Click OK again.
    Power on the virtual machine and migrate

  3. Bilby
    February 4th, 2010 at 05:15 | #3

    ^^This worked for me, even thou I had update 1 already

*

code