This document provides answers to common questions regarding Enhanced vMotion Compatibility (EVC) and CPU compatibility.
What is EVC?
EVC is short for Enhanced vMotion Compatibility. EVC allows you to migrate virtual machines between different generations of CPUs.
What is the benefit of EVC?
Because EVC allows you to migrate virtual machines between different generations of CPUs, with EVC you can mix older and newer server generations in the same cluster and be able to migrate virtual machines with vMotion between these hosts. This makes adding new hardware into your existing infrastructure easier and helps extend the value of your existing hosts. With EVC, full cluster upgrades can be achieved with no virtual machine downtime whatsoever. As you add new hosts to the cluster, you can migrate your virtual machines to the new hosts and retire the older hosts.
How do I use EVC?
EVC is enabled for a cluster in the VirtualCenter or vCenter Server inventory. After it is enabled, EVC ensures that migration with vMotion is possible between any hosts in the cluster. Only hosts that preserve this property can be added to the cluster.
How does it work?
After EVC is enabled, all hosts in the cluster are configured to present the CPU features of a user-selected processor type to all virtual machines running in the cluster. This ensures CPU compatibility for vMotion even though the underlying hardware might be different from host to host. Identical CPU features are exposed to virtual machines regardless of which host they are running on, so that the virtual machines can migrate between any hosts in cluster.
Can migrations with vMotion still fail for virtual machines within an EVC cluster?
Yes. More factors than CPU compatibility are used to determine vMotion compatibility. For example, if the virtual machine does not reside on storage shared by the source and destination hosts, then migration with vMotion cannot occur between these hosts.
Does EVC allow AMD and Intel CPUs to be vMotion compatible?
No. An EVC-enabled cluster only allows CPUs from a single vendor in the cluster. VirtualCenter and vCenter Server do not allow you to add a host from a different vendor into an EVC-enabled cluster.
What EVC modes are available?
ESXi/ESX 3.5 Update 2 and later supports two EVC modes:
ESXi/ESX 4.0 supports these EVC modes:
To determine the EVC modes compatible with your CPU, search the VMware Compatibility Guide. Search for the server model or CPU family, and click the entry in the CPU Series column to display the compatible EVC modes.
What are the differences between the modes?
Each mode corresponds very closely to the features available in processors with the same name. Newer processors and their corresponding modes include additional features such as new instructions. For example, the Intel Xeon Core i7 processor added the SSE4.2 instruction set. These instructions are not available in earlier processors or their corresponding EVC modes.
AMD EVC modes expose these features:
How can I find the EVC mode supported by a host?
In the vSphere Client connected to vCenter Server 4.0, the host Summary tab indicates whether EVC is enabled, and displays the current EVC mode for the host. Click the blue icon next to the EVC mode to display a list of all the supported EVC modes for the host.
VMware also provides a free CPU identification utility that displays compatible EVC modes, in addition to other CPU features. You can download this utility and boot the host from the ISO image.
If I have an EVC-enabled cluster set to a particular EVC mode, and I have a host outside the cluster that matches the EVC mode, can I migrate a virtual machine with vMotion between that host and the cluster?
Yes, in both directions.
What is the difference between EVC and the old CPUID masking feature (accessed from the Virtual Machine Settings dialog box, Options tab, CPUID mask option)?
The older masking feature involved applying manual masks to individual virtual machines. EVC takes effect on a whole cluster and all virtual machines in the cluster. More accurately, EVC affects the hosts themselves, making all the hosts in the cluster appear to be the same type of CPU hardware, even if they are different.
Why is the . vmx config file for the virtual machine updated with CPUID bits?
The .vmx file contains information about the features that the virtual machine is using. VirtualCenter and vCenter Server need this information to provide accurate vMotion compatibility checks. These values are unrelated to the old CPUID mask values that also appear in the .vmx file.
What happens when a host is removed from an EVC-enabled cluster?
When a host leaves an EVC-enabled cluster, it reverts to its normal behavior. New virtual machines started on that host can access all the features of the CPU, and are not limited by the EVC mode that was in effect while the host was in the EVC cluster. Note that virtual machines that were once able to migrate to the host might no longer be permitted to do so.
If I raise the EVC mode, will virtual machines be able to access the new features available in that mode?
Not until you power off and then power on the virtual machines. A reboot of the guest operating system is not sufficient. A virtual machine determines which features are available to it at power on, and cannot access any new features that are added until it is powered off.
What is an ill-behaved application, and why does it affect EVC?
An ill-behaved application is one that does not use CPU-vendor-recommended methods of detecting features supported on a CPU. The recommended method is to run the CPUID instruction and look for the correct feature bits for the capabilities the application is expected to use. Unsupported methods used by ill-behaved applications include try-catch-fail or inferring the features present from the CPU version information. When unsupported methods are used, an application might detect features on a host in an EVC cluster that are being masked from the virtual machines. The CPUID-masking MSRs provided by CPU vendors do not disable the actual features. Therefore, an application can still use masked features. If a virtual machine running such an application is then migrated with vMotion to a host that does not physically support those features, the application might fail. VMware is not aware of any commercially-available ill-behaved applications. For more information, see Detecting and Using CPU Features in Applications (1005763).
Can I create an EVC-enabled cluster if some of the hosts I want to add do not have AMD-V Extended Migration or Intel VT FlexMigration?
Yes. EVC-enabled clusters can contain hosts that do not feature AMD-V Extended Migration or Intel VT FlexMigration technology. For example, Intel Xeon Core2 processors do not have Intel VT FlexMigration technology, but can be added to clusters configured with an Intel Xeon Core2 EVC mode.
Can I add an ESXi/ESX 3.5 Update 1 or earlier host to an EVC-enabled cluster?
No. EVC is supported only on ESXi/ESX 3.5 Update 2 and later. vCenter Server does not permit the addition of hosts that do not provide support for EVC into an EVC-enabled cluster.
If I add newer hardware into an EVC-enabled cluster with a lower EVC mode, do I lose performance?
All CPU features provided by your host hardware are available to the hypervisor. Optimizations for CPU virtualization such as AMD-V and Intel VT-x or facilities for MMU virtualization such as AMD RVI or Intel EPT support are still used by the hypervisor. Only those CPU instructions that are unique to the new CPU are hidden from virtual machines when the host joins the EVC-enabled cluster. Typically this includes new SIMD instructions, such as the latest SSE additions. It is possible, but unlikely, that an application running in a virtual machine would benefit from these features, and that the application performance would be lower as the result of using an EVC mode that does not include the features. Check with the application vendor to determine which CPU features are used by the application.
How do I know when I can raise the EVC mode for a cluster?
Source - VMware knowledge base
What is EVC?
EVC is short for Enhanced vMotion Compatibility. EVC allows you to migrate virtual machines between different generations of CPUs.
What is the benefit of EVC?
Because EVC allows you to migrate virtual machines between different generations of CPUs, with EVC you can mix older and newer server generations in the same cluster and be able to migrate virtual machines with vMotion between these hosts. This makes adding new hardware into your existing infrastructure easier and helps extend the value of your existing hosts. With EVC, full cluster upgrades can be achieved with no virtual machine downtime whatsoever. As you add new hosts to the cluster, you can migrate your virtual machines to the new hosts and retire the older hosts.
How do I use EVC?
EVC is enabled for a cluster in the VirtualCenter or vCenter Server inventory. After it is enabled, EVC ensures that migration with vMotion is possible between any hosts in the cluster. Only hosts that preserve this property can be added to the cluster.
How does it work?
After EVC is enabled, all hosts in the cluster are configured to present the CPU features of a user-selected processor type to all virtual machines running in the cluster. This ensures CPU compatibility for vMotion even though the underlying hardware might be different from host to host. Identical CPU features are exposed to virtual machines regardless of which host they are running on, so that the virtual machines can migrate between any hosts in cluster.
Can migrations with vMotion still fail for virtual machines within an EVC cluster?
Yes. More factors than CPU compatibility are used to determine vMotion compatibility. For example, if the virtual machine does not reside on storage shared by the source and destination hosts, then migration with vMotion cannot occur between these hosts.
Does EVC allow AMD and Intel CPUs to be vMotion compatible?
No. An EVC-enabled cluster only allows CPUs from a single vendor in the cluster. VirtualCenter and vCenter Server do not allow you to add a host from a different vendor into an EVC-enabled cluster.
What EVC modes are available?
ESXi/ESX 3.5 Update 2 and later supports two EVC modes:
- AMD Opteron Generation 1 (Rev. E)
- Intel "Merom" Generation (Intel Xeon Core 2)
ESXi/ESX 4.0 supports these EVC modes:
- AMD Opteron Generation 1 (Rev. E)
- AMD Opteron Generation 2 (Rev. F)
- AMD Opteron Generation 3 (Greyhound)
- Intel "Merom" Generation (Intel Xeon Core 2)
- Intel "Penryn" Generation (Intel Xeon 45nm Core2)
- Intel "Nehalem" Generation (Intel Xeon Core i7)
- AMD Opteron Generation 1 (Rev. E)
- AMD Opteron Generation 2 (Rev. F)
- AMD Opteron Generation 3 (Greyhound)
- Intel "Merom" Generation (Intel Xeon Core 2)
- Intel "Penryn" Generation (Intel Xeon 45nm Core2)
- Intel "Nehalem" Generation (Intel Xeon Core i7)
- Intel "Westmere" Generation (Intel Xeon 32nm Core i7)
- AMD Opteron Generation 1 (Rev. E)
- AMD Opteron Generation 2 (Rev. F)
- AMD Opteron Generation 3 (Greyhound)
- AMD Opteron Generation 3 (no 3DNow!) (Greyhound)
- Intel "Merom" Generation (Intel Xeon Core 2)
- Intel "Penryn" Generation (Intel Xeon 45nm Core2)
- Intel "Nehalem" Generation (Intel Xeon Core i7)
- Intel "Westmere" Generation (Intel Xeon 32nm Core i7)
- AMD Opteron Generation 1 (Rev. E)
- AMD Opteron Generation 2 (Rev. F)
- AMD Opteron Generation 3 (Greyhound)
- AMD Opteron Generation 3 (no 3Dnow!) (Greyhound)
- AMD Opteron Generation 4 (Bulldozer)
- Intel "Merom" Generation (Intel Xeon Core 2)
- Intel "Penryn" Generation (Intel Xeon 45nm Core2)
- Intel "Nehalem" Generation (Intel Xeon Core i7)
- Intel "Westmere" Generation (Intel Xeon 32nm Core i7)
- Intel "Sandy Bridge" Generation
- AMD Opteron Generation 1 (Rev. E)
- AMD Opteron Generation 2 (Rev. F)
- AMD Opteron Generation 3 (Greyhound)
- AMD Opteron Generation 3 (no 3Dnow!) (Greyhound)
- AMD Opteron Generation 4 (Bulldozer)
- AMD Opteron "Piledriver" Generation
- Intel "Merom" Generation (Intel Xeon Core 2)
- Intel "Penryn" Generation (Intel Xeon 45nm Core2)
- Intel "Nehalem" Generation (Intel Xeon Core i7)
- Intel "Westmere" Generation (Intel Xeon 32nm Core i7)
- Intel "Sandy Bridge" Generation
- Intel "Ivy Bridge" Generation
ESXi 5.5 supports these EVC modes:
Which CPUs are compatible with each EVC mode?- AMD Opteron Generation 1 (Rev. E)
- AMD Opteron Generation 2 (Rev. F)
- AMD Opteron Generation 3 (Greyhound)
- AMD Opteron Generation 3 (no 3Dnow!) (Greyhound)
- AMD Opteron Generation 4 (Bulldozer)
- AMD Opteron "Piledriver" Generation
- Intel "Merom" Generation (Intel Xeon Core 2)
- Intel "Penryn" Generation (Intel Xeon 45nm Core2)
- Intel "Nehalem" Generation (Intel Xeon Core i7)
- Intel "Westmere" Generation (Intel Xeon 32nm Core i7)
- Intel "Sandy Bridge" Generation
- Intel "Ivy Bridge" Generation
To determine the EVC modes compatible with your CPU, search the VMware Compatibility Guide. Search for the server model or CPU family, and click the entry in the CPU Series column to display the compatible EVC modes.
What are the differences between the modes?
Each mode corresponds very closely to the features available in processors with the same name. Newer processors and their corresponding modes include additional features such as new instructions. For example, the Intel Xeon Core i7 processor added the SSE4.2 instruction set. These instructions are not available in earlier processors or their corresponding EVC modes.
AMD EVC modes expose these features:
EVC Mode | Available Features |
AMD Opteron Generation 1 | All features of AMD Opteron Rev. E CPUs |
AMD Opteron Generation 2 | All features of AMD Opteron Generation and additional CPU features including CMPXCHG16B and RDTSCP |
AMD Opteron Generation 3 | All features of AMD Opteron Generation 2 and additional CPU features including SSE4A, MisAlignSSE, POPCOUNT, ABM (LZCNT) |
AMD Opteron Generation 3 (no 3Dnow!) | Applies
baseline feature set of AMD Opteron Generation 3 (Greyhound)
processors, with 3DNow! support removed, to all hosts in the cluster. This mode allows you to prepare clusters containing AMD hosts to accept AMD processors without 3DNow! support. |
AMD Opteron Generation 4 | Applies baseline feature set of AMD Opteron Generation 4 (Bulldozer) processors to all hosts in the cluster. This EVC mode exposes additional CPU features including SSSE3, SSE4.1, AES, PCLMULQDQ, XSAVE, AVX, XOP and FMA4. |
AMD Opteron "Piledriver" Generation | Applies baseline feature set of AMD Opteron "Piledriver" Generation processors to all hosts in the cluster. This EVC mode exposes additional CPU features including FMA3 (three operand FMA, aka Intel's FMA), BMI1 and TBM. |
Intel EVC modes expose these features:
EVC Mode | Available Features |
Intel "Merom" Generation (Intel Xeon Core2) | All features of Intel Core2 CPUs |
Intel "Penryn" Generation (Intel Xeon 45nm Core2) | All features of Intel Core2 CPUs and additional CPU features including SSE4.1 |
Intel "Nehalem" Generation (Intel Xeon Core i7) | All features of Intel Core2 CPUs and additional CPU features including SSE4.2 and POPCOUNT |
Intel "Westmere" Generation (Intel Xeon 32nm Core i7) | Applies
baseline feature set of Intel Xeon 32nm Corei7 (Westmere) processors to
all hosts in the cluster. Compared to the Intel Xeon Corei7 mode, this
EVC mode exposes additional CPU features including AES and PCLMULQDQ. Note: Intel i3/i5 Xeon Clarkdale Series processors that do not support AESNI and PCLMULQDQ cannot be admitted to EVC modes higher than the Intel Xeon Corei7 mode. Note: Intel® Atom™ C2300-C2700 processors support the Intel® "Westmere" Gen. EVC baseline although their architecture is different from the architecture of the Intel® "Westmere" Generation processors. |
Intel "Sandy Bridge" Generation | Applies baseline feature set of Intel “Sandy Bridge” Generation processors to all hosts in the cluster. This EVC mode exposes additional CPU features including AVX, XSAVE and ULE. Note: Some Sandy Bridge microarchitecture processors do not provide the full Sandy Bridge feature set. Such processors do not support EVC mode; they will only be admitted to the Intel Nehalem Generation mode or below. |
Intel "Ivy Bridge" Generation | Applies baseline feature set of Intel “Ivy Bridge” Generation processors to all hosts in the cluster. This EVC mode exposes additional CPU features including ENFSTRING, F16C, FSGSBASE, SMEP and CPUID Faulting. |
How can I find the EVC mode supported by a host?
In the vSphere Client connected to vCenter Server 4.0, the host Summary tab indicates whether EVC is enabled, and displays the current EVC mode for the host. Click the blue icon next to the EVC mode to display a list of all the supported EVC modes for the host.
VMware also provides a free CPU identification utility that displays compatible EVC modes, in addition to other CPU features. You can download this utility and boot the host from the ISO image.
If I have an EVC-enabled cluster set to a particular EVC mode, and I have a host outside the cluster that matches the EVC mode, can I migrate a virtual machine with vMotion between that host and the cluster?
Yes, in both directions.
What is the difference between EVC and the old CPUID masking feature (accessed from the Virtual Machine Settings dialog box, Options tab, CPUID mask option)?
The older masking feature involved applying manual masks to individual virtual machines. EVC takes effect on a whole cluster and all virtual machines in the cluster. More accurately, EVC affects the hosts themselves, making all the hosts in the cluster appear to be the same type of CPU hardware, even if they are different.
Why is the . vmx config file for the virtual machine updated with CPUID bits?
The .vmx file contains information about the features that the virtual machine is using. VirtualCenter and vCenter Server need this information to provide accurate vMotion compatibility checks. These values are unrelated to the old CPUID mask values that also appear in the .vmx file.
What happens when a host is removed from an EVC-enabled cluster?
When a host leaves an EVC-enabled cluster, it reverts to its normal behavior. New virtual machines started on that host can access all the features of the CPU, and are not limited by the EVC mode that was in effect while the host was in the EVC cluster. Note that virtual machines that were once able to migrate to the host might no longer be permitted to do so.
If I raise the EVC mode, will virtual machines be able to access the new features available in that mode?
Not until you power off and then power on the virtual machines. A reboot of the guest operating system is not sufficient. A virtual machine determines which features are available to it at power on, and cannot access any new features that are added until it is powered off.
What is an ill-behaved application, and why does it affect EVC?
An ill-behaved application is one that does not use CPU-vendor-recommended methods of detecting features supported on a CPU. The recommended method is to run the CPUID instruction and look for the correct feature bits for the capabilities the application is expected to use. Unsupported methods used by ill-behaved applications include try-catch-fail or inferring the features present from the CPU version information. When unsupported methods are used, an application might detect features on a host in an EVC cluster that are being masked from the virtual machines. The CPUID-masking MSRs provided by CPU vendors do not disable the actual features. Therefore, an application can still use masked features. If a virtual machine running such an application is then migrated with vMotion to a host that does not physically support those features, the application might fail. VMware is not aware of any commercially-available ill-behaved applications. For more information, see Detecting and Using CPU Features in Applications (1005763).
Can I create an EVC-enabled cluster if some of the hosts I want to add do not have AMD-V Extended Migration or Intel VT FlexMigration?
Yes. EVC-enabled clusters can contain hosts that do not feature AMD-V Extended Migration or Intel VT FlexMigration technology. For example, Intel Xeon Core2 processors do not have Intel VT FlexMigration technology, but can be added to clusters configured with an Intel Xeon Core2 EVC mode.
Can I add an ESXi/ESX 3.5 Update 1 or earlier host to an EVC-enabled cluster?
No. EVC is supported only on ESXi/ESX 3.5 Update 2 and later. vCenter Server does not permit the addition of hosts that do not provide support for EVC into an EVC-enabled cluster.
If I add newer hardware into an EVC-enabled cluster with a lower EVC mode, do I lose performance?
All CPU features provided by your host hardware are available to the hypervisor. Optimizations for CPU virtualization such as AMD-V and Intel VT-x or facilities for MMU virtualization such as AMD RVI or Intel EPT support are still used by the hypervisor. Only those CPU instructions that are unique to the new CPU are hidden from virtual machines when the host joins the EVC-enabled cluster. Typically this includes new SIMD instructions, such as the latest SSE additions. It is possible, but unlikely, that an application running in a virtual machine would benefit from these features, and that the application performance would be lower as the result of using an EVC mode that does not include the features. Check with the application vendor to determine which CPU features are used by the application.
How do I know when I can raise the EVC mode for a cluster?
You can raise the EVC mode for an EVC-enabled cluster if all hosts
in the EVC cluster support the higher mode. This may occur periodically
over time as older hosts are retired from the cluster and newer hosts
are added. Use the Change EVC Mode dialog box to determine the EVC modes
currently available to your cluster.
When disabling EVC mode on a cluster, it affects vSphere Cluster Features in the following ways:
- vSphere HA (High Availability): vSphere HA is not impacted due to the failover power-cycling the virtual machines when starting them on a a new host. This allows the virtual machine to pick up the new CPU ID and allows it to start without a problem
- vSphere DRS (Distributed Resource Scheduler):
- vMotion: Virtual machines is only able to move to other ESX/ESXi hosts that are at the same CPU generation or older. If DRS is configured for Fully Automated, this can result in virtual machines eventually being migrated to hosts in the cluster with the oldest CPU generation where they are unable to be moved via vMotion to different hosts with a newer CPU generation.
- Storage vMotion: Virtual machines are able to be moved to Storage vMotion with EVC mode disabled within a cluster
- Swapfile Location: This is not impacted.
Source - VMware knowledge base