Friday, August 17, 2012
Moving Service Profiles can disconnect your Nexus 1000v!
I created a Service Profile, applied it to a C200-M2, and installed VMware ESXi 5.0 to a storage LUN. Nothing was stored on the local disk. I put all the networking through the Nexus 1000v, so neither of the network cards (P81E Cisco CNA nics) were connected to VMware vSwitches. Networking worked at this point.
Then, one of my other C200-M2 servers died. I'm not sure why, but it wouldn't power up any more. So I decided to use one of the C200-M2 servers that were in my FlexPod. But all my other servers were B series blades, and I didn't want a lone C200-M2 running with a blade in HA mode. So that prompted me to move my Service Profiles from the C200-M2 servers to some B200-M3 servers I had.
I suspended all the VMs on the FlexPod, shut down the ESXi servers, and disassociated the Service Profiles from the C200-M2's and associated them to the B200-M3's. When the B200-M3 servers came back up, they were disconnected from the network. I had to restore the networking from ESXi local console in order to recover them.
I wondered why the Service Profiles didn't come back up properly, so I performed an experiment. I put one of the servers in Maintenance Mode. Then I disassociated the Service Profile from the B200-M3 and associated it with a C220-M3 I had. When I rebooted it, it didn't connect to the network.
I then moved the Service Profile back to the B200-M3, migrated the management network to a local vSwitch (off of the Nexus 1000v dVS), and moved the Service Profile back to the C220-M3. This time, the management networking came up, but all the rest of the networking remained disconnected!
I conclude from this that when migrating to/from the P81E and to/from the VIC 1240, the Nexus 1000v can tell the difference and won't allow the physical nics to connect.
Next experiment is to determine whether the VMware vDS has the same behavior.