Friday, August 17, 2012

Moving Service Profiles can disconnect your Nexus 1000v!

I was just working on the vSphere 5.0 on FlexPod CVD, and found an issue where moving a Service Profile from a B series to a C series can cause your server to lose connection to your Nexus 1000v. Here's what happened:

I created a Service Profile, applied it to a C200-M2, and installed VMware ESXi 5.0 to a storage LUN. Nothing was stored on the local disk. I put all the networking through the Nexus 1000v, so neither of the network cards (P81E Cisco CNA nics) were connected to VMware vSwitches. Networking worked at this point. 

Then, one of my other C200-M2 servers died. I'm not sure why, but it wouldn't power up any more. So I decided to use one of the C200-M2 servers that were in my FlexPod. But all my other servers were B series blades, and I didn't want a lone C200-M2 running with a blade in HA mode. So that prompted me to move my Service Profiles from the C200-M2 servers to some B200-M3 servers I had. 

I suspended all the VMs on the FlexPod, shut down the ESXi servers, and disassociated the Service Profiles from the C200-M2's and associated them to the B200-M3's. When the B200-M3 servers came back up, they were disconnected from the network. I had to restore the networking from ESXi local console in order to recover them. 

I wondered why the Service Profiles didn't come back up properly, so I performed an experiment. I put one of the servers in Maintenance Mode. Then I disassociated the Service Profile from the B200-M3 and associated it with a C220-M3 I had. When I rebooted it, it didn't connect to the network. 
I then moved the Service Profile back to the B200-M3, migrated the management network to a local vSwitch (off of the Nexus 1000v dVS), and moved the Service Profile back to the C220-M3. This time, the management networking came up, but all the rest of the networking remained disconnected! 

I conclude from this that when migrating to/from the P81E and to/from the VIC 1240, the Nexus 1000v can tell the difference and won't allow the physical nics to connect. 

Next experiment is to determine whether the VMware vDS has the same behavior. 


  1. Great blog. All posts have something to learn. Your work is very good and i appreciate you and hopping for some more informative posts.

    illinois movers | chicago moving and storage

  2. Thanks, alansaki! I am surprised and gratified that someone reads this blog other than me. :-) I'll try to keep it useful and not marketing-oriented.