August 2011

We finally bit the bullet and procured a Cisco Catalyst 3750g 24 port gigabit layer 3 switch in order to consolidate and upgrade the networking hardware.
After basic configuration of the switch including VLANs and DHCP server, I shut down all VMs and proceeded to move all the wirings over.
Once satisfied I tried to connect to the management network of both hosts with no success. After repeated efforts of cisco speak I finally resorted to the internet (from my Forefront Threat Management server!). The first hit was Scott’s article about ESX Server, NIC Teaming and VLAN Trunking.
My error was not setting the load-balancing algorithm after creating a port channel that was compatible between the two ports. The command in a nutshell:

Cat3750g(config)#int port-channel1
Cat3750g(config-if)#description ESX-01 NIC Team
Cat3750g(config-if)#int gi0/1
Cat3750g(config-if)#channel-group 1 mode on
Cat3750g(config-if)#int gi0/2
Cat3750g(config-if)#channel-group 1 mode on
Cat3750g(config)#show etherchannel load-balance (this revealed “Source MAC Address” instead of “src-dst-ip”)
Cat3750g(config)#port-channel load-balance src-dst-ip (to enable IP-based load balancing)

Ensuring that all the NIC teams on the ESX servers were set to “Route based on IP hash” and I was off to the races.
Thanks to Scott Lowe for the resolution!