10G Ethernet: More Than a Big Pipe

10G Ethernet can deliver about 9 Gbit/s per second. We compared it with the typical link aggregated quad-port gigabit “NIC”. Quad-port link aggregation (IEEE 802.3 ad) is considered the “sweet spot”, both from a performance and an economic point of view. The results made 10G Ethernet really attractive: 10Gbit Ethernet actually consumed less CPU cycles than the quad-port link aggregated solution while delivering more than twice the bandwidth (9.5 Gbit/s vs 3.8 Gbit/s). It also features lower latency than quad-port NICs.
And 10G is hardly expensive: the least expensive dual 10G NICs cost about 50% ($600-700) more than the quad-port gigabit NICS ($400-450). Even the more expensive 10G cards (>$1000) offer a competitive bandwidth (2x 10G) per dollar ratio and offer a much better performance per watt ratio too. Typical power usage of dual 10G card is between 6W to 14W. The best quad gigabit NICs go as low as 4.3 W, although 8.4W is also possible.
10G Ethernet is much more than “a bigger pipe” for your (virtualized) network traffic. Let our knowledgeable IT professionals commenting on our last 10G Ethernet article enlighten you:
“In the market for a new SAN for a server in preparation for a consolidation/virtualization move, one of my RFP requirements was for 10GbE capabilities now. Some peers of mine have questioned this requirement stating there is enough bandwidth with etherchanneled 4Gb NICs and FC would be the better option if that is not enough. The biggest benefit for 10Gb is not bandwidth, it's port consolidation, thus reducing total cost.”
To understand this just look at the picture below.

A virtualized server might need I/O ports for:

• Console and management traffic (Ethernet)
• VM migration (Ethernet)
• VM Application network I/O (Ethernet)
• Block Storage I/O (Fibre Channel)
• File Storage I/O (Ethernet)

For speed and availability reasons, you quickly end up with two ports for each traffic flow, so you might end up with up to 10 ports coming out of one server. You might even need more: an IP based KVM to access the physical host and another port for server management interface (ILO, DRAC…).