FreeBSD* Driver for the 10 Gigabit Server Adapters

NOTE: This release includes two 10 gigabit FreeBSD Base Drivers for Intel® Network Connection. These drivers are named ixgbe and ixgb.  All 82598EB-based network connections require the ixgbe driver. All other network connections require the ixgb driver.  First identify your adapter.  Then follow the appropriate steps for building, installing, and configuring the appropriate driver.

ixgbe Driver

ixgb Driver

Identifying Your Adapter

First identify your adapter.  Then select the name of the specified base driver: ixgbe or ixgb

The following Intel network adapters are compatible with the drivers in this release:

Controller Adapter Name Physical Layer
82598EB Intel® 10 Gigabit XF SR Dual Port Server Adapter  
82598EB Intel® 10 Gigabit XF SR/LR Server Adapter  
82598EB Intel® 82598EB 10 Gigabit AF Dual Port Network Connection  
82598EB Intel® 82598EB 10 Gigabit AF Network Connection  
82598EB Intel®  82598EB 10 Gigabit AT CX4 Network Connection  
82597EX Intel® PRO/10GbE SR Server Adapter 10GBase-SR (850 nm optical fiber)
82597EX Intel® PRO/10GbE LR Server Adapter 10GBase-LR (1310 nm optical fiber)
82597EX Intel® PRO/10GbE CX4 Server Adapter 10GBase-CX4 (twin-axial copper cabling)

To verify that your Intel adapter is supported, find the board ID number on the adapter. Look for a label that has a barcode and a number in the format of 123456-001 (six digits hyphen three digits).  For more information on how to identify your adapter, go to the Adapter & Driver ID Guide at:

http://support.intel.com/support/go/network/adapter/home.htm

For the latest Intel network drivers for FreeBSD, see:

http://downloadcenter.intel.com/scripts-df-external/Support_Intel.aspx

Using Your ixgbe Driver

Building and Installation

Additional Configurations

Known Limitations

Support

The following Intel network adapters are compatible with the drivers in this release:

Controller Adapter Name Physical Layer
82598EB Intel® 10 Gigabit XF SR Dual Port Server Adapter  
82598EB Intel® 10 Gigabit XF SR/LR Server Adapter  
82598EB Intel® 82598EB 10 Gigabit AF Dual Port Network Connection  
82598EB Intel® 82598EB 10 Gigabit AF Network Connection  
82598EB Intel®  82598EB 10 Gigabit AT CX4 Network Connection  

To verify that your Intel adapter is supported, find the board ID number on the adapter. Look for a label that has a barcode and a number in the format of 123456-001 (six digits hyphen three digits).  For more information on how to identify your adapter, go to the Adapter & Driver ID Guide at:

http://support.intel.com/support/go/network/adapter/home.htm

For the latest Intel network drivers for FreeBSD, see:

http://downloadcenter.intel.com/scripts-df-external/Support_Intel.aspx

Building and Installation

NOTE: You must have kernel sources installed in order to compile the driver module.

In the instructions below, x.x.x is the driver version as indicated in the name of the driver tar file.

ixgbe instructions

  1. Move the base driver tar file to the directory of your choice. For example, use /home/username/ixgbe or /usr/local/src/ixgbe.
  2. Untar/unzip the archive:

        tar xfz ixgbe-x.x.x.tar.gz

  3. To install man page:

        cd ixgbe-x.x.x
        gzip -c ixgbe.4 > /usr/share/man/man4/ixgbee.4.gz

  4. To load the driver onto a running system:

        cd ixgbe-x.x.x/src
        make load

  5. To assign an IP address to the interface, enter the following:

        ifconfig ixgbe<interface_num> <IP_address>

  6. Verify that the interface works. Enter the following, where <IP_address> is the IP address for another machine on the same subnet as the interface that is being tested:

        ping <IP_address>

  7. If you want the driver to load automatically when the system is booted:

        cd ixgbe-x.x.x/src
        make
        make install

    Edit /boot/loader.conf, and add the following line:
        if_ixgbe_load="YES"

    OR

    compile the driver into the kernel (see item 8).

    Edit /etc/rc.conf, and create the appropriate ifconfig_ixgbe<interface_num> entry:

        ifconfig_ixgbe<interface_num>="<ifconfig_settings>"

    Example usage:

        ifconfig_ixgbe0="inet 192.168.10.1 netmask 255.255.255.0"

    NOTE: For assistance, see the ifconfig man page.

  8. If you want to compile the driver into the kernel, enter:

    FreeBSD 7.x or later:

        cd ixgbe-x.x.x/src

        cp *.[ch] /usr/src/sys/dev/ixgbe

        cp Makefile.kernel /usr/src/sys/modules/ixgbe/Makefile


    FreeBSD 6.x

    Old version ixgbe.
        cp *.[ch] /usr/src/sys/dev/ixgbe

    If you have an i386 platform:
    Edit the kernel configuration file (i.e., GENERIC or MYKERNEL) in  /usr/src/sys/i386/conf (replace "i386" with the appropriate system architecture if necessary), and ensure the following line is present:

        device ixgbe

    Compile and install the kernel. The system must be reboot for the kernel updates to take affect. For additional information on compiling the kernel, consult the FreeBSD operating system documentation.

Additional Configurations and Tuning

The driver supports Transmit/Receive Checksum Offload and Jumbo Frames on all 10GbE adapters.

Jumbo Frames

To enable Jumbo Frames, use the ifconfig utility to increase the MTU beyond 1500 bytes.

NOTES: Only enable Jumbo Frames if your network infrastructure, including switches, supports them.
The Jumbo Frames setting on the switch must be set to at least 22 bytes larger than that of the MTU. 

There are known performance issues with this driver when running UDP traffic with Jumbo Frames.

The Jumbo Frames MTU range for Intel Adapters is 1500 to 16114. The default MTU range is 1500. To modify the setting, enter the following:

ifconfig ixgb<interface_num> <hostname or IP address> mtu 9000

To confirm an interface's MTU value, use the ifconfig command. To confirm the MTU used between two specific devices, use:

route get <destination_IP_address>

VLANs

To create a new VLAN interface:

ifconfig <vlan_name> create

To associate the VLAN interface with a physical interface and assign an IP address, netmask, and VLAN ID:

ifconfig <vlan_name> <ip_address> netmask <subnet_mask> vlan <vlan_id> vlandev <physical_interface>

Example:

ifconfig vlan10 10.0.0.1 netmask 255.255.255.0 vlan 10 vlandev ixgb0

In this example, all packets will be marked on egress with  802.1Q VLAN tags, specifying a VLAN ID of 10.

To remove a VLAN interface:

ifconfig <vlan_name> destroy

Checksum Offload

Checksum offloading supports both TCP and UDP packets and is supported for both transmit and receive.

Checksum offloading can be enabled or disabled using ifconfig. Both transmit and receive offloading will be either enabled or disabled together. You cannot enable/disable one without the other.

To enable checksum offloading:

ifconfig <interface_num> rxcsum

To disable checksum offloading:

ifconfig <interface_num> -rxcsum

To confirm the current setting:

ifconfig <interface_num>

Look for the presence or absence of the following line:

options=3 <RXCSUM,TXCSUM>

See the ifconfig man page for further information.

Important System Configuration Changes for ixgbe Driver

The default scheduler SCHED_4BSD is not smart about SMP locality issues. Significant improvement can be achieved by switching to the ULE scheduler.

This is done by changing the entry in the config file from SCHED_4BSD to SCHED_ULE. Note that this is only advisable on FreeBSD 7, on 6.X there have
been stability problems with ULE.

Change the file /etc/sysctl.conf, add the line:

hw.intr_storm_threshold: 8000 (the default is 1000)

Best throughput results are seen with a large MTU; use 9000 if possible.

The default number of descriptors is 256, increasing this to 1024 or even 2048 may improve performance.

Known Limitations

Under small packets UDP stress test with the 10GbE driver, the FreeBSD system will drop UDP packets, due to socket buffers being full. You may want to change the driver's Flow Control variables to the minimum value for controlling packet reception.

Using Your ixgb Driver

Building and Installation

Additional Configurations

Known Limitations

Support

The following Intel network adapters are compatible with the drivers in this release:

Controller Adapter Name Physical Layer
82597EX Intel® PRO/10GbE SR Server Adapter 10GBase-SR (850 nm optical fiber)
82597EX Intel® PRO/10GbE LR Server Adapter 10GBase-LR (1310 nm optical fiber)
82597EX Intel® PRO/10GbE CX4 Server Adapter 10GBase-CX4 (twin-axial copper cabling)

To verify that your Intel adapter is supported, find the board ID number on the adapter. Look for a label that has a barcode and a number in the format of 123456-001 (six digits hyphen three digits).  For more information on how to identify your adapter, go to the Adapter & Driver ID Guide at:

http://support.intel.com/support/go/network/adapter/home.htm

For the latest Intel network drivers for FreeBSD, see:

http://downloadcenter.intel.com/scripts-df-external/Support_Intel.aspx

Building and Installation

NOTE: You must have kernel sources installed in order to compile the driver module.

In the instructions below, x.x.x is the driver version as indicated in the name of the driver tar file.

ixgb instructions

  1. Move the base driver tar file to the directory of your choice. For example, use /home/username/ixgb or /usr/local/src/ixgb.

  2. Untar/unzip the archive:

    tar xfz ixgb-x.x.x.tar.gz

    This will create an ixgb-x.x.x directory.
     

  3. To install man page:
    cd ixgb-x.x.x
    gzip -c ixgb.4 > /usr/share/man/man4/ixgb.4.gz
     

  4. To load the driver onto a running system, perform the following steps:

      cd ixgb-x.x.x/src
      make load
     

  5. To assign an IP address to the interface, enter the following:
      ifconfig ixgb<interface_num> <IP_address>
     

  6. Verify that the interface works. Enter the following, where <IP_address> is the IP address for another machine on the same subnet as the interface that is being tested:
      ping <IP_address>

  7. If you want the driver to load automatically when the system is booted:

    8. cd ixgb-x.x.x/src
    make
    make install

    Edit /boot/loader.conf, and add the following line:

    if_ixgb_load="YES"

    OR

    compile the driver into the kernel (see item 8).


    Edit /etc/rc.conf, and create the appropriate ifconfig_ixgb<interface_num> entry:

    ifconfig_ixgb<interface_num>="<ifconfig_settings>"

    Example usage:

    ifconfig_ixgb0="inet 192.168.10.1 netmask 255.255.255.0"

    NOTE: For assistance, see the ifconfig man page.
     

  8. If you want to compile the driver into the kernel, enter:

      cd ixgb-x.x.x/src
      mkdir -p /usr/src/sys/dev/ixgb
      cp if_ixgb* /usr/src/sys/dev/ixgb
      cp ixgb* /usr/src/sys/dev/ixgb
      mkdir -p /usr/src/sys/modules/ixgb
      cp Makefile.kernel /usr/src/sys/modules/ixgb/Makefile

    If you have an i386 platform: Edit the kernel configuration file (i.e., GENERIC or MYKERNEL) in /usr/src/sys/i386/conf (replace "i386" with the appropriate system architecture if necessary), and ensure the following line is present:

      device ixgb

    Compile and install the kernel. The system must be rebooted for the kernel updates to take effect.
    For additional information on compiling the kernel, consult the FreeBSD operating system documentation.

Additional Configurations and Tuning

The driver supports Transmit/Receive Checksum Offload and Jumbo Frames on all 10GbE adapters.

Jumbo Frames

To enable Jumbo Frames, use the ifconfig utility to increase the MTU beyond 1500 bytes.

NOTES: Only enable Jumbo Frames if your network infrastructure, including switches, supports them.
The Jumbo Frames setting on the switch must be set to at least 22 bytes larger than that of the MTU. 

There are known performance issues with this driver when running UDP traffic with Jumbo Frames.

The Jumbo Frames MTU range for Intel Adapters is 1500 to 16114. The default MTU range is 1500. To modify the setting, enter the following:

ifconfig ixgb<interface_num> <hostname or IP address> mtu 9000

To confirm an interface's MTU value, use the ifconfig command. To confirm the MTU used between two specific devices, use:

route get <destination_IP_address>

VLANs

To create a new VLAN interface:

ifconfig <vlan_name> create

To associate the VLAN interface with a physical interface and assign an IP address, netmask, and VLAN ID:

ifconfig <vlan_name> <ip_address> netmask <subnet_mask> vlan <vlan_id> vlandev <physical_interface>

Example:

ifconfig vlan10 10.0.0.1 netmask 255.255.255.0 vlan 10 vlandev ixgb0

In this example, all packets will be marked on egress with  802.1Q VLAN tags, specifying a VLAN ID of 10.

To remove a VLAN interface:

ifconfig <vlan_name> destroy

Polling

To enable polling in the driver, add the following options to the kernel configuration, and then recompile the kernel:

options DEVICE_POLLING
options HZ=1000

To turn on polling::

ifconfig <interface_num> polling

To turn off polling:

ifconfig <interface_num> -polling

NOTES: DEVICE POLLING is only valid for non-SMP kernels.

The driver has to be built into the kernel for DEVICE POLLING to be enabled in the driver.

Checksum Offload

Checksum offloading supports both TCP and UDP packets and is supported for both transmit and receive.

Checksum offloading can be enabled or disabled using ifconfig. Both transmit and receive offloading will be either enabled or disabled together. You cannot enable/disable one without the other.

To enable checksum offloading:

ifconfig <interface_num> rxcsum

To disable checksum offloading:

ifconfig <interface_num> -rxcsum

To confirm the current setting:

ifconfig <interface_num>

Look for the presence or absence of the following line:

options=3 <RXCSUM,TXCSUM>

See the ifconfig man page for further information.

Known Limitations

Under small packets UDP stress test with the 10GbE driver, the FreeBSD system will drop UDP packets, due to socket buffers being full. You may want to change the driver's Flow Control variables to the minimum value for controlling packet reception.

Support

For general information and support, go to the support website

If an issue is identified, support is through email only at: freebsdnic@mailbox.intel.com.

Last modified on 11/05/07 2:46p Revision 30