|
|||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
![]() Certifications ![]() Cisco ![]() IP ![]() PC ![]() Protocols ![]() RemoteAccess ![]() Resources ![]() Security ![]() Telecommunications ![]() Tools ![]() Unix ![]() Web ![]() mindterm |
FreeBSD Network PollingI've Intel PRO/1000 Gigabit Ethernet adapter installed in my server. How do I set network card (NIC) polling and disable interrupts under FreeBSD operating systems to boost network performance for 100M and 1000M network links?Device polling is a technique that lets the operating system periodically poll devices, instead of relying on the devices to generate interrupts when they need attention. This increase server network responsiveness and performance. In particular, polling reduces the overhead for context switches which is iincurred when servicing interrupts, and gives more control on the scheduling of the CPU between various tasks (user processes, software interrupts, device handling) which ultimately reduces the chances of livelock in the system. Make Sure Your Card Supports Network PollingDevice polling requires support in the device drivers. As of this writing FreeBSD 7.2 support only following NIC device drivers:
Refer above man pages to get information about polling and driver loading settings settings. Compile FreeBSD KernelTo enable polling in the driver, add the following options to the kernel configuration, and then recompile the kernel. The default kernel file name is GENERIC, which is located at the following location:
Open kernel config file: # cd /usr/src/sys/`uname -p`/conf # vi GENERICAdd the following for em driver (see above man pages for device specific polling settings) options DEVICE_POLLING options HZ=1000 It is strongly recommended to use HZ=1000 or 2000 with DEVICE_POLLING to achieve smoother behaviour. Save and close the file. Recompile FreeBSD kernel, enter: # make buildkernel KERNCONF=GENERIC # make installkernel KERNCONF=GENERIC Reboot the box and boot into new kernel: # reboot How Do I Enable Polling? At runtime use ifconfig command to turn polling on: # ifconfig device polling # ifconfig em0 polling How Do I Disable Polling? At runtime use ifconfig command to turn polling off: # ifconfig device -polling # ifconfig em0 -polling Make Configuration PersistenceUpdate /etc/rc.conf as follows: ifconfig_em0="inet 10.21.111.100 netmask 255.255.255.192 media 10baseT/UTP mediaopt full-duplex polling" Save and close the file. How Do I Verify Polling Settings?To verify settings enter: # ifconfig device # ifconfig em0 To see MIB related to polling enter: # sysctl -a kern.polling Sample Output: kern.polling.idlepoll_sleeping: 1 kern.polling.stalled: 0 kern.polling.suspect: 0 kern.polling.phase: 4 kern.polling.enable: 0 kern.polling.handlers: 2 kern.polling.residual_burst: 0 kern.polling.pending_polls: 0 kern.polling.lost_polls: 0 kern.polling.short_ticks: 0 kern.polling.reg_frac: 20 kern.polling.user_frac: 50 kern.polling.idle_poll: 0 kern.polling.each_burst: 5 kern.polling.burst_max: 150 kern.polling.burst: 150 Understanding MIB VariablesQuoting from the man page: kern.polling.user_frac
When polling is enabled, and provided that there is some work to
do, up to this percent of the CPU cycles is reserved to userland
tasks, the remaining fraction being available for polling pro-
cessing. Default is 50.
kern.polling.burst
Maximum number of packets grabbed from each network interface in
each timer tick. This number is dynamically adjusted by the ker-
nel, according to the programmed user_frac, burst_max, CPU speed,
and system load.
kern.polling.each_burst
The burst above is split into smaller chunks of this number of
packets, going round-robin among all interfaces registered for
polling. This prevents the case that a large burst from a single
interface can saturate the IP interrupt queue
(net.inet.ip.intr_queue_maxlen). Default is 5.
kern.polling.burst_max
Upper bound for kern.polling.burst. Note that when polling is
enabled, each interface can receive at most (HZ * burst_max)
packets per second unless there are spare CPU cycles available
for polling in the idle loop. This number should be tuned to
match the expected load (which can be quite high with GigE
cards). Default is 150 which is adequate for 100Mbit network and
HZ=1000.
kern.polling.idle_poll
Controls if polling is enabled in the idle loop. There are no
reasons (other than power saving or bugs in the scheduler's han-
dling of idle priority kernel threads) to disable this.
kern.polling.reg_frac
Controls how often (every reg_frac / HZ seconds) the status reg-
isters of the device are checked for error conditions and the
like. Increasing this value reduces the load on the bus, but
also delays the error detection. Default is 20.
kern.polling.handlers
How many active devices have registered for polling.
kern.polling.enable
Legacy MIB, that was used to enable or disable polling globally.
Currently if set to 1, polling is enabled on all capable inter-
faces. If set to 0, polling is disabled on all interfaces.
How Do I Verify Interrupt Usage Dropped Or Not?Run top command on both server and look for interrupt line in output: # top How Do I Test Network Throughput?Install iperf, enter: # portsnap fetch update # cd /usr/ports/benchmarks/iperf # make install clean && rehash After polling is enabled, use iperf tool for measuring the maximum data throughput rate of a communications link or network access. On server start iperf as follows: # iperf -s -B 10.21.111.100 On client type the same command as follows: # iperf -c 10.21.111.100 -d -t 60 -i 10Where,
Test ResultsAfter enabling polling you should see nice performance increase while large number of clients connects to server under heavy load. My testing result between two WAN links connected via 10Mbits/sec metro Ethernet network:
You can clearly see increased network throughput while RX CPU load went down while using network polling. Recommended readings:
This page was created in 0.00134 seconds
Comments and Questions
Last modified: March 03 2010. |
||||||||||||||||||||||||||||||||||||||||||||||||