Buffer Bloat: Experiment 1

Ever since I first saw Jim Getty’s post on testing bufferbloat, I wanted in. But at the time I was living with room mates, and I didn’t feel like taking the router out of our network for testing that had already been done multiple times over.

Now that I’m on my own though, I have a chance to experiment on a clean network without disturbing anyone else’s network throughput. I expect his numbers will be remarkably similar to mine, simply because we both have very similar hardware (revisions of Linksys WRT54, I have the WRT54G).

Experiment 1a:

$ nttcp -t -D -n2048000 nfsserver & ping -n nfsserver 
[1] 5193 
PING nfsserver.lan ( 56(84) bytes of data. 
64 bytes from icmp_req=1 ttl=64 time=0.293 ms 
64 bytes from icmp_req=2 ttl=64 time=48.3 ms 
64 bytes from icmp_req=3 ttl=64 time=91.5 ms 
64 bytes from icmp_req=4 ttl=64 time=140 ms 
64 bytes from icmp_req=5 ttl=64 time=183 ms 
64 bytes from icmp_req=6 ttl=64 time=221 ms 
64 bytes from icmp_req=7 ttl=64 time=222 ms 
64 bytes from icmp_req=8 ttl=64 time=224 ms 
64 bytes from icmp_req=9 ttl=64 time=220 ms 
64 bytes from icmp_req=10 ttl=64 time=222 ms 
64 bytes from icmp_req=11 ttl=64 time=223 ms 
64 bytes from icmp_req=12 ttl=64 time=224 ms 

Experiment 1b:

For the rest of the experiment, I’m just going to use the format that Gettys uses in comment 843.

First tune down the tx ring buffer to 64 as he has done with ethtool -G eth0 tx 256. Then run across multiple txqueuelen values. These can be tuned with ifconfig. For example:

ifconfig eth0 txqueuelen 0 

Here are the numbers in tabular form:

txqueuelen tx-ring RTT 

1000 256 220ms (default - Experiment 1a) 

0 64 2ms 
2 64 3ms 
5 64 10ms 
10 64 45ms 
25 64 110ms 
50 64 190ms 
100 64 210ms 
1000 64 225ms 
10000 64 228ms 

Experiment 1b:

Experiment with a different tx-ring value. Set this with ethtool -G eth0 tx 4096:

txqueuelen tx-ring RTT 

0 4096 130ms 
2 4096 140ms 
5 4096 160ms 
10 4096 180ms 
25 4096 190ms 
50 4096 200ms 
100 4096 225ms 
1000 4096 230ms 
10000 4096 230ms 

Experiment 1d:

Don’t have a Windows or Mac system, and not intending to get either any time soon. By the way, for those that are curious:

$ uname -r 3.0.0-gentoo 


This hardware configuration seems to give very similar results to Gettys results. My results were a little better in general, with the same trend. Also for big buffers my performance seemed to suffer more .We’re both running with Intel cards, and a Linksys WRT-54 variant. If there’s any bit of difference, it’s that I’m using an ancient 3Com NIC in my server, but that didn’t seem to have a huge effect on the tests. Tomorrow I’ll continue with Experiments 2+3, but for now I want to try my new router 🙂



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: