Some days ago I reported on what appeared to be a problem with the Ada Web Server (AWS) project:
After a lot of testing, it now appears that this specific behavior has nothing to do with AWS, and everything to do with Apache Bench.
I grabbed a copy of the Apache Bench sources, and took a good look at them, and I think I actually managed to track down what looks like a classic race condition. When things are going really fast, Apache Bench can actually start more connections than requested by the -n parameter, and the data returned on these excessive connections are happily added to the accumulated totals by Apache Bench.
At this point it’s probably a good thing to note that my C skills are, at best, pretty bad, which is why I’ve send my speculations to the AWS maillist, where I’m sure some more experienced developers will let me know if I’m wrong or right.
What does support my findings, is that httperf does _not_ exhibit this behavior. Both I and Pascal have not been able to reproduce this problem using httperf.
So at this point, the finger of blame is pointing at Apache Bench.