YJL

This morning, I noticed the system temperature was oddly low, only just above 40°C. I didn’t mind at first, but then I felt it seemed to run slower than before, which might just from confusion after I checked the frequencies from /proc/cpuinfo.

I had not looked at file for years, and I saw the frequency was fixed at 1833MHz and 1000MHz, two cores, respectively. So, I tried the ultimate fix, turning it off and on again, didn’t work. I began to wonder if anything got updated recently, not the kernel nor any system/hardware stuff that I could remember.

At this point, I laughed before I knew there must be a setting wrong and for years, I had not realized that. So I went back to the power management in kernel configuration and found that I might have been using the wrong governor since 2012-08-27 as kernel 3.4 recommended ondemand governor according to ArchWiki.

That was 3.5 years ago.

Or it could be since 2011-05-18 with kernel 2.6.37, at the time, the userspace governor became the default over performance governor, and SpeedStep configurations were gone.

Nearly 5 years ago.

I knew the setting was correct at very beginning, had seen the frequencies bouncing in Conky, but since I started using my own dzen-status (2010-12-08), CPU frequency was never displayed. I rely on CPU utilization and the temperature to gauge, how hard CPU was working. I never thought about the frequency.

After it’s fixed, I could feel a boost in performance, fully 100% as needed, not only $(1000+1833)/(2*1833)=77.28\text{\%}$. Also, the frequency is ranging from 1000 to 1833 MHz, not fixed, anymore.

CPU0 was working super hard at full speed while CPU1 was licensed to slack.

First of all this isn’t meant to be accurate or very reliable, even flawed, I just want to see some numbers, because in pymux‘s README, it mentions about the performance, but not actual numbers:

Tmux is written in C, which is obviously faster than Python. This is noticeable when applications generate a lot of output. Where tmux is able to give fast real-time output for, for instance find / or yes, pymux will process the output slightly slower, and in this case render the output only a few times per second to the terminal. Usually, this should not be an issue. If it is, Pypy should provide a significant speedup.

Six months ago, I wrote Performance in shell and with C about converting a Bash project td.sh into a C. During it, I got some numbers, which showed me the performances of shell builtin and C. Almost two years ago, I wrote about sleep command, also talking about the cost of invoking external command.

Last night, suddenly I realized that I could have just given a very simple example by using true, using Bash’s time builtin to time the loops:

The builtin true is 23,484% faster than the external /bin/true, numbers got from looping the following commands for 1000 iterations, also as another example:

The e above is a shell builtin, from the e.bash project I forked off e, tiny expression evaluator. After I learned about the cost, I was no longer using bc just for simple calculation for floating numbers; but with e, I do calculate floating number again in Bash scripts.

These numbers should be very clear abort using external commands. If you have a long loop and there are lots of uses of external command, then you should really consider rewriting in other programming languages.

This June, I started to working on transitioning td.sh into a C project as well as using GNU Autotools to build. When I decided to do it, it didn’t come to my mind that I would have the following statistics to look at:

As you could see, the C loadable extension for Bash undoubtedly beats everyone else, 10+ times better in performance. Because of this, I decided to bring back vimps1, which was quietly disabled last August, when I couldn’t get it to work with latest vcprompt code at the time, which was linked into vimps1.

At the time, when I didn’t really have the way to measure the performance, but I knew it ran faster. How would I know? Well, that’s simple, just holding down Enter and see the rate of prompt showing up. You could feel it when you used loadable extension or just pure Bash PS1.

Because of td.sh, now I have just brought back vimps1, although it has a problem with Git repositories, you could still see the benefits:

It’s more than 10 times because bash_ps1 is a seriously stripped down version of my normal Bash PS1.

Not many people are aware how bad their shell or prompt is wasting, probably nobody else cares. However, I begin to think, maybe I need a shell that having performance in its feature list. Bash, although it’s not a bad shell, from time to time, I do hope it could run faster. It might be wrong to think so, since I haven’t really seen anyone really talk about shell performance.

After I finished td.sh’s C transition, I thought about making an extension that would do arithmetic with float number, but that’s just wrong to do so, because if you want to make Bash does float, you need to patch Bash or just use other programming language to write the whole code, I have seen a lot of people using bc. Frankly, they truly are doing it all wrong.

They don’t understand how costly of an invocation, just look at the first table, 52 vs 21631 runs, that’s the prize you have to pay. If you haven’t known, here is a tip for you, don’t use external command in Bash script unless it’s something Bash can’t do.

And if most of code is relying on external commands, maybe you should consider writing in a language could do all the tasks. Nevertheless, if that’s a “because I can,” then be my guest.

I want to see the performance between with and without C extension. I downloaded the latest 2.1.3 version. And make three builds using Python 2.6.6:

1. rm -rf build ; python setup.py build && rm build/lib*/simplejson/_speedups.so
2. rm -rf build ; python setup.py build
3. rm -rf build ; CFLAGS="-march=core2 -O2 -pipe -fomit-frame-pointer" python setup.py build

The first one removes the compiled C extension, the second one is normal, then third one uses the current CFLAGS I use in emerge. And the following lines is how C extensions got compiled:

Without customized CFLAGS
x86_64-pc-linux-gnu-gcc -pthread -fPIC -I/usr/include/python2.6 -c simplejson/_speedups.c -o build/temp.linux-x86_64-2.6/simplejson/_speedups.o
x86_64-pc-linux-gnu-gcc -pthread -shared build/temp.linux-x86_64-2.6/simplejson/_speedups.o -L/usr/lib64 -lpython2.6 -o build/lib.linux-x86_64-2.6/simplejson/_speedups.so

With customized CFLAGS
x86_64-pc-linux-gnu-gcc -pthread -march=core2 -O2 -pipe -fomit-frame-pointer -fPIC -I/usr/include/python2.6 -c simplejson/_speedups.c -o build/temp.linux-x86_64-2.6/simplejson/_speedups.o
x86_64-pc-linux-gnu-gcc -pthread -shared -march=core2 -O2 -pipe -fomit-frame-pointer build/temp.linux-x86_64-2.6/simplejson/_speedups.o -L/usr/lib64 -lpython2.6 -o build/lib.linux-x86_64-2.6/

I use the following code to do the test,

import timeit
t = timeit.Timer('json.loads(json_str)', 'import simplejson as json;json_str=open("test.json","r").read()')
print t.timeit(100)

The test.json is http://googleblog.blogspot.com/feeds/posts/default?alt=json&max-results=500, over 3 MB, 500 entires.

The results is:

With C extension is at least 10 times faster. I also put the simplejson 1.9 which in Python 2.6.6 in the result. Without extension, 1.9 -> 2.1.3, twice more slower. I didn’t download simplejson 1.9 to double check, but I don’t think it’s modified for being shipped with Python.

I was just curious how fast can a terminal window (in X) get a refresh draw. So I wrote a script to test, termfps.sh. It prints characters to fill up the whole window by default, then reset the cursor to home using ANSI escape code. Print again, doing so for 100 times by default. 100 / elapsed time is the FPS.

I ran several tests using ./termfps.sh 1000 80 25. I used 80 by 25 because that’s my VT’s terminal size, and I maximize window before I run the test. Here are the result for 80x25 and 1000 frames, sort by fps:

The slowest one is vt1, I didn’t test framebuffer. urxvt is my terminal, but I also have xterm installed. I installed lxterminal for vte-based terminal test. My normal urxvt uses Rxvt.font: xft:Envy Code R:style=Regular:size=9:antialias=false. I ran more on urxvtc, invoked without .Xdefaults, so I could test without changes I made. Since I use tmux, I tested tmux invoked with 256 colors or not, running in a panel.

This script can’t test the real FPS since Bash script takes some time to process, but the results aren’t really much lower and it does show the significant difference between terminals, or those FPS should all be capped around a same number. As you can see, urxvt runs fastest, then xterm, then lxterminal. Though there are some configuration differences, say the fonts, but it’s quite conclusive from the numbers I see.

For maximized urxvt+tmux terminal window I normally use in one screen with video played in another screen, here is the result:

For 239x65 100 frames, elapsed time: 18.567 seconds
Frames per second: 5.385