The Party Line

Books giveaway

2009-03-24T10:24:07Z

For reasons which I am not going to delve into here (this is a topic for another post), we are going to get rid of about half of our books.

There are some (low) hundreds of books for the taking, slightly more than half in English, the rest being mostly Russian with a sprinkling of Danish here and there.

Fiction, non-fiction, textbooks, science fiction, you name it.

So, if you are in Copenhagen area and are interested, write me a note and consider coming over to have a look, maybe you’ll find something you’d like to keep. All books are to be had for free, although we would not mind selling them if you will insist.

Heart-attack date (mis)calculation

2009-03-18T10:52:41Z

Me and my wife are going to Paris in April. So, some time ago we have ordered tickets from a popular Danish travel site, rejsefeber.dk.

The tickets are for 2009-04-18.

This morning I have got a shiny (HTML) email from them which said the following (loosely translated from Danish):

Bon Vojage.

Hello Anton.

In a moment you will be traveling to Paris! There are often many things that should be taken care of before the journey, and we would like to help you with practical details. We hope that you will find in this mail something that will make your journey even better. Have a good journey!

After realizing that today is 2009-03-18, I had my moment of panic, frantically searching for the PDF with the electronic ticket, verifying that I have not made a major fuckup and that our tickets are indeed for April the 18th, not March the 18th.

So the fuckup is not mine. Fine. But it would be nice to be absolutely really positively sure, so I called their customer service. The robot helpfully told me that

this is a paid call (I don’t remember the exact amount per minute, but it was not peanuts);
I am number 10 in the queue.

And there is no contact E-mail on the website.

So what do you think - is it a simple, albeit embarrasing programming error on behalf of rejsefeber.dk programmers, or a secret plot to get some more money out of customers induced into a state of panic?

At any rate, I do not think I will be using their services again. If I need some excitement in my life, there are better ways to obtain it.

rrdtool legend box rant

2009-03-05T13:21:41Z

Last time I ranted about one thing that is wrong with rrdtool.

It turns out that I, unfortunately, have more gripes to share. Let’s talk today about the legend control.

The rrdtool’s language for controlling how the graph is going to look and feel is pretty rich and complex. Among other things, it has a bunch of graphing commands that are able to manipulate the legend section of the graph. Some of those commands do nothing else, while the other commands only modify the legend as a side effect, and primarily exist for doing something else.

Let’s take as an example the graph below:

rrdtool graph x.png
  --end 1235800000 --start end-150000
  'DEF:data=some.rrd:data:AVERAGE'
  'CDEF:sine=data,POP,1235800000,TIME,-,10000,/,SIN'
  'LINE1:sine#0000ff:a sine of foo\l'
  'CDEF:cosine=data,POP,1235800000,TIME,-,5000,/,COS'
  'LINE1:cosine#00ff00:a cosine of bar'

The trouble is that there is no instruction which can produce a so called “legend box” (a small blue box next to the “a sine of foo” legend text) without doing something else in the process. All of the commands that actually graph something can optionally produce a legend box, but there is no other way to make it.

If we extend our sine example with some nice downtime background (in Soviet Russia sines can have a downtime!), we get a nice picture like this:

rrdtool graph x.png
  --end 1235800000 --start end-150000
  'DEF:data=some.rrd:data:AVERAGE'
  'CDEF:downtime=data,0,0,IF,
    TIME,1235769489,GE,TIME,1235779489,LE,*,
    TIME,1235689489,GE,TIME,1235729489,LE,*,+,+'
  'CDEF:sine=data,POP,1235800000,TIME,-,10000,/,SIN'
  'LINE1:sine#0000ff:a sine of foo\l'
  'CDEF:cosine=data,POP,1235800000,TIME,-,5000,/,COS'
  'LINE1:cosine#00ff00:a cosine of bar\l'
  'TICK:downtime#ffe0e0:1.0:downtime'

But let’s suppose that we’d also like to add background coloring for uptime, and indicate that in a legend. Let’s use another CDEF for uptime, which will be just the opposite of our downtime CDEF:

rrdtool graph x.png
  --end 1235800000 --start end-150000
  'DEF:data=some.rrd:data:AVERAGE'
  'CDEF:downtime=data,0,0,IF,
    TIME,1235769489,GE,TIME,1235779489,LE,*,
    TIME,1235689489,GE,TIME,1235729489,LE,*,+,+'
  'CDEF:uptime=downtime,0,1,IF'
  'CDEF:sine=data,POP,1235800000,TIME,-,10000,/,SIN'
  'LINE1:sine#0000ff:a sine of foo\l'
  'CDEF:cosine=data,POP,1235800000,TIME,-,5000,/,COS'
  'LINE1:cosine#00ff00:a cosine of bar\l'
  'TICK:uptime#e0ffe0:1.0:uptime'
  'TEXTALIGN:left'
  'TICK:downtime#ffe0e0:1.0:downtime'

Oops. Of course. The graphing commands are processed in sequence, so the uptime “background” has obscured our actual sinusoidal data! We will have to put the uptime TICK before anything else to avoid this:

rrdtool graph x.png
  --end 1235800000 --start end-150000
  'DEF:data=some.rrd:data:AVERAGE'
  'CDEF:downtime=data,0,0,IF,
    TIME,1235769489,GE,TIME,1235779489,LE,*,
    TIME,1235689489,GE,TIME,1235729489,LE,*,+,+'
  'CDEF:uptime=downtime,0,1,IF'
  'CDEF:sine=data,POP,1235800000,TIME,-,10000,/,SIN'
  'TICK:uptime#e0ffe0:1.0:uptime'
  'LINE1:sine#0000ff:a sine of foo\l'
  'CDEF:cosine=data,POP,1235800000,TIME,-,5000,/,COS'
  'LINE1:cosine#00ff00:a cosine of bar\l'
  'TICK:downtime#ffe0e0:1.0:downtime'

See the problem? Now the legend boxes are not in the right order.

Naturally, it is pretty trivial to solve - we just remove the legend (it is optional) from the uptime TICK, and add another graphing command in the end with the right legend text. We’ll simply have to make sure that this “graphing command” does not actually graph anything:

rrdtool graph x.png
  --end 1235800000 --start end-150000
  'DEF:data=some.rrd:data:AVERAGE'
  'CDEF:downtime=data,0,0,IF,
    TIME,1235769489,GE,TIME,1235779489,LE,*,
    TIME,1235689489,GE,TIME,1235729489,LE,*,+,+'
  'CDEF:uptime=downtime,0,1,IF'
  'CDEF:sine=data,POP,1235800000,TIME,-,10000,/,SIN'
  'TICK:uptime#e0ffe0:1.0'
  'LINE1:sine#0000ff:a sine of foo\l'
  'CDEF:cosine=data,POP,1235800000,TIME,-,5000,/,COS'
  'LINE1:cosine#00ff00:a cosine of bar\l'
  'TICK:uptime#e0ffe0:0.0:uptime'
  'TEXTALIGN:left'
  'TICK:downtime#ffe0e0:1.0:downtime'

Just like the last time - the problem is easily solvable, but it annoys me, since rrdtool again forces us to work against it instead of working with us. It would be nicer to have a way to put a legend box where we want it in some more direct way (maybe a variant of the COMMENT command)?

rrdtool CDEF restrictions rant

2009-03-03T19:40:06Z

Last time I spread some rrdtool love around. Love with no hate won’t do, however. Taking a page from the way the BBC and many other news outlets cover certain conflicts, it seems prudent to present an unbiased view of rrdtool. And what better way to do that if not by spreading some hate this time?

Some time ago I had a task of adding background coloring to the portions of a graph, representing downtime of a device. This information was not available in the RRD archive itself, but could be obtained from elsewhere.

The information was in the form of epoch ranges, like this:

1235639489-1235646689,1235725889-1235729489

After some thought, I decided that making a CDEF with some simple RPN magic, in combination with a TICK graphing command would do the trick. The TICK command draws a vertical line of a specified height when a data point is not a zero or an UNKNOWN value, so I naïvely thought that this simple CDEF will achieve what I want (one long line here is split into several lines for your browsing convenience):

CDEF:downtime=0,TIME,1235639489,GE,
    TIME,1235646689,LE,*,
    TIME,1235725889,GE,
    TIME,1235729489,LE,*,+,+

The boolean values in rddtool’s RPN are represented using the usual C-like convention (0 = false, 1 = true), so for any given time point the result of the CDEF above will be 0 if the point is outside any of the downtime intervals, 1 if it is inside one of the intervals, and more than 1 if it is inside several intervals (which should not happen, but is possible if we specify overlapping downtime epoch ranges).

If you did not understand how this is supposed to work, go read the rrdgraph_rpn manpage.

Anyway, that was easy enough, even if it was not very clean. So where is the promised hate, you’ll ask? Here it is:

rrdtool graph x.png --end 1235800000 --start end-150000
    'DEF:data=some.rrd:someds:AVERAGE'
    'CDEF:downtime=0,TIME,1235639489,GE,TIME,1235646689,LE,*,
     TIME,1235725889,GE,TIME,1235729489,LE,*,+,+'
    'TICK:downtime#ffe0e0:1.0'

ERROR: rpn expressions without DEF or CDEF variables are not supported

You’ve read that right! Rrdtool insists that everything you put into a graph must ultimately come from some data source in some RRD archive. How stupid is that?

And the answer is - very stupid indeed, for at least two reasons.

First, it tries to pretend it knows what I might want to do better than me, and I really hate that in software. Whatever has happened to the idea that a tool must not get in the way of its user?

Secondly, this check is easily circumvented by adding a real DEF somewhere in the CDEF and then (mis)using RPN language to get rid of any value which comes from the DEF. There are many ways to do it, some more creative than others, but even this ham-fisted “solution” will do the job:

CDEF:downtime=data,0,0,IF,TIME,1235639489,GE,
    TIME,1235646689,LE,*,
    TIME,1235725889,GE,
    TIME,1235729489,LE,*,+,+

So we have a pointless restriction, and we are forced to work around it. Some might say that working against the tool to reach the goal is somehow challenging and maybe even fun. If you ask me, this is akin to programming in Brainfuck - a kind of activity better performed as a hobby, if that’s your idea of how to spend your copious free time.

To rephrase the Perl’s motto, rrdtool makes simple things hard.

Some rrdtool love

2009-03-01T14:16:50Z

Everybody uses rrdtool. Everybody loves rrdtool.

I love rrdtool as well. Sometimes it even pleasantly surprizes me.

For example, it used to be the case that massive updates of a large number of RRD archives were a bit on the slow side (this operation is IO-bound, you see).

Quite unexpectedly I found that this inconvenience can be easily circumvented by throwing more RAM into the problem. By itself, this statement is not at all suprizing - after all, we all know about effects of disk caching. It is the magnitude of the improvement that simply astonished me: on a moderately busy box with primary purpose entirely different from doing RRD updates, I am now routinely getting around 18500 RRD acrhives updated in under 1 second. The large majority of the archives has 40+ data sources each, and the rest has 140+ data sources each. The machine has measly 8G of RAM.

Sometimes, when the box is busy doing something with its disks, it might take 3 minutes to finish the updates. In very rare instances it can take up to 15 minutes. But in more than 80% of cases it takes under 1 second.

The setup is such that the actual data collection is completely separated from the RRD update step. In fact, the collection step actually occurs on a different machine, for reasons I am not going to delve into in this post. The collector generates an “RRD update order” file, which is transferred to the target machine, where it is parsed by a simple Perl script, which subsequently executes the corresponding RRDs::update statements, with a small degree of parallelism in this last operation.

The application has been running for some months now, and every time I recall the numbers involved, I cannot help but say “wow!”.

pcaptail

2006-08-21T12:08:22Z

The situation: tcpdump is running pretty much continuously on one of your boxes, busily snooping the traffic. The resulting pcap file is huge, and you like it that way.

The task: you want to analyze the current traffic using Wireshark, which is not installed on the box tcpdump is running on. Even X11 is not there, and again, you would like to keep it that way.

The problem: copying the whole file to your desktop machine is an option, but it will take too long. Running another instance of tcpdump and piping its output to Wireshark on your desktop machine would work, but you don’t like it, since the data is already there, and it is nice to use what you have.

The solution: download and compile the pcaptail little program. Run it with the pcap file name as the only parameter. It will behave like tail -f but on the level of individual packets. Then you can feed its output to an instance of wireshark on your desktop machine.

Enjoy. It’s an ugly hack, but you can still enjoy it.

P.S. Long time no C. :-)

On intelligent design

2005-08-08T09:57:35Z

Google bombing is baaad, but Intelligent Design “theory” is worse still. Seriously.

Cyborgs are among us

2005-05-31T11:57:20Z

This is seriously cool:

On June 8, three volunteers will be hooked up to electronics that will give them new sensory powers. For two days, they will adapt to life as a cyborg. One cyborg will feel magnetism. A second will “see” the world through a robot’s artificial eye. The third will sense metal objects hidden behind walls.

``They have small RJ45 adapted claws''

2005-03-30T20:46:49Z

One of my cats managed to unplug my landline phone the other day. I’ve no idea how he managed the feat: you have to press in exactly the right place to disconnect the cable, just like you do it with an ethernet cable.

That’s the guy:

The title of this post courtesy of Phil Regnauld.

Screen color grab to X11 selection

2005-03-17T15:08:03Z

Step 1: install grabc (FreeBSD: ports/x11/grabc) Step 2: install xclip (ports/x11/xclip) Step 3: assign some key in your window manager to the following sequence:

grabc 2>/dev/null|perl -pe 'chomp'|xclip -i

Voila.

Originally, I wanted to make a dockapp to click on, which would do that, but then decided that it’s not worth the bother.

What you get for playing with Firefox'es about:config too much

2005-01-03T18:38:41Z

Go to about:config, set browser.cache.memory.enable to false, then go to any not heavily modified WordPress site and observe a blank page…

The problem: wp-blog-header.php utilizes Cache-Control: no-store, which Firefox apparently tries hard to respect, to the point that if there is no memory cache enabled, the page just is not shown.

Nice bug.

Beware exponential code

2004-12-12T17:38:16Z

I am not discovering America, but this:

s/(\S+)$//s); $x = $1; $y = $_;

is the hell of a lot faster than an equivalent

/^(.*?)(\S+)$/); $x = $2; $y = $1;

One can really feel that with moderately long lines (50K or so).