Anystat daemon

This project has been on my mind for years. The first notes are some 4 years old and the first, now abandoned, code about 3. I love gathering statistics about computers, particularly servers, and networks. But I don’t very much like the various software-packages that exist for gathering and displaying those statistics. Many are difficult to configure and rather bloated. Due to my target platform, Linux, adhering to the UNIX-philosophy of ‘everything is a file’, it really shouldn’t be that difficult. Using the console, you can pretty much get anything on screen, either by reading files in the /proc-filesystem, reading logfiles or using specific commands. So the idea was born to write a statistics gathering daemon that would be able to read those files and run those commands, while being told what fields to look for in the output. This idea has led to a prototype which is able to cover a lot of this ground with just a few basic functions. For input-methods, I now have:

  • type CAT: periodically read a file
  • type TAIL: continuously track additions to a file
  • type CMD: periodically run a command and read the output
  • type PIPE: continuously run a command and keep reading the output

Then, I have a few methods to process the output:

  • subtype COUNT: count the number of lines
  • subtype VALPOS: read the value from a specified (whitespace-separated) position on each line
  • subtype LINEVALPOS: read the value from a specified position on a specified line
  • subtype NAMECOUNT: read the name from a specified position and count the number of lines for each name
  • subtype NAMEVALPOS: read the name from a specified position and the corresponding value from another position

Obviously, not all these types and subtypes go together, but as should be clear, the method of specifying statistics inputs can be very generic this way. I’ve also added the ability to use regular expressions to filter output and to use regex submatches to find the name and value fields instead of using a whitespace-separated position.

My prototype is up and running and gathering a lot of statistics already, displaying averages and writing the time-series to plaintext files. The next step is to figure out an equally generic and simple way of displaying the data. Graphs tend to be the easiest and most natural way to study statistics, so naturally I’m looking into that. But drawing graphs programmatically is not a simple matter and I’m not sure whether I will be able avoid presenting a lot of that complexity to the end user.