Version 1.23 of LogMiner, my Apache/IIS log analysis package, is now available.

This release mainly fixes some build problem on modern distributions.

Also it finally includes the cleanup function I was talking about some days ago...

Recently I needed to gather some statistics about the distribution of the different Mac OS X versions installed by users of a certain application.

Since the application has an update-check feature, I thought I'd use the web server logs to infer the data I was interested in. The only thing I could use was the User-Agent string sent by the application when requesting the file containing the update information.

After some digging in Darwin's build plists and a bit of googling, I came up with this list of patterns:

            OS            |                user agent pattern

--------------------------+---------------------------------------------------

 Mac OS X 10.2            | ^CFNetwork/1\.1$

 Mac OS X 10.3.2-10.3.8   | ^CFNetwork/1\.2\.1$

 Mac OS X 10.3.9          | ^CFNetwork/1\.2\.[2-6]$

 Mac OS X 10.4            | ^CFNetwork/128$

 Mac OS X 10.4.10         | ^CFNetwork/129\.21$

 Mac OS X 10.4.11         | ^CFNetwork/129\.22$

 Mac OS X 10.4.2          | ^CFNetwork/128\.2$

 Mac OS X 10.4.3          | ^CFNetwork/(129\.5|10\.4\.3)$

 Mac OS X 10.4.4          | ^CFNetwork/(129\.(9|10)|10\.4\.4)$

 Mac OS X 10.4.5          | ^CFNetwork/129\.11$

 Mac OS X 10.4.6          | ^CFNetwork/129\.13$

 Mac OS X 10.4.7          | ^CFNetwork/(129\.16|4\.0)$

 Mac OS X 10.4.8          | ^CFNetwork/129\.1(8|9)$

 Mac OS X 10.4.9          | ^CFNetwork/129\.20$

 Mac OS X 10.5            | ^CFNetwork/21[7-9]$

 Mac OS X 10.5.1          | ^CFNetwork/220$

 Mac OS X 10.5-prerelease | ^CFNetwork/1[4-9][0-9](\.[0-9])?|20[0-9]|21[0-9]$

They might not be 100% correct, but they're good enough.

Version 1.21 of LogMiner, my Apache/IIS log analysis package, is now available.

This release sports a new report: Networks.

It allows you to monitor where your visitor come from, in terms of organization and network. To use this report, you need to define a list of organizations and a list of networks owned by them.

For instance, if you wish to check how many hits you get from Google's crawlers, you can define a Crawler network owned by Google with the address 66.249.64.0/19 (this is just an example and may not be entirely correct, although my logs always show GoogleBot coming from that IP range).

Version 1.20 of LogMiner, my Apache/IIS log analysis package, is now available.

This version doesn't require anymore libpqxx 2.5.5: it compiles as well with version 2.6.9 (I don't understand why they had to remove a couple of really useful functions in the 2.6.x series, though).

The other most important change is that in case of SQL errors while inserting data into the DB, LogMiner won't abort the parsing of the log. Instead, it will record the error and skip the log line that caused the problem. This fix has actually been inspired by lame spammers who keep on trying to exploit the contact page on this site; too bad that they didn't study HTTP well enough: the content of a POST request follows the headers, it doesn't precede the method. They were causing LogMiner to try to fit the spam body into the varchar(32) method column.