Simone 'Wiz' Tellini - SysAdmin

LogMiner 1.23

nospam@example.com (Simone) — Tue, 30 Jun 2009 10:42:32 -0700

Version 1.23 of LogMiner, my Apache/IIS log analysis package, is now available.

This release mainly fixes some build problem on modern distributions.

Also it finally includes the cleanup function I was talking about some days ago...

Perplexed about Ubuntu's sudoers configuration

nospam@example.com (Simone) — Wed, 29 Apr 2009 13:47:47 -0700

Recently I've been administering my first Ubuntu machine and I already feel a bit perplexed about its security setup.

The issue is that by default, the root account password is locked in Ubuntu and you are encouraged never to use root, but rely on sudoers for system administration.

I've read the rationale on Ubuntu's wiki and I agree on some points, but their mindset is totally biased towards a desktop setup.

Let's consider these points from their site:

Benefits of using sudo: Users don't have to remember an extra password.
This is cool for Average Joe, but don't tell me that it's an advantage on a serious server: it's actually a downside, see the next point.

Every cracker trying to brute-force their way into your box will know it has an account named root and will try that first. What they don't know is what the usernames of your other users are. Since the root account password is locked, this attack becomes essentially meaningless, since there is no password to crack or guess in the first place.

Sure, the attack on root becomes useless, too bad that SSH brute-force attacks usually try lots of different usernames. If the manage to break a single account of a sudoer, they automatically have control of your machine. If you had to remember a different password to su your way, it would make their life a bit harder.

Allows easy transfer for admin rights, in a short term or long term period, by adding and removing users from groups, while not compromising the root account.

err... what does "not compromising the root account" actually mean? If I get admin rights, I can do whatever I want, including compromising any account. Unless I get authorisation just to run a limited set of commands through sudo, but that's not the point of this post.

The root account password does not need to be shared with everybody who needs to perform some type of administrative task(s) on the system.

ok, but what problem does it solve? (apart from avoiding people shouting the root password when they shouldn't )

If I want to remove a person from the admin group, I need to trust her not to have planted any malicious program (rootkits, backdoors...) or rebuild the system if I'm paranoid, if I really want to be on the safe and paranoid side.

IMHO, it's just the same as changing the root password when one is gone, or even better, periodically.

I'm not convinced. I still prefer the common su approach typical of almost all the other distributions...

mod_tunnel for Apache 2.2.x

nospam@example.com (Simone) — Sun, 12 Apr 2009 17:52:07 -0700

mod_tunnel is a simple Apache module that can be used to create TCP tunnels using your Web server. It is useful to expose services which can be reached through a proxy, bypassing firewalls.

After only 4 years since the previous version, you can now download version 2.0 which works with Apache 2.2.x (and maybe 2.0.x, but I haven't tested it).

You'll find it on sourceforge.

Don't shout at your hard drives!

nospam@example.com (Simone) — Mon, 05 Jan 2009 13:47:43 -0800

It appears that hard drives don't like it...

It also proves that some sysadmins have far too much time to spare.

Who's linking to your site?

nospam@example.com (Simone) — Fri, 20 Jun 2008 14:57:28 -0700

If you're using LogMiner to analyse your Apache/IIS logs, here's an easy recipe to be notified of sites that send visitors your way.

First, let's define a Pg/PLSQL function to extract the new referrers:

CREATE TYPE new_referrer AS (
site_id int8,
site_name text,
referrer text,
hits int
);

CREATE OR REPLACE FUNCTION get_new_referrers()
RETURNS SETOF new_referrer AS $body$
DECLARE
rec new_referrer;
lastRef int8;
BEGIN

SELECT INTO lastRef CAST( value AS int8 )
FROM catalog
WHERE name = 'LastReferrer';

IF lastRef IS NULL THEN
INSERT INTO catalog ( name, value ) VALUES ( 'LastReferrer', 0 );
lastRef := 0;
END IF;

FOR rec IN
EXECUTE 'SELECT s.id AS site_id, s.name AS site_name, r.referrer, tmp.hits ' ||
'FROM ( ' ||
' SELECT site, referrer, COUNT(*) AS hits ' ||
' FROM accesses ' ||
' WHERE referrer > ' || lastRef ||
' GROUP BY site, referrer ' ||
') tmp ' ||
'JOIN sites s ON s.id = tmp.site ' ||
'JOIN referrers r ON tmp.referrer = r.id ' ||
'WHERE r.extern = true ' ||
'ORDER BY s.name, tmp.hits DESC, r.referrer'
LOOP

RETURN NEXT rec;

END LOOP;

UPDATE catalog
SET value = ( SELECT MAX( id ) FROM referrers )
WHERE name = 'LastReferrer';

RETURN;
END;
$body$ LANGUAGE plpgsql;

Then, a little PHP script to run the query and format the results:

#!/usr/local/bin/php
<?php

$lines = array();
$lastSite = '';

// quickest and dirtiest way to execute a query in PHP ;-)
exec ( "echo 'SELECT r.* " .
"FROM get_new_referrers() r " .
"LEFT JOIN search_engines se ON ( r.referrer ~* se.pattern AND r.referrer ~* se.query_pattern )" .
"WHERE se.id IS NULL' | /usr/local/pgsql/bin/psql -U logminer -At logminer",
$lines );

foreach( $lines as $line ) {

$parts = explode( '|', $line, 4 );

if( $lastSite != $parts[ 0 ] ) {

printf( "\n\n\n\n" .
"-------------------------------------------------------\n" .
"Referrers for %s\n" .
"-------------------------------------------------------\n\n",
$parts[ 1 ] );

$lastSite = $parts[ 0 ];
}

printf( "[%d] %s\n\n", $parts[ 3 ], $parts[ 2 ] );
}
?>

The query above grabs the referrers for every site available in the DB, filtering requests coming from known search engines to reduce the level of "noise".

This script is finally invoked by the cron job that processes the logs:

referrers_report.php | mail -s 'New referrers report' email@example.com

lm_sensors config for Asus T3-P5945GCX (P5L13L motherboard)

nospam@example.com (Simone) — Sat, 08 Mar 2008 11:24:00 -0800

I've just installed the latest Mandriva 2008.1 RC on this fine Asus barebone.

Everything went fine, except that the output of sensors showed too many ALARM messages for my taste... so, here's how I've modified the pre-installed sensors.conf to set some min/max values closer to reality and to hide useless lines:

# Winbond W83627EHF configuration originally contributed by Leon Moonen
# This is for an Asus P5P800, voltages for A8V-E SE.
chip "w83627ehf-*" "w83627dhg-*"

label in0 "VCore"
label in2 "AVCC"
label in3 "3VCC"
label in7 "VSB"
label in8 "VBAT"

# +12V is in1 and +5V is in6 as recommended by datasheet
compute in1 @*(1+(56/10)), @/(1+(56/10))
compute in6 @*(1+(22/10)), @/(1+(22/10))
set in1_min 12.0*0.9
set in1_max 12.0*1.1
set in6_min 5.0*0.95
set in6_max 5.0*1.05

set in4_min 1.6*0.9
set in4_max 1.6*1.05
set in9_min 1.6*0.9
set in9_max 1.6*1.05

# Set the 3.3V
set in2_min 3.3*0.95
set in2_max 3.3*1.05
set in3_min 3.3*0.95
set in3_max 3.3*1.05
set in7_min 3.3*0.95
set in7_max 3.3*1.05
set in8_min 3.3*0.95
set in8_max 3.3*1.05

# Fans
label fan2 "CPU Fan"

ignore fan1
ignore fan3
ignore fan4
ignore fan5

# Temperatures
label temp1 "Sys Temp"
label temp2 "CPU Temp"
label temp3 "AUX Temp"

set temp1_over 45
set temp1_hyst 40

Corrections are welcome.

LogMiner: purging old accesses

nospam@example.com (Simone) — Sat, 26 Jan 2008 18:53:22 -0800

I've just committed a function to easily purge old data from LogMiner's database, useful to prevent it from growing too much.

It'll appear in the next version, but since its release might still be far from now, here's the code for those who need it:

CREATE OR REPLACE FUNCTION cleanup( _site int8, _upToDate date )
RETURNS INT AS $body$
BEGIN

DELETE FROM accesses
WHERE req_time < _upToDate
AND site = _site;

ALTER TABLE accesses DROP CONSTRAINT accesses_request_fkey;
ALTER TABLE accesses DROP CONSTRAINT accesses_search_fkey;
ALTER TABLE accesses DROP CONSTRAINT accesses_referrer_fkey;

DELETE FROM requests
WHERE id NOT IN ( SELECT DISTINCT request FROM accesses );

DELETE FROM search_referrals
WHERE id NOT IN ( SELECT DISTINCT search FROM accesses );

DELETE FROM referrers
WHERE id NOT IN ( SELECT DISTINCT referrer FROM accesses );

ALTER TABLE accesses
ADD CONSTRAINT "accesses_request_fkey"
FOREIGN KEY ( request )
REFERENCES requests( id ) ON DELETE CASCADE;

ALTER TABLE accesses
ADD CONSTRAINT "accesses_search_fkey"
FOREIGN KEY ( search )
REFERENCES search_referrals( id ) ON DELETE SET NULL;

ALTER TABLE accesses
ADD CONSTRAINT "accesses_referrer_fkey"
FOREIGN KEY ( referrer )
REFERENCES referrers( id ) ON DELETE SET NULL;

RETURN 0;
END;
$body$ LANGUAGE plpgsql;

For instance, if you want to keep only the last six month of data, you can set up a cron job which runs at the first day of every month executing the command:

# supposing 1 is the id of your site
echo "SELECT * FROM cleanup( 1, CAST( date_trunc( 'month', now() ) - interval '5 months' AS date ));" | psql -U logminer logminer

Mac OS X version statistics through CFNetwork analysis

nospam@example.com (Simone) — Fri, 07 Dec 2007 12:05:00 -0800

Recently I needed to gather some statistics about the distribution of the different Mac OS X versions installed by users of a certain application.

Since the application has an update-check feature, I thought I'd use the web server logs to infer the data I was interested in. The only thing I could use was the User-Agent string sent by the application when requesting the file containing the update information.

After some digging in Darwin's build plists and a bit of googling, I came up with this list of patterns:



            OS            |                user agent pattern

--------------------------+---------------------------------------------------

 Mac OS X 10.2            | ^CFNetwork/1\.1$

 Mac OS X 10.3.2-10.3.8   | ^CFNetwork/1\.2\.1$

 Mac OS X 10.3.9          | ^CFNetwork/1\.2\.[2-6]$

 Mac OS X 10.4            | ^CFNetwork/128$

 Mac OS X 10.4.10         | ^CFNetwork/129\.21$

 Mac OS X 10.4.11         | ^CFNetwork/129\.22$

 Mac OS X 10.4.2          | ^CFNetwork/128\.2$

 Mac OS X 10.4.3          | ^CFNetwork/(129\.5|10\.4\.3)$

 Mac OS X 10.4.4          | ^CFNetwork/(129\.(9|10)|10\.4\.4)$

 Mac OS X 10.4.5          | ^CFNetwork/129\.11$

 Mac OS X 10.4.6          | ^CFNetwork/129\.13$

 Mac OS X 10.4.7          | ^CFNetwork/(129\.16|4\.0)$

 Mac OS X 10.4.8          | ^CFNetwork/129\.1(8|9)$

 Mac OS X 10.4.9          | ^CFNetwork/129\.20$

 Mac OS X 10.5            | ^CFNetwork/21[7-9]$

 Mac OS X 10.5.1          | ^CFNetwork/220$

 Mac OS X 10.5-prerelease | ^CFNetwork/1[4-9][0-9](\.[0-9])?|20[0-9]|21[0-9]$

They might not be 100% correct, but they're good enough.