Steven Engelhardt

1. Diagnosing IIS NTLM authentication failures is a real pain, until…
2. You use Microsoft’s Authentication and Access Control Diagnostics package, which can tell you…
3. You need to enable HTTP Keep-Alives for NTLM to work.

A few days ago, I was messing around with the settings on my Linksys WRT54GL router and moved my home network from 192.168.1.1/24 to 192.168.0.1/24. Changing my mind, I immediately moved back.

Some time later (perhaps after a DHCP lease renewal) I was unable to reach the Internet. Everything looked fine — my connection to the router was working, the router’s connection to the DSL model was working — but I could not reach any websites from my PC. I suspected my DSL connection was down and reset my DSL modem, to no avail.

Today I was preparing to call AT&T and plugged my laptop directly into the DSL modem. I fired up a web browser and, to my astonishment, everything worked!

Assuming AT&T had fixed something on their end, I reconnected my router and tried to access the Internet again. No luck. However, I now knew that the problem was with the router and not the DSL modem.

After a few dead ends (such as cloning my laptop’s MAC address), I took a close look at the results of ipconfig /all:

IPv4 Address. . . . . . . . . . . : 192.168.1.109(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.1.1
DHCP Server . . . . . . . . . . . : 192.168.1.1
DNS Servers . . . . . . . . . . . : 192.168.0.1

For some reason, my router was handing out an incorrect DNS server with its DHCP leases! I reset my router’s settings to the factory defaults, but that didn’t help. Instead, I forced the router to hand out 192.168.1.1 as a DNS server and voila, everything worked.

I suspect that when I moved the router to 192.168.0.1/24 and back I triggered a bug in the WRT54GL firmware where it didn’t update the DNS server correctly.

After I got everything working again, I decided to upgrade the WRT54GL’s firmware to a third-party product: DD-WRT 23 SP2. With the exception of a minor hiccup where I had to manually reset the router’s settings after flashing (something I should have done beforehand), DD-WRT is working flawlessly.

Courtesy of Dave, here’s a script I put into crontab to perform nightly full backups of my WordPress MySQL database:

#!/bin/sh

DATE=`date +%Y-%m-%d`
BACKUPDIR=$HOME/.mysql-backup

USERNAME=XXXXX # TODO
PASSWORD=XXXXX # TODO
DATABASE=XXXXX # TODO

if [ ! -d $BACKUPDIR/$DATABASE ]; then
  mkdir $BACKUPDIR/$DATABASE
fi

mysqldump –user=$USERNAME –password=$PASSWORD $DATABASE > $BACKUPDIR/$DATABASE/$DATE.dump
bzip2 -9 $BACKUPDIR/$DATABASE/$DATE.dump

Microsoft Internet Explorer 7 has been released. While I will likely remain a dedicated Firefox user, I warmly welcome the improvements that IE7 brings.

Developers: Be sure to test your applications against IE7. My product ran into a bug in our code related to IE7’s move to UTF-8 URL encoding.

In order to cope with the Christmas rush, Amazon[.com] has far more computing capacity than it needs for most of the year. As much as 90% of it is idle at times. Renting out pieces of that network to other businesses, such as SmugMug, an online photo site that uses the S3 service, is a way to get extra return on Amazon’s $2 billion investment in technology. Amazon is renting out its physical infrastructure too. Last month it announced Fulfillment by Amazon, which allows other firms to use Amazon’s staff and warehouse space to send out goods and handle returned items. It also introduced WebStore by Amazon, which provides access to all of Amazon’s back-end technology, including the ability to offer third-party products.

Amazon: Lifting the bonnet The Economist 5 Oct 2006.

Google, are you paying attention? Maybe there are ways to make money from your more than 450,000 computers outside of writing your own applications.

This looks quite interesting: Advanced Bash Scripting Guide: An in-depth exploration of the art of shell scripting.

After a long time without making any changes, I have finally added TLS support to my libESMTP patch for mutt. Thanks to Shaun Eack at Wolfram for providing the initial patch.

[14 sengelha@dt]% cat test.cc                                               ~/t
#include <string>

int main(void)
{
    try
    {
        throw std::string(”sigh”);
    }
    catch (…)
    {
        // Do nothing
    }

    return 0;
}
[15 sengelha@dt]% g++ -g -o test test.cc                                    ~/t
[16 sengelha@dt]% ./test                                                    ~/t
zsh: 24046 abort (core dumped)  ./test
[17 sengelha@dt]% gdb ./test test.core                              < 134 > ~/t
GNU gdb 4.16.1
Copyright 1996 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type “show copying” to see the conditions.
There is absolutely no warranty for GDB.  Type “show warranty” for details.
This GDB was configured as “sparc64-unknown-openbsd3.5″…
Core was generated by `test’.
Program terminated with signal 6, Abort trap.
#0  0×43b7aa40 in ?? ()
(gdb) bt
#0  0×43b7aa40 in ?? ()
#1  0×102a84 in uw_init_context_1 (context=0xfffffffffffeedb0,
    outer_cfa=0xfffffffffffef130, outer_ra=0×44208088)
    at /usr/src/gnu/usr.bin/gcc/gcc/unwind-pe.h:77
#2  0×102e14 in _Unwind_RaiseException (exc=0×643d5c455b4a3a63)
    at /usr/src/gnu/usr.bin/gcc/gcc/unwind-pe.h:77
#3  0×44208090 in ?? ()
#4  0×1012c4 in main () at test.cc:3
#5  0×101054 in ___start ()
(gdb) quit
The program is running.  Quit anyway (and kill it)? (y or n) y
[18 sengelha@dt]% g++ –version                                             ~/t
g++ (GCC) 3.3.2 (propolice)
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[19 sengelha@dt]% uname -a                                                  ~/t
OpenBSD dt.xxxxxxx.xxx 3.5 GENERIC#123 sparc64

Such are the perils of running the fairly uncommon hardware/software combination of OpenBSD/sparc64. *sigh*

URL “purity” is important to me, so I wanted to note the following evolt.org article: Making “clean” URLs with Apache and PHP.

To quickly summarize, my email setup is a combination of fetchmail, procmail, postfix, bogofilter, and mutt (with my libESMTP patch) using mbox files on a colocated Sun Fire V100 running OpenBSD/sparc64. For the most part I very much like this setup. In the past I have tinkered with SpamAssassin, maildir, and IMAP; I may end up setting up IMAP again in the future. BTW, I have an old entry describing how to properly integrate bogofilter and mutt.

The purpose of this post is to describe how I archive a copy of every piece of incoming email using these tools. I find these archives extremely valuable, whether for the occasional historical search or in that it enables me to be highly aggressive in keeping my mailboxes clean by deleting messages. Recently I have been contemplating performing statistical analysis of received spam based on these archives.

The first step to take when setting up a historical email archive is to consider how the email will be stored. For my regular email, I use the system-default mailbox for my inbox (/var/mail/sengelha) and store all my sorted email in mbox files in ~/.mail/mailboxes/. For backups, I keep mail in monthly archives in the mbox file ~/.mail/backups/YYYY/YYYY-MM, where YYYY is the current year and MM is the two-digit month (01 = January and so forth). For all past months I compress the mail using bzip2 -9 to save hard drive space. I used to keep daily archives but per the suggestion of Dan Sachs I recently moved to monthly to achieve better compression ratios. To give you an idea of the sizes that are involved, my October 2004 email archive is 21,947,942 bytes (7,734,053 bytes compressed). Undoubtedly much of this is spam.

My setup has two major parts:

1. Save a copy of every incoming email to the ~/.mail/backups/YYYY/YYYY-MM file.
2. Compress all past monthly archives using bzip2 -9.

To save a copy, I must first ensure that the directory ~/.mail/backups/YYYY/ already exists. In fact, I lost some email in January one year because the directory didn’t exist so the email backups were never saved. I have the following rule to my .procmailrc to ensure the directory always exists:

:0wc
* ! ? test -d $HOME/.mail/backups/`date +%Y`
| mkdir -p $HOME/.mail/backups/`date +%Y`

I then use the following rule to save a copy of every incoming email:

# Make backups of all mail received in format YYYY/YYYY-MM
:0c
$HOME/.mail/backups/`date +%Y`/`date +%Y-%m`

I wrote the scripts bz2compressdir and compress-mail-backups (see below) to compress all past monthly email archives. I execute compress-mail-backups nightly using cron. (Monthly would suffice but there is effectively no penalty for running compress-mail-backups too frequently.)

bz2compressdir:

#!/usr/bin/perl -w
#
# bz2compressdir: Compress all files in a directory with bzip2
#
# $Id: bz2compressdir,v 1.2 2004/02/19 05:42:35 sengelha Exp $

use strict;
use File::Find;
use Getopt::Std;

my $usage = <<EOF;
Usage: $0 [options] dir1 [dir2 dir3 ...]

-e <regexp>  exclude <regexp> from files to compress
-h           print usage and exit
EOF
my %opts = ();

sub handleFile {
    if (-f && !/.bz2$/ && (!$opts{’e'} || !/$opts{’e'}/)) {
        `bzip2 -9 $_`;
    }
}

getopts(’he:’, %opts) or die $usage;

die $usage if ($opts{’h'});
die $usage if ($#ARGV == -1);

foreach my $dirname (@ARGV) {
    die “$dirname: Not a directory” if (! -d $dirname);

    find(&handleFile, $dirname);
}

compress-mail-backups:

#!/bin/sh
#
# compress-mail-backups: Go through every mail backup directory,
# compressing with bz2compressdir
#
# $Id: compress-mail-backups,v 1.2 2004/02/19 05:42:35 sengelha Exp $

exec nice bz2compressdir -e `date +%Y-%m` $HOME/.mail/backups/200?