A collection of thoughts, tips, musings and the occasional recipe. Basically all the stuff I am supposed to remember.

You know you have a problem when you have 3 DNS servers for 2 people.

I was waiting for a Windows server to reboot so I took the time to tune my DNS servers at home (ssh + bind + Linux = FTW). I then tested my setup with the DNS benchmark utility from grc.com. To summarize the output, it turns out (not surprisingly given that there are only two people who use them) that my DNS servers are faster than all of the publicly available ones, including those provided by my ISP. Maybe I have a problem and should find some less geeky hobbies. 

Here is the relevant output from that DNS benchmark utility:

Using Linux Software RAID and LVM to build your own NAS


** Update 2014 **
I am transitioning away from RAID arrays (and no longer use LVM). For smaller amounts of data I am migrating to btrfs (RAID 10 or RAID 6 and larger amounts of data a cluster (using ceph or swift) is usually the answer.
** End Update **

I am a big fan of RAID arrays and I use them everywhere. After having been bitten by failed, end of life hardware devices, I have become a fan of the software RAID. In particular I am quite happy with the Linux software RAID capabilities when paired with the Logical Volume Manager (LVM).

First, I create some partitions on the drives I want to use, in my case, I have four 1TB drives that I am putting into an array. You do not need to create partitions as Linux can use the whole drive as it is, where you may have issues is if you plug the drive into a computer that does not support Linux RAID, it may prompt you to format what appears to be a blank drive. If you use partitions, at least the OS will see an unsupported partition and probably do nothing. This is not a big deal but it can help to prevent you from shooting yourself in the foot, especially when you have more than a few drives on the workbench and more than one OS in daily use.

Formatting a DROBO with Linux

I picked up a DROBO storage device from a friend of mine who had too many. I plugged the device in to my Linux file server and the DROBO appeared as a standard hard drive. I was able to partition the drive normally, but I had a problem when I tried to format the device.

Apparently the DROBO appears as a 16 Terabyte drive regardless of the amount of storage actually installed. The DROBO keeps track of how much space is used internally and communicates this info via a series of LED lights on the front as well as by its own API (in other words you need a 3rd party app to get this info since the DROBO is essentially lying to your OS). The idea is that if you run out of space you simply plug in a bigger drive and the Drobo will on the fly integrate the new bigger drive.

Anyway, a mkfs.ext3 (the DROBO supposedly supports this file system, and it needs to understand the file system at the block level to do its magic) fails because it tries to verify each inode. Of course the majority of them will fail since they don't actually exist yet (I only had 4 x 1 terabyte drives installed). What I needed to do was pass the -F flag to mkfs.ext3 to make it ignore the errors and proceed with the format. man mkfs.ext3 for more details and other flags.

Disk usage

Just posting a handy little trick to get your disk usage from the command line.

While the df command will show you what the disk utilization for your partitions, sometimes you need more specific details, like which files and folders are taking up all of your space.

Here is how I do this:

  1. df -Pacmx --max-depth=1 . |sort -g

This is kind of awkward to type out all the time, so you create an alias.

  1. echo "alias dus=\"df -Pacmx --max-depth=1 . |sort -g\"" >> ~/.bashrc

Now type

  1. source ~/.bashrc

From any directory, you can now type "dus" without the quotes to see all the files and directories sorted by size. It should also be noted that df is rounding up to the nearest megabyte.

One final note, the flags I used are specific to the GNU version of these utilities. You BSD folks will have to adjust the df and sort options.

vmware arrow keys not mapped correctly

Ever since I upgraded to Ubuntu 8.10 the arrow keys in VMWare workstation are not mapped correctly. For example the down arrow key behaves like the Windows key, but only inside a VM (including the simulated BIOS). Outside of VMWare the keyboard behaves normally.

It turns out I was not the only one experiencing this. The solution is to add xkeymap.nokeycodeMap = true to your ~/.vmware/config file. You could do this from the command line with:
echo 'xkeymap.nokeycodeMap = true' > ~/.vmware/config but any text editor will do.

More information can be had from the VMWare forums

Using lsof to figure out which process is responsible for a particular network connection

Edit:
It looks someone else has a more complete set of examples for lsof. His page can be found HERE.

Have you ever tried to track down what process or program is making a network connection on a Linux machine? Why would you do this? Say you have some dropped packets in your firewall logs coming from a Linux machine on your network (you do do outbound filtering don't you?) and you would like to track down the process on the offending machine.

Anyway, to cut to the chase:


lsof -i -P

Will show you processes have active network connections. The "-P" option does not do a lookup of port names, it just shows the port number.

You can specify a host name, port number, or protocol, or any combination thereof:

Syndicate content