Tekker Blog

Ah, the command line. It can be awesome and a pain in the ass at the same time. In a GUI file manager, you can just highlight the files you want and get a total size of the selected files. On a *nix command line? Not so simple.

find . -iname '*.jpg' | sed "s/ /\\\\ /g" | xargs du -ksb | cut -f1 | xargs echo | sed "s/ /+/g" | bc

That does it Breaking this monster down gives us the following.

First cd to the directory you want to check in.
cd Pictures

List the files. find finds files.
find . -iname '*.jpg'
./Picture3 001.jpg
./Picture3 002.jpg
...
./Picture4 099.jpg


List the files, excaping spaces. sed allows you to perform actions against the string being passed to it.
find . -iname '*.jpg' | sed "s/ /\\\\ /g"
./Picture4\ 001.jpg
./Picture4\ 002.jpg
./Picture4\ 003.jpg
...
./Picture4\ 099.jpg


List the files, getting file size. xargs allows you to take a line of input and execute it. du is used to get the disk usage size.
find . -iname '*.jpg' | sed "s/ /\\\\ /g" | xargs du -ksb
696009 ./Picture4 001.jpg
675879 ./Picture4 002.jpg
666862 ./Picture4 003.jpg
...
658225 ./Picture4 099.jpg


Cut the file size out of the listings. cut -f1 gives us the first field using ‘space’ as a delimeter.
find . -iname '*.jpg' | sed "s/ /\\\\ /g" | xargs du -ksb | cut -f1
700696
702453
703594
...
696009


Take all of our lines of file sizes and concatenate them to a single line.
find . -iname '*.jpg' | sed "s/ /\\\\ /g" | xargs du -ksb | cut -f1 | xargs echo
700696 702453 703594 ... 696009

Replace the spaces in our single line with ‘+’.
find . -iname '*.jpg' | sed "s/ /\\\\ /g" | xargs du -ksb | cut -f1 | xargs echo | sed "s/ /+/g"
700696+702453+703594+ ... +696009

Now take the output above and use bc to calculate it. bc is a simple command line calculator.
find . -iname '*.jpg' | sed "s/ /\\\\ /g" | xargs du -ksb | cut -f1 | xargs echo | sed "s/ /+/g" | bc
54631548

How big are all the jpg files in my pictures directory? 54,631,548 bytes.

So, command line? Pain in the ass to get something so simple, but pretty awesome that there are so many different, tiny programs for *nix that can work together to get you the results you want.

I have backup routines that I have setup on my laptop and my file server. I have Cygwin installed on both since both machines are Windows (hey, I love Linux too, shut up!). I use rsync over ssh to copy files from my laptop to the file server and then again use rsync to sync all the shared files from 1 500GB drive to another in the same machine (raid 1 didn’t work out so well for me, even with a Promise Raid card installed). Having to type my password in 4 times to backup my laptop was annoying, so I decided to setup ssh keys between the laptop at the file server.

Fairly simple setup:

1. Log into your shell (doesn’t matter if its Cygwin or a standard Linux distro)
2. Change to the .ssh directory
   $ cd .ssh

3. Generate the private/public key pair (use defaults, including the password, unless you wish to still type in a password on each connect)
   $ ssh-keygen -t dsa
   Generating public/private dsa key pair.
   Enter file in which to save the key (/home/[username]/.ssh/id_dsa):
   Enter passphrase (empty for no passphrase):
   Enter same passphrase again:
   Your identification has been saved in /home/[username]/id_dsa.
   Your public key has been saved in /home/[username]/id_dsa.pub.
   The key fingerprint is:
   aa:d3:81:89:31:13:01:7c:80:d8:e5:4d:90:b1:25:a6 [username]@[hostname]

4. Transfer the file to the remote machine you wish to use the key to log into
   $ scp id_dsa.pub administrator@192.168.111.45:./id_dsa.pub

5. SSH to the remote machine
   $ ssh [remoteusername]@[remotehostname]

6. Touch the authorized_keys2 file, just in case it doesn’t exist yet
   $ touch ~/.ssh/authorized_keys2

7. Set permissions on the authorized_keys2 file so that only the owner can read/write to it
   $ chmod 600 ~/.ssh/authorized_keys2

8. Copy the contents of the public key into the authorized_keys2 file
   $ cat id_dsa.pub >> ~/.ssh/authorized_keys2

9. Delete the public key file, just because it’s safer that way!
   $ rm id_dsa.pub

10. Logout of the remote server
    $ exit

11. SSH back into the remote server and test to see if it asks you for a password (assuming you did not enter a password while generating the keys earlier)
    $ ssh [remoteusername]@[remotehostname]

Now you can ssh from your machine to the remote machine without using passwords but still keeping the remote machine secure.

Ok, so, a few weeks ago I posted about how great my experience was VPSLand was. At the time, it was all true. Since then, they upgraded their plans and I took advantage of that and got an update plan. Since then my pages started being slow when loading. When I RDP’ed into the VPS, it was extremely slow if it even would allow me to connect. I know I am not paying top dollar so I was willing to accept it until my pages didn’t load at all.

Enter Linode.com. Yes, I switched from a Windows VPS to a Linux VPS.

First off, let me say that I looked at Slicehost and Linode because a friend recommended them both and said she uses Slicehost and heard lots of good stuff about Linode. I checked them both out over at WHT and found a few complaints about both but everyone’s experience is different and the majority of posts were good for both hosts. There were 2 reasons I went with Linode over Slicehost.

• Monthly payments. Slicehost does allow monthly payments but initial signup requires 3 months up front. Even with their cheapest plan, that is $60.
• Better plans. Not huge differences, but Linode offers 12GB hard disk space and 360MB ram while Slicehost offers 10GB hard disk space and 256MB ram.

Linode’s control panel is cool as hell! You are able to setup a Linux Linode from various flavors right thru the control panel. If you decide you don’t like that flavor or you screwed up your installation, you simply delete the Linode and create a new one. I could run DNS on my Linode, but it really isn’t necessary. Their DNS manager is awesome. Once you create a zone for one of your domains, you are able to clone that zone with another domain. Cloning a zone to a new domain is a 2 step process. You click on “Clone an Existing Zone” and then type your domain in and hit submit. You can add private IPs if you have multiple Linodes so they can communicate and not use your monthly bandwidth allowance. There is also an Ajax console that lets you have KVM type access to your Linode so if you lock yourself out of your server while messing with iptables, there is no reason to freak out, you just log in at the Ajax console and undo that nasty iptables rule. We have all done that a few times .

All in all, I’ve only been messing with Linode for a week, but all my sites (including this one) are now running on it and things seem to be going great so far. I’d recommend trying out Linode if you are looking for a cheap, fast and so far, excellent Linux host. If you decide to check it out and want to help me out, use this link to give me $20 credit toward my next bill .

So, from time to time, I am gonna start posting. This is going to be my tech blog. No one will read it, but it will be a good place for me to post little tech things that I will need again but will definatly forget because it isn’t used every day!

Todays?

du -ks ./* | sort -n

That little beauty shows the directories in your current directory and it’s size. It’s useful to find out which folders are biggest to find who or what is hogging your disk space!