Sunday, December 30, 2012

Tracking Israel Post Items

Tracking numbers for post items are great way to track your online shopping orders, sent packages, etc. What is more great, is the ability to get email notifications about status change of your items. Yet, Israel Post website does not provide email notifications. You have to manually enter your tracking number each time you want to check its status. There is not even a way to create a bookmark to your tracking number status page.

Frustrated by this, I started investigating how their system works by looking at the website code/html, trying to create a direct URL to my item status page. Maybe I can take this link and use webpage monitor service which can send a notification once something is changed within that page. What I found was that the Israel Post website sends a GET form with the tracking number and gets a JSON response and then dynamically updates the page. I was disappointed since I couldn't find a web monitor that can handle JSON.

Introducing PostIL Tracker


So I decided to write a script using Google App Script which does just that: sends a GET form and interprets the returned JSON data. One thing led to another and I found my self adding a web interface to it (AKA web app). Another nice feature of Google App Script is the ability to set a timed trigger which runs a specific routine, so I created a routine which routinely checks the status of the post items and send email when the status is changed. Voila!

I have decided to share this app with all other Israel Post frustrated, and added it to Chrome Web Store. Alternatively, you can simply run the script by clicking here.

You must have a Google account to run this. You must also authorise the script to use the mentioned resources (don't worry, I don't still your data. Everyone's privacy is respected).

Here is what it looks like:

Screen shot from PostIL Tracker

All you have to do is add your items tracking numbers at the top and your email address at the bottom (for status change notifications). The script will do the rest - in the cloud (using Google App Script resources).






Sunday, November 11, 2012

Small Script to Backup Using s3cmd

After many disappointments of using EC2 to perform regular backups of my important files on my NAS, I have decided to go back to S3 backup tools. It works better then I remembered (I guess two years can make a difference), so I wrote a little script to backup my data on a weekly basis. Basically it uses s3cmd to perform an rsync-like backup (well it's not really rsync, but it's good enough for my purposes). S3 is much cheaper especially because you pay for what you use (unlike EBS volumes) and outbound data transfer costs nothing. Here it goes:


#!/bin/bash

SCRIPTDIR=`dirname $0`

# The '--dry-run' will make sure the script won't really do 
# anything. Comment it only after carefully reviewing
# the output
DEBUG_SWITCHES="--dry-run"

# The following switches will use "reduced redundancy" backup
# (cheaper) and will delete files from backup when removed 
# locally. The files.exclude contains file patterns to exclude
# from backup
COMMON_SWITCHES="--rr --delete-removed --rexclude-from=$SCRIPTDIR/files.exclude $DEBUG_SWITCHES"

echo "Running $0"echo Backup started at `date` 

# The following will cause printing of the command before 
# executing it. Useful for logging
set -o xtrace 

# s3cmd command for each dir I need to backup.
# Need to replace "bucket" with real bucket name
s3cmd sync $COMMON_SWITCHES $DEBUG_SWITCHES /mnt/vg1/lv0/users/ s3://bucket/users/

s3cmd sync $COMMON_SWITCHES /mnt/vg1/lv0/media/Audio/ s3://bucket/media/Audio/

s3cmd sync $COMMON_SWITCHES /mnt/vg1/lv0/media/Music/ s3://bucket/media/Music/

s3cmd sync $COMMON_SWITCHES /mnt/vg1/lv0/media/Pictures/ s3://bucket/media/Pictures/
set +o xtrace 

echo Backup finished at `date`


Friday, October 22, 2010

DIY: Add Auxiliary Port to Blaupunkt RD4

As a Citroen C4 owner with RD4 unit I always wanted to connect external devices (iPod) into the auxiliary input. However, Citroen/Peugeot are making this a difficult and expensive task. It requires you to:
1. Go to your local dealer to enable AUX1/AUX2 input in your RD4 unit.
2. Buy and install 9706AG part.

The 1st part is mandatory unless if you have access to Citroen's software, cable and a password. I just went to a local dealer to do this. It's not expensive and you must have AUX1 or AUX2 enabled before you continue with this tutorial. You can check if you have it enabled already by pressing on the 'SOURCE' button in the RD4 unit. If you have AUX enabled you will see "AUX" displayed after two or three presses. If you don't, go to the dealer.

The 2nd part can be avoided by using simple parts and little bit of work.

DISCLAIMER: DO IT AT YOUR OWN RISK!

What you need

1. ATA connector - this can be any IDE or floppy disk connector from an old computer. I prefer the floppy disk connector since it has 34 wires while IDE usually has 80 wires for crosstalk reduction. It should look like this:

2. 2 RCA chassis sockets.  I bought from a local hardware store. I also found this in Farnell.
3. Small plastic box (optional). Found this in Farnell.
4. Wires to connect all these components.

You also need a soldering iron and a release keys to release your RD4 unit (you can buy release keys from eBay or try releasing by putting 2 small screwdrivers in the RD4 release holes and pulling easily).

Cut the ATA connector

First get rid of one connector by cutting it (we need only one side). You should also expose the wires for later.

Second, we need the ATA connector to be a 5 pins width (meaning a total of 10 pins). I used a small kitchen knife to cut it but I am sure there are more elegant ways to achieve this.

Find which wires correspond to pins 4, 10 and 11.

I assume we are going to plug the cable into AUX1 (RD4 has 2 auxiliary inputs), so we should find which wires corresponds to pins 4, 10 and 11 (these are the only pins we need). If you are going to connect to AUX2 you need pins 7,1 and 2 accordingly. For the rest of this article I assume AUX1, but you can do it with AUX2 as well.
When you look at the connector holes, pin 1 is the upper right hole, pin 2 is the one to the left, etc... Beneath pins 1 to 6 we have pins 7 to 12. Since our connector is 5 pins width (2 rows of 5 pins, total 10 pins), we don't really have pins 6 and 12. We have only the blue pins:
6  -  5 -  4 - 3 - 2 - 1 
12 - 11 - 10 - 9 - 8 - 7


To find which wires corresponds to holes 4,10 and 11, you can stick a small paper clip in the pin hole and use multimeter to find which wire short circuits the paper clip and the exposed wire.

Extend wires 4, 10 and 11 by soldering

After we found the correct 3 wires, we should now extend theme. The length of extension is depends on your needs, but I guess 50cm extension is enough.

Drill holes in the plastic box and put the RCA sockets (optional)

This part is optional. You can skip the whole box thing if you don't care about the aesthetic or that you just hides your RCA behind the car dashboard.

Drill 3 holes: 2 for the RCA sockets and one for the wires input. My RCA sockets are 6mm, so I drilled 6mm holes. You can avoid drilling if you have the extra time of digging these holes using a screw...


Screw the sockets and the exposed cable end into the box:

Solder everything together

Wire from pin 4 should go to the right RCA, wire 10 to the left RCA, and wire 11 (ground) is common for both. In this picture, the orange wire is coming from pin 4, the blue is coming from pin 10, and the white is from pin 11:
This is the final result:


Release the RD4 unit and connect the cable

Releasing the RD4 unit is easy if you have release keys: just put the keys into the holes and pull easily.

Now we can connect the cable. AUX1 is marked as input C, AUX2 is input D. Pay attention to the cable orientation (wire 4 should reach pin 4, etc...). You should be able to see the numbers in the RD4 connector.

Put the RD4 unit back in place, make sure to wire the cable in the dashboard so you have access to the 2 RCA sockets. You can't test it before putting back the RD4 unit because the car anti-theft mechanism disables the car. Plug the iPod and enjoy - it should sound great! Note that if you are connecting through the iPod dock connector you should increase the volume, since the iPod does not amplify its audio out.

I mounted the black box near the passenger seat. This is how it looks like in my car:




Thanks to Amnon who wrote this forum post (in Hebrew), C4 owners ICE/Sat Nav etc forum, and Tzahi for helping me mount the black box.

Sunday, March 14, 2010

Backup to Amazon EC2 Using Spot Instances

Following this nice tutorial explaining how to automate backup to Amazon EC2 using rsync, I have decided to improve this script a little bit and add support for spot instances.

Spot instances are like any other EC2 instance, except the price you pay per hour is determined by demand. This should always be cheaper than running a regular instance, since if spot instance price was higher than regular instance, people would shift from spot instance while causing a price drop (in short: supply and demand). More on spot instances here: http://aws.amazon.com/ec2/spot-instances/. Today, for example, the cheapest instance is $0.085 per hour, while a spot instance is about $0.031 - more than 60% save. If you have a lot of backup to do, and you are running on slow internet connection, this might save you a significant amount.

To use the script you should first read the tutorial in Black Peper blog, and then update the script to this one. You should save the script as backup.bash if you want all features to work. See script inline comments for more details:

#!/bin/bash

# Amazon login parameters
export EC2_PRIVATE_KEY=~/.ec2/pk-.pem
export EC2_CERT=~/.ec2/cert-.pem

# EC2 tools path
export JAVA_HOME=/usr/lib64/jvm/sun-java-5.0u17/jre
export EC2_HOME=/root/ec2-api-tools-1.3-46266/

### Machine image you want to use as the base for the machine you want to start up, more on this later.
# ubuntu 9.1 32-bit for me
export amiid="ami-bb709dd2"

### SSH key to use to setup the machine with. In the EC2 console you need to setup an SSH key that you can connect to your new machine with as by default they do not allow access by any other means. This is my own keyname on the console.
export key="backup console"

### Where do launch your machine. N-Virginia for me, since it's cheaper.
export zone="us-east-1a"

### Local SSH key to connect to machine with. Location of the actual SSH key that you also put in the EC2 console.
export id_file="/root/backup_to_ec2/backupconsole.pem"

### Volume to mount to machine. The volume that we've previously created.
export vol_name="vol-12345678"

## Where to mount the volume on our new machine.
export mount_point="/mnt/vol"

### Device name for the mount
export device_name="/dev/sdf"

### Security group. To help me identify my machine, I use security groups as EC2 doesn't have real instance labels.
export group="backup"

### Maximum price for amazon spot instance
export price=".05"

# See if the backup is still running. If this script is already running then abort. This is necessary if you are running a cron job and want only one instance of this script at a time. You may delete it if you don't care for this check. It is important to name the script file 'backup.bash' for this check to work.
export pidof_out=`/sbin/pidof -x backup.bash`
num=`echo $pidof_out | wc -w`
if [ "$num" != "1" ]
then
echo "backup.bash is already running"
exit
fi
echo backup.bash is not running

#
# Start the instance
# Capture the output so that
# we can grab the INSTANCE ID field
# and use it to determine when
# the instance is running
#

echo Requesting spot instance ${amiid} with price of ${price}

${EC2_HOME}/bin/ec2-request-spot-instances ${amiid} --price ${price} -z ${zone} -k "${key}" --group ${group} > /tmp/a
if [ $? != 0 ]; then
echo Error requesting spot instance for image ${amiid}
exit 1
fi

export rid=`cat /tmp/a | grep SPOTINSTANCEREQUEST | cut -f2`

#
# Loop until the status changes to 'active'
#

sleep 30
echo Checking request ${rid}
export ACTIVE="active"
export done="false"
while [ $done == "false" ]
do
export request=`${EC2_HOME}/bin/ec2-describe-spot-instance-requests ${rid} | grep SPOTINSTANCEREQUEST`
export status=`echo $request | cut -f6 -d' '`
if [ $status == ${ACTIVE} ]; then
export done="true"
export iid=`echo $request | cut -f8 -d' '`
else
echo Waiting...
sleep 60
fi
done
echo Request ${rid} is active

#
# Loop until instance is running
#

echo Waiting for instance to start...
export done="false"
export RUNNING="running"
while [ $done == "false" ]
do
  export status=`${EC2_HOME}/bin/ec2-describe-instances ${iid} grep INSTANCE  | cut -f6`
  if [ $status == ${RUNNING} ]; then
    export done="true"
  else
    echo Waiting...
    sleep 10
  fi
done
echo Instance ${iid} is running.
#
# Attach the volume to the running instance
#
echo Attaching volume ${vol_name}

${EC2_HOME}/bin/ec2-attach-volume ${vol_name} -i ${iid} -d ${device_name}
sleep 15

#
# Loop until the volume status changes
# to 'attached'
#

export ATTACHED="attached"
export done="false"
while [ $done == "false" ]
do
export status=`${EC2_HOME}/bin/ec2-describe-volumes | grep ATTACHMENT | grep ${iid} | cut -f5`
if [ "$status" == ${ATTACHED} ]; then
export done="true"
else
echo Waiting...
sleep 10
fi
done

echo Volume ${vol_name} is attached

export EC2_HOST=`${EC2_HOME}/bin/ec2-describe-instances | grep "${iid}" | tr '\t' '\n' \
| grep amazonaws.com`

### Important trick here.
### 1. Because you will be starting up a different machine every time you run this script, you'll be forced to say yes to accepting the change of host for the SSH key, the options here make sure the doesn't happen and you can run this completely automated without human interaction.
### 2. Since we don't want to save the host SSH key, we will redirect the known hosts list to a temp file
export KNOWN_HOSTS='/tmp/known_hosts.$$'
rm $KNOWN_HOSTS

### This line logs on and mounts our volume to our machine.
ssh -i ${id_file} -o "StrictHostKeyChecking no" -o "UserKnownHostsFile=$KNOWN_HOSTS" ubuntu@$EC2_HOST "sudo mkdir /mnt/data-store && sudo mount ${device_name} /mnt/data-store"

### Run rsync, whatever options you'd like, here are a couple of examples I use.
rsync -e "ssh -i ${id_file} -o 'UserKnownHostsFile=$KNOWN_HOSTS'" --rsync-path "sudo rsync" --delete -avz /mnt/vg1/lv0/backup ubuntu@$EC2_HOST:/mnt/data-store/backup

### Clean up. Disconnect the volume
ssh -i ${id_file} -o "UserKnownHostsFile=$KNOWN_HOSTS" ubuntu@$EC2_HOST "sudo umount -d ${device_name}"

### Detach volume from machine
${EC2_HOME}/bin/ec2-detach-volume ${vol_name} -i ${iid}

### Shutdown instance
${EC2_HOME}/bin/ec2-terminate-instances ${iid}