Ephemeral software joys

I think I've arrived in Alicante the best period of the year. Between the festival de St. Juan, the Corrida and the Spain winner of the football Euro Cup I couldn't have been more lucky.

Check out some pictures:

and

Friday 23rd May 2008 was my last working day at Vega. Unordered sounds and images from the latest two years and a half of my life are flustering in my mind.

I met extraordinary people and I'm taking pride in having worked with them.
I've been at CNES, at EUMETSAT and at European Space Agency centers like ESTEC, ESOC and ESAC.

I won't never forget how minuscule I felt in front of the SMOS Science Advisory Group members when presenting a software I developed for the SMOS mission, nor the impressive amount of knowledge of the Open Geospatial Consortium (OGC) people.

And most of all I won't forget the words of appreciation and incouragement I received by my now ex-collegues.

It is truly an outstanding experience to work (and live) in a foreign country but it's over now, I'm turning the page, time to move to a new adventure.

Recently I wanted to extract the sound track from a MP4 file.
You actually need a single command-line to do that, either using Mplayer or GStreamer.
It's amazing what you can do with GStreamer pipelines, but unfortunately I ran into several problems trying to play H.264 video files with AAC audio (the files are produced by a Sanyo camera and make Totem crash after about few seconds...) so I gave up and I took a look at MPlayer. After all I really didn't need all the GStreamer functionalities.
A single line actually does the job:

mplayer -ao pcm:file=$OUTPUT_FILE $INPUT_FILE

Couldn't be easier.

Nevertheless, I wanted to command the extraction from a GUI as it makes it more user friendly for people that don't deal with shells everyday. Additionally, I didn't want the MPlayer window to popup, I actually wanted to embed Mplayer inside my application.

Again, it turns out that a single line does the job:

mplayer -ao pcm:file=$OUTPUT_FILE -wid $WINDOW_XID

The -wid option tells Mplayer to connect to the window represented by the ID $WINDOW_XID.
This ID is actually the identifier given by the X Server to the window.
We'll see later how to retrieve it.

With this setup, the video is shown in the application, the problem is that there is no indication about the progress. What I wanted was to periodically ask to MPlayer the status of the playback, and displaying it in a progress bar. Checking the MPlayer's man page I discovered that it can be controlled by putting it into a slave mode and by sending him the commands through a named pipe. Bingo !

To start the mplayer in slave mode, I modified the command line to look like this:

mplayer -ao pcm:file=$OUTPUT_FILE -wid $WINDOW_XID -slave -idle -input file=$FIFO

where FIFO is the absolute path of the named pipe where the commands are sent
(for a complete list of commands, type mplayer -input cmdlist).

On the Python side, you have to embed the video in a GtkDrawingArea.
The documentation for this component just says:
The GtkDrawingArea widget is used for creating custom user interface elements.
It's essentially a blank widget; you can draw on widget->window

It is actually everything I need, as the only thing required to do is to get the X ID of the window and pass it to Mplayer. You do it like this:

xid = canvas.window.xid # where canvas is an instance of GtkDrawingArea

We have almost everything required to start Mplayer, the only missing thing is the FIFO.
In Python, you create a FIFO using

os.mkfifo("path_to_fifo")

The full code to start Mplayer in another process would then be:

command = "mplayer -ao pcm:file=%s -wid %i -slave -idle -input file=%s" % (outputFile,xid,fifo)
command = command.split().append("input_video_file.mp4")
subprocess.Popen(command, stdout=logfile, stderr=logfile)

I added the input file after the call to split because if there are whitespaces in the filename or in a parent directory, then the filename would break down in multiple tokens with the call to split.

Now that MPlayer has started, we have to tell him what to do. To do that, we write to the pipe:

mplayerClient = open(FIFO,"w")
mplayerClient.write("play\n") # commands must ends with a newline

(Actually, I still haven't get MPlayer to not start automatically the playback, even if I'm using the -idle option. Please post a comment if you can shed a light on the subject...)

At this point the application is showing the video, but as we said before, we also want to update a progress bar to show the progress, and to do that we've to repeatedly ask the status to Mplayer.

The easiest way to do that, since I was using Gtk, is to use the gobject.timeout_add call, like this:

# call the myUpdateFunction each UPDATE_INTERVAL milliseconds
gobject.timeout_add(UPDATE_INTERVAL, myUpdateFunction)

and myUpdateFunction is defined as:

 
def myUpdateFunction():
    keepGoing = True
    mplayerClient.write("get_percent_pos\n")
    mplayerClient.flush() # really important, or commands won't be always sent !
    line = logfile.readline()
    if line and line.startswith("ANS_PERCENT_POSITION"):
    # +1 because progress starts from 0, it would end to 99 otherwise
        progress = int(line[line.index('=')+1:-1]) + 1
        progressBar.set_text("%s %%" % progress)
        progressBar.update(progress/100.0)
    else:
        keepGoing = False
 
return keepGoing
 

That's it, progressBar is a normal GtkProgressBar object and logfile is where I previously redirected Mplayer stdout and stderr.

The above code won't actually works out of the box as there are some corner cases you have to deal with, for example it can happen that the update function is called before MPlayer starts to play the video (which should actually happen with the -idle option, but I couldn't get it to work). In that case line would be an empty string and the playback will immediately stop as myUpdateFunction will return False.

You can find the complete but fairly hackish code here:
AudioExtractor.zip

The program also pops up a dialog asking you if you want to burn the file with Serpentine or save it to a different location. I can't guarantee it will work for you as I've been written it mostly as an educational tool.

Piopiotahi is the Maori word for Milford Sound, one of the most idyllic places I've ever been.

Pictures are really not enough to describe this place, but here the ones I prefer the most:

More pictures I took in New Zealand.

I just filled up my hard-disk and I didn't quite know what was actually filling it, but it was so trivial to find out with a simple python script:

 
 
import os
from os.path import join
 
def getSize(path):
    for f in os.listdir(path):
        fullPath = join(path,f)
        if os.path.isdir(fullPath):
            getSize(fullPath)
        elif os.path.islink(fullPath):
            pass
        else:
            size = os.stat(fullPath)[6]
            print "%d tt %s" % (size,fullPath)
 
if __name__ == '__main__':
    import sys
    if len(sys.argv) > 1:
        getSize(argv[1])
    else:
        getSize(".")
 

Usage: diskusage.py | sort -nr > out.txt

gedit is a pretty cool text editor for the GNOME desktop environment.

The problem is that sometimes I found myself using it to edit XML documents but there is no way to pretty print them.

I found yesterday a quick solution to this problem thanks to the External Tools support in gedit and a small utility called xmlindent.

Once you have installed xmlindent click on Tools->External tools, and on the dialog enter the following informations:

Of course you can customize the options as you wish, the displayed configuration just indents the whole document and replace the unindented one.

To use it just click External Tools -> xmlindent.