I'm currently investigating ways to replace Apache httpd with Nginx on srcbox.net. One of the services that runs on the webserver is Trac for hosting a wiki-based website and bug tracking for software projects. Because I'm already using Phusion Passenger for hosting Ruby on Rails projects, I decided to also give it a try to host WSGI applications together with Nginx.

Please note that this post is quite similar to the one written by Graham Edgecombe about the same subject. I found a few things missing and notable so I decided to do my own little writeup on what I had to do to get things working.

Trac Setup

As the actual Trac setup is already present on my server and I just wanted to replicate a similar setup in my testing VM, I setup Trac like this:

• Installed package trac, I used version 0.12.x which is available via Debian Backports
• Added a system user that owns all trac project files and is used to run the WSGI webserver processes:

  adduser --system --home /var/lib/trac --group --disabled-password --disabled-login trac
  

• Initialized a trac environment below trac home:

  trac-admin /var/lib/trac/project initenv
  

• Deployed the webserver files using:

  trac-admin /var/lib/trac/project deploy /var/www/trac/
  

• Added symlinks to make Phusion Passenger accept Trac as a WSGI application:

  ln -s /var/www/trac/htdocs /var/www/trac/public
  ln -s /var/www/trac/cgi-bin/trac.wsgi /var/www/trac/passenger_wsgi.py
  

Installing Nginx

Nginx installation is quite a bit of work if you're used to a single aptitude install package, all features/modules have to be enabled/added at build-time which means rebuilding Nginx if you want to use Phusion Passenger. Because I plan to move authentication to a central place soon I also added Nginx HTTP Auth PAM to the mix. Furthermore I'm also tinkering with the idea to give owncloud a spin so WebDAV support is probably helpful too.

The installation basically went like this (rough steps, this is not a place for step-by-step tutorials):

• Downloaded latest stable Nginx release
• Downloaded latest ngx_http_auth_pam_module release
• Unpacked both in a convenient place
• Installed Phusion Passenger via rubygems (I used ruby and gem from ruby1.9.1 in Debian Squeeze)
• Ran passenger-install-nginx-module
• Chose 2. No: I want to customize my Nginx installation.
• Told the installer where to find unpacked Nginx sources
• Added additional configure arguments. For Auth PAM and optionally needed WebDAV support I used:

  --add-module=/path/to/unpacked/ngx_http_auth_pam_module --with-http_dav_module
  

• Grabbed an Espresso until the build was finished (didn't really take long)

On Debian it's probably helpful to install a fake webserver package to satisfy dependencies for other packages that depend on http-server. I used equivs to generate myself an nginx-dummy package.

Configuring Nginx

For starting up Nginx I simply grabbed the init-script from the Debian source package of nginx, , put it into /etc/init.d/, adapted the paths to my Nginx install in /opt/nginx and installed it in the default runlevels using insserv nginx.

Adding a virtual host to Nginx that serves Ruby on Rails or a WSGI application is usually very easy, for Trac however a tiny bit more care had to be taken...

Pitfall 1: HTTP Auth

With Trac there's one catch: HTTP Authentication. It took me quite some time to find out why HTTP-Auth worked fine according to the Nginx access.log but Trac still thought I wasn't logged in: Nginx needs to forward the user name to the WSGI application. For Phusion Passenger this can be done using the following line:

passenger_set_cgi_param REMOTE_USER $remote_user;

Pitfall 2: Location Blocks

Nginx uses location blocks for different behavior depending on the url. For Trac the /login path should trigger HTTP authentication. Together with Phusion Passenger all I got after successful login however was a simple "404 Not Found" page.

It turned out that for location blocks one has to enable Phusion Passenger again, a prior passenger_enabled on; line in the parent block does not apply to the inner location block.

Final Nginx Config

The complete host configuration for my test setup that contains all needed things to avoid the above pitfalls looks like this:

server {
    listen                  80;
    server_name             trac.domain.lan;
    root                    /var/www/trac/public;
    passenger_enabled       on;
    # Forward HTTP-Auth user to WSGI
    passenger_set_cgi_param REMOTE_USER $remote_user;

    location /login {
        auth_pam                "Trac on domain.lan - Restricted Access";
        # Use the same PAM service as dovecot for now
        auth_pam_service_name   "dovecot";
        # Reenable Passenger to get a redirect after login instead of a 404
        passenger_enabled       on;
        # Forward HTTP-Auth user to WSGI
        passenger_set_cgi_param REMOTE_USER $remote_user;
    }
}

What next?

This setup is probably still far from complete. Most notable things that I did not setup or test yet include:

• Paths for a manual Nginx install do not obey the FHS at all, logs and temporary files end up on the wrong partition which is a bad idea security wise.
• Log rotation is missing. I can probably grab parts from the Debian Nginx package again.
• Serve multiple projects/environments with the above webserver setup
• Replace the rather ugly HTTP Authentication with Trac Account Manager plugin
• Use TLS and enforce login to happen encrypted only

Of course Trac is by far not the only web-application, I'm probably going to document a few other things I have installed as part of my new Nginx test installation :)

After about two years of complete TV absence both due to technical reasons (TV socket in wrong room on the other side of my flat) as well as the quality of the programme (ads, fake reality shows, lack of real movies), I nevertheless decided to get myself a network-enabled TV-tuner. Being network-enabled was both an easy way to get around the need for new cabling as well as making TV available to all the machines scattered throughout the rooms.

Hardware

The most interesting device I found and also ordered is a HDHomerun HDHR3-EU from SiliconDust, a small black box with coax input for TV (DVB-C or DVB-T), the usual 5V power-supply and an ethernet port. It contains two tuners so two channels can be watched and/or recorded in parallel.

The good thing about HDHomerun devices is, that they work with a broad range of software. They are also fairly well documented and SiliconDust even has created a library and a cross-platform commandline tool to configure their devices. In fact, the software is even available in Debian Squeeze (hdhomerun-config and libhdhomerun).

Getting the device running network-wise was only a matter of plugging things together. It automatically grabbed an IP-address via DHCP and the hdhomerun-config tool could immediately find the device.

Software

At first I was mostly interested in getting the box working with EyeTV 3 which I used together with my FireDTV in the past. A first channel-scan however did not find a single channel. After some cable unplugging, checking again with trusty FireDTV and attempts to manually set a frequency on one of the tuners I found two settings that were quite important for my DVB-C setup:

$ hdhomerun_config DEVICEID set /sys/dvbc_modulation \
    "a8qam256-9600 a8qam64-6900 a8qam128-6900"
$ hdhomerun_config DEVICEID set /tuner0/channelmap eu-cable
$ hdhomerun_config DEVICEID set /tuner1/channelmap eu-cable

The first command sets the DVB-C modulations used when autotuning on a certain frequency. Unfortunately the current HDHR3 firmware (20111025) seems to have an off-by-one bug that drops the last modulation in the list and I only tried setting the two needed values at first. After adding the unneeded QAM128 value at the end I had all modulations set that I needed. According to the very helpful support-folks at #hdhomerun on FreeNode this should already be fixed in the next firmware release.

The other two lines set the range of channels used when doing a full channel-scan. This might be unneeded for software that has its own list of "interesting" frequencies but at least for scanning with the commandline tool this has a huge impact on the frequency-range that gets scanned.

As I later found out, most of this would have propably been set by the first run of the HDHomerun Config GUI which I did not install on my Mac, d'oh.

First Impressions

So far I'm pretty happy with my current TV setup. Watching TV while recording something else works fine and so does EPG, plus I even found some new HD channels since I last had TV at home. The initial setup was a bit tricky but the extensive documentation as well as the geek-friendly support channel (IRC) pretty much make up for all the tinkering. Also I now have a new network-enabled toy to play with so depending on my free-time maybe I can whip up some helpful software for these devices (GKrellM HDHomerun status plugin anyone?).

While maintaining GKrellM for Windows I have found several things in the code that I wanted to improve. Apart from the Preferences Dialog which needs quite a bit of cleanup, the biggest offender both in terms of portability as well as readability is the socket code.

Currently, the socket handling in gkrellm and gkrellmd is really low-level, it uses plain socket API, i.e. raw calls to accept(), send(), select() etc. GLib and GIO in particular have support for abstracting most of this away. They also feature asynchronous versions of most of the blocking system-calls around socket handling which is a good thing especially for GUI-applications.

So I started to get this into GKrellM...

Starting with an event loop

I began porting with gkrellmd first because this is presumably the simpler case. The biggest change needed for gkrellmd was to make it use a GLib mainloop instead of its own event loop, that one was limited to socket events.

The old loop looked a bit like the following, the complete version is available at server/main.c in git.

while (1) {
    /* Block until at least one filedescriptor needs attention or
       maximum wait time was reached */
    result = select(max_fd + 1, &test_fds, NULL, NULL, &tv);

    /* Iterate over filedescriptors and check for activity */
    for (fd = 0; fd <= max_fd; ++fd) {
        if ( is_server_fd ) {
            /* activity on listening socket */
            client_fd = accept(...);
            /* host lookup, reverse lookup etc. */
        } else {
            /* activity on client socket, try reading */
            ioctl( fd, FIONREAD, &nbytes );
            gkrellmd_client_read( fd, nbytes );
        }
    }

    /* Update krells, send updated data to clients */
    gkrellmd_update_monitors();
}

This loop works remarkably well most of the time, but it has a few shortcomings:

• updating monitors has to wait for all socket-work to finish
• hostname lookups while accepting new clients block the whole loop until they finish or a system-dependent timeout is reached
• errors of ioctl() are not checked for (also nbytes is never initialized, oops)
• the time that select() waits also defines the update frequency for the monitors

Especially the possible blocking while accepting a new client may affect monitor update timing and keep already connected clients from receiving updates. This is usually not a big problem if clients are on the same network and name lookup for the network is working reliable but that doesn't mean things can never go wrong.

The GLib way of creating, running and deleting an event loop that already updates gkrellm monitors regularly looks like this:

g_timeout_add(1000 / _GK.update_HZ, gkrellmd_update_monitors, NULL);
gk_main_loop = g_main_loop_new(NULL, FALSE);
g_main_loop_run(gk_main_loop);
g_main_loop_unref(gk_main_loop);

Of course this does not yet have all the features of the old gkrellmd event loop but I'll tackle that in my next post about me diving into socket-communication using GIO.

After quite some hiatus (more than two years according to my last post) I decided to give this site a facelift and actually start some "real" blogging, meaning more frequent posting even in the absence of new software releases.

The Why

Main reason for getting this site back on track is that I finally started having some real free time again (weekends that actually deserve being called "weekends" instead of "extended workdays") and a refreshed interested to get back into Open Source development.

Visiting this years Desktop Summit in Berlin was quite inspiring with all the interesting talks and projects. Especially the mixture of both Gnome and KDE people was quite cool. Having a look at both camps shows that there's similar problems on both sides (where's the desktop heading towards with so many new mobile gadgets, keeping or dropping cross-platform support) as well as (continued) plans to integrate both desktop "worlds". The most interesting project in terms of integration for me right now is pk11-kit: GKrellM could benefit from good key/trust management once it gets TLS connection support (stay tuned, I'll definetely come back to this topic).

The What

So what is to be expected here in the future? For a start, the projects formerly hosted here will stay as a static archive. Although all projects are unmaintained by now and I don't think anybody still uses software like Floodcast, there still might be some interest in getting the source code and maybe reusing parts of it.

For the future, I plan to post more about what I'm working on or wasting my time with. This includes for instance code snippets for interesting problems or chronically undocumented APIs (yes, I'm looking at you GLib). As I'm also some sort of admin for quite a few systems, I'll also "document" my endeavours of software I haven't worked with before (possible first candidates include Nginx, Unbound and NSD).