metz·log

Embedding GtkBuilder files in GTK+ 2.x

·

As part of modernizing GKrellWeather (a weather plugin for GKrellM) I changed the hand-coded configuration UI to a file that can be edited visually using Glade. GTK+ 2.24 already supported generating UI from XML files directly via the GtkBuilder API and editing is possible as long as you have Glade 3.8.x around (Glade dropped support for GTK+ 2.x afterwards, a pity Debian/jessie only contains Glade 3.18.x).

Because most GKrellM plugins are pretty self-contained, loading the UI from an external file was not what I had in mind though, so I started looking for ways to embed the UI file directly into the plugin binary. After looking a bit into the gritty details of using ld or objcopy I discovered that GLib already covers this scenario, enter GResource.

Using GResource

The documentation for GResource already shows the steps of how to get files embedded. It's actually very similar to the Qt Resource System, just the tool names and file syntax differ.

  • Create an XML file (by hand, eek) that explains which files to embed and what resource path they should have.
  • Call glib-compile-resources on the XML file and tell it to generate C sources (--generate-source)
  • Add the name of the generated file to your project source-list as usual

In the end I had my UI file inside the plugin already, but it still had to be loaded somehow.

Tieing GtkBuilder to GResource

The next question was how to use GtkBuilder and GResource together and searching for that combo only lead me to GTK+ 3.x examples. Since GTK+ 3.10 doing this is as easy as calling gtk_builder_new_from_resource() or creating your own GtkWidget subclass and call gtk_widget_class_set_template_from_resource().

Because GKrellM is currently still bound to GTK+ 2.x and the mentioned functions were not present back then I began digging a bit on how other applications do this on GTK+ 2.x, quite a lot of Read-The-Fine-Source was caused by that.

In the end it turned out to be fairly easy if you know how to get the data out of a global GResource and push data (including length) into a GtkBuilder object. The following code snippet does exactly that.

/* Simple GTK+2 variant of gtk_builder_add_from_resource() */
static guint builder_add_from_resource(GtkBuilder *builder
    , const gchar *resource_path
    , GError **error)
{
    GBytes *data;
    const gchar *buffer;
    gsize buffer_length;
    guint ret;

    g_assert(error && *error == NULL);

    data = g_resources_lookup_data(resource_path, 0, error);
    if (data == NULL) {
        return 0;
    }

    buffer_length = 0;
    buffer = g_bytes_get_data(data, &buffer_length);
    g_assert(buffer != NULL);

    ret = gtk_builder_add_from_string(builder, buffer, buffer_length, error);

    g_bytes_unref(data);

    return ret;
}

void gkrellweather_config_ui_create(GtkWidget *parent)
{
    GtkBuilder *builder;
    GError *error;
    GtkWidget *config_widget;

    builder = gtk_builder_new();
    error = NULL;
    if (!builder_add_from_resource(builder, "/org/gkrellweather/config.ui", &error)) {
        g_warning("Could not build config ui: %s", error->message);
        g_error_free(error);
        g_object_unref(G_OBJECT(builder));
        return;
    }

    config_widget = GTK_WIDGET(gtk_builder_get_object(builder, "config_widget"));

    /* And off you go with that shiny config_widget… */

    g_object_unref(G_OBJECT(builder));
}

The actual code to get GResource and GtkBuilder working together can be summed up as:

User Interface changes for GKrellWeather

After all this fiddling now for the actual results. Here is a screenshot showing the main GKrellWeather configuration tab with the old layout that had been created in C years ago

GkrellWeather Configuration UI - Before

The layouting clearly is a bit odd, especially the radiobuttons and their labels are quite uncommon since it is difficult to tell which group they belong to.

And here is the current state based on a Glade file targetting Gtk+ 2.24.

GkrellWeather Configuration UI - After

The dialog layout now starts following GNOME Human Interface Guidelines a bit more, at least in terms of alignment and spacing of elements.

Right now the UI also lost a few features temporarily, i.e. the link to find your station id or the unit names for switch and update intervals. With the UI now being editable via Glade, fixing the last remaining bits should be a lot faster than it was with hand-crafted C code before. The next step for GKrellWeather however is improving functionality itself and hopefully see an official release soon.

Seafile on Nginx with uWSGI

·

I've been using Seafile as my personal cloud storage for quite some time now and it's working really really well. Since I've departed quite a bit from the standard setup instructions for the web frontend (Seahub) I'm going to document the setup a tiny little bit. For complete Seafile setup instructions that include all components and not only the web frontend please refer to the Seafile Server Manual.

Running Seahub as a WSGI application

The standard setup instructions recommend to run Seahub behind Nginx as a FastCGI application. However internally Seahub is actually a WSGI application which is the standard way to communicate with Python based web-applications. Since I'm using uWSGI for managing and running several other services already I opted to skip the FastCGI wrapper in front of Seahub and let uWSGI start the web application with the correct environment instead. This also means seahub.sh is not used anymore.

The configuration I use for uWSGI and Seafile Server 3.1.6 looks like this:

/etc/uwsgi/apps-available/seafile.ini

[uwsgi]
plugins = python
uid = seafile
gid = seafile

topdir = /srv/seafile
env = CCNET_CONF_DIR=%(topdir)/ccnet
env = SEAFILE_CONF_DIR=%(topdir)/seafile-data
env = SEAHUB_LOG_DIR=%(topdir)/logs

installpath = /srv/seafile/seafile-server-latest
env = PYTHONPATH=%(installpath)/seahub:%(installpath)/seahub/thirdpart: \
  %(installpath)/seafile/lib64/python2.6/site-packages
env = DJANGO_SETTINGS_MODULE=seahub.settings

module = seahub.wsgi:application

Most environment variables set here are normally set in seahub.sh before starting the Python-based HTTP- or FastCGI-Server. The job of configuring the environment is now done by uWSGI and the above configuration file instead. For Python 2.7 the PYTHONPATH contents obviously need to be adapted.

The only exception in this configuration is DJANGO_SETTINGS_MODULE which is normally set inside seahub/wsgi.py early on. Unfortunately this did not seem to work when running the application from uWSGI (probably since the app cannot permanently modify the environment) so I had to set this inside the uWSGI configuration file.

Configuring Nginx

For Nginx the configuration is almost identical to the Seafile FastCGI setup documented in the manual. The only change is that the root location element is a lot simpler and uses uwsgi_pass instead of fastcgi_pass.

/etc/nginx/sites-available/seafile.conf

server {
    listen 80;
    server_name seafile.example.com;
    return 301 https://$server_name$request_uri;
}
server {
    listen 443 ssl;
    server_name seafile.example.com;

    location / {
        include uwsgi_params;
        uwsgi_pass unix:/var/run/uwsgi/app/seafile/socket;
    }

    location /seafhttp {
        rewrite ^/seafhttp(.*)$ $1 break;
        proxy_pass http://127.0.0.1:8082;
        client_max_body_size 0;
    }

    location /media {
        root /srv/seafile/seafile-server-latest/seahub;
    }
}

Running (almost) anything on Nginx with uWSGI

·

This is a sort of follow-up post to Running Trac on Nginx with Phusion Passenger. As my hosting needs have slightly changed lately I now have no immediate need for running Ruby, Rails or Rack applications. I'm lately using more and more web applications but they are based on different technologies. The current list looks like:

From the list above, Phusion Passenger currently only supports Trac. Also it doesn't seem to work with the Firefox Sync Server as its standard installation recommends virtualenv and Passenger always uses the system-wide Python installation unless you want to apply ugly hacks (loading another interpreter from a running Python script).

So overall Phusion Passenger did not look like a good match for my needs anymore and I started looking for something else that could handle as many technologies as possible without me needing to setup a different application server for each technology (and learning a different configuration syntax for each of them).

Enter uWSGI

It turns out that there's already something that does handle most (if not all) of the above mentioned languages or frameworks and it's called uWSGI. Despite its name, uWSGI supports far more than running Python-based UWSGI applications thanks to its plugin system and a truckload of plugins that are part of the standard distribution. This also includes Rack-based applications (probably includes Rails as well), PHP and even plain old CGI.

Unfortunately uWSGI is not yet available in Debian Squeeze (stable) but it's part of Debian Wheezy (testing). Because Wheezy is currently in feature freeze and has proven to be "stable enough" on all my Linux boxes I've decided to use Debian Wheezy for my new root-server.

Installing uWSGI on Debian Wheezy

Installing uWSGI is just a matter of apt-get install uwsgi-core and then add any of uwsgi-plugin-foo packages to the mix. I started with uwsgi-plugin-python and uwsgi-plugin-cgi.

For some reason Debian does not package the PHP plugin for uWSGI, fortunately building it manually is not that much work:

  • Install packages libphp5-embed and php5-dev
  • Fetch uwsgi package sources using apt-get source uwsgi
  • Copy debian/buildconf/uwsgi-plugin.ini.in to debian/buildconf/uwsgi-plugin.ini and replace @@curdir@@ with the absolute path to the source directory
  • Execute python uwsgiconfig.py --plugin plugins/php debian/buildconf/uwsgi-plugin.ini to build the plugin
  • The resulting plugin_php.so can now be copied to /usr/lib/uwsgi/plugins/

Please note that every time libphp5-embed received an update so far I also had to rebuild the plugin. I don't know exactly why that's the case but I guess it's a trick to make me submit a patch to the Debian uWSGI packager(s) ;)

Configuring uWSGI and Nginx

In the following I'll show a few configuration examples how I got different applications and languages working with uWSGI and Nginx.

Because all this is on Debian Wheezy every uWSGI app creates a socket at /var/run/uwsgi/app/APPNAME/socket. For more information about the default configuration of an application just check /usr/share/uwsgi/conf/default.ini and uWSGI Configuration Options.

In my examples all uWSGI apps run as their own user instead of www-data. This is of course not needed but should be a little bit more secure. For even more separation it would probably make sense to add chrooting and setting POSIX Capabilites as documented in the Securing uWSGI section of uWSGI documentation but I'll omit that to keep the examples small.

Cgit on Nginx with uWSGI

Getting a CGI application like cgit working is quite easy, at least if there's only one CGI binary to execute.

/etc/uwsgi/apps-available/cgit.ini

[uwsgi]
plugins = cgi
chown-socket = www-data:www-data
uid = cgit
gid = cgit
processes = 1
threads = 8
cgi = /usr/local/lib/cgi-bin/cgit.cgi

For Nginx it's just a matter of adding a new virtual host and pointing it to the socket created by the above configuration:

/etc/nginx/sites-available/com.example.git.conf

server {
    root /srv/www/com.example.git;
    server_name git.example.com;
    location / {
        try_files $uri @cgit;
    }
    location @cgit {
        include uwsgi_params;
        uwsgi_modifier1 9;
        uwsgi_pass unix:/var/run/uwsgi/app/cgit/socket;
    }
}

Trac on Nginx with uWSGI

Setting up Trac is very similar, except that defining the entry point of a Python-based WSGI application works a bit different:

/etc/uwsgi/apps-available/trac.ini

[uwsgi]
plugins = python
chown-socket = www-data:www-data
uid = trac
gid = trac
env = TRAC_ENV=/srv/trac/projectname
module = trac.web.main:dispatch_request

Adding a virtual host to Nginx is also only a few lines:

/etc/nginx/sites-available/com.example.trac.conf

server {
    root /srv/www/com.example.trac;
    server_name trac.example.com;
    location / {
        include uwsgi_params;
        uwsgi_pass unix:/var/run/uwsgi/app/trac/socket;
    }
}

Piwik on Nginx with uWSGI

Getting Piwik running is easy in terms of uWSGI and shows little changes except for the plugins key:

/etc/uwsgi/apps-available/trac.ini

[uwsgi]
plugins = php
chown-socket = www-data:www-data
uid = piwik
gid = piwik
cheaper = 1
processes = 4

Now the Nginx configuration is a bit more elaborate. Parts of it are based on François Lefèvre's post about Piwik Hardening with Nginx and PHP-FPM and perusio's piwik-nginx config example at GitHub.

/etc/nginx/sites-available/com.example.trac.conf

server {
    listen 80;
    root /srv/www/com.example.piwik;
    server_name piwik.example.com;
    index piwik.php;
    # For administrative access
    location = /index.php {
        include uwsgi_params;
        uwsgi_modifier1 14;
        uwsgi_pass unix:/var/run/uwsgi/app/piwik/socket;
        allow 127.0.0.1; # only via ssh tunnel
        deny all;
    }
    # For public access
    location = /piwik.php {
        include uwsgi_params;
        uwsgi_modifier1 14;
        uwsgi_pass unix:/var/run/uwsgi/app/piwik/socket;
    }
    # Any other attempt to access PHP files is forbidden
    location ~* ^.+\.php$ {
        return 403;
    }
    # Redirect to the root if attempting to access a txt file.
    location ~* (?:DESIGN|(?:gpl|README|LICENSE)[^.]*|LEGALNOTICE)(?:\.txt)*$ {
        return 302 /;
    }
    # Disallow access to several helper files.
    location ~* \.(?:bat|html?|git|ini|sh|svn[^.]*|txt|tpl|xml)$ {
        return 404;
    }
    # Disallow access to directories
    location ~ ^/(config|core|lang|misc|tmp)/ {
        deny all;
    }
}

Good bye Apache and mod-fcgid

uWSGI is a real breeze to configure. Combining it with Nginx gave me an IMHO pretty clean solution for separating web applications from eachother and having a single point for configuring apps regardless of what technology they use.

Especially the configuration part got me sold on uWSGI after I looked back at all the hoops I had to jump through in the past with apache fcgid/fastcgi modules, suexec, php-fpm and numerous wrapper-scripts to get fastcgi and suexec to play nicely together.

Finally I also have gained easier control over the number of spawned processes, something which wasn't so straightforward with mod-fcgid and php5-cgi.