Taschenorakel.de

Postings in May 2011

Writing QML-based input methods for Maliit

This week I pushed a simple QML-based virtual keyboard to our MeeGo Keyboard repository. It's functional, but don't expect too much. This functionality will most likely never arrive in MeeGo 1.2, sorry folks.

Getting Started

Let's continue with the good parts instead: The required C++ code was kept to an absolute minimum and in fact it's only needed to wrap the QML resources into a Qt/C++ plugin that can be loaded by our framework. If you want to write your own QML-based input method plugin for Maliit, then this is the required interface:

class MyQmlBasedInputMethodPlugin
    : public QObject, public MInputMethodQuickPlugin
{
    Q_OBJECT
    Q_INTERFACES(MInputMethodQuickPlugin MInputMethodPlugin)

public:
    MyQmlBasedInputMethodPlugin();

    //! \reimpl
    virtual QString name() const;
    virtual QString qmlFileName() const;
    //! \reimpl_end
};

The plugin's name is used for a specific GConf key. Once set to that name, the framework will load this plugin as the currently active one:

$ gconftool-2 -s /meegotouch/inputmethods/plugins/onscreen -t string "WhateverNameYouCameUpWith"

qmlFileName returns the (absolute) file name (or URL) to your main QML file. The framework will set up the required QML environment and try to load this file.

QML components can interface with the application through a set of properties and slots, tied to a root context named MInputMethodQuick. It exposes the following read-only Qt properties:

Additionally, QML components can invoke the following Qt slots:

Please refer to the header file of MInputMethodQuick to access the full documentation (sorry, not available in MeeGo API docs).

Also, big thanks to Kimmo Surakka and Miika Silfverberg for all their help with this side project.

Self Compositing

Testdriving Maliit on the N900

Any QML plugin using MInputMethodQuickPlugin will support self compositing out of the box. That is, instead of letting the system compositor blend the application window and virtual keyboard window together into one, the virtual keyboard bypasses the system compositor and redirects the application's window to itself. This usually improves raw FPS but also cuts down on latency. Try running the MeeGo Input Methods service like so:

$ meego-im-uiserver -use-self-composition

This should result in a noticable speedup for your QML plugin. Self compositing is tricky, as it requires a well-behaving window manager (and believe it or not, mcompositor is the only one available getting it nearly 100% right). Usually an additional parameter combination of -manual-redirection, -bypass-wm-hint and -software will do the trick. For MeeGo 1.3, we want to activate self compositing by default, so it's good to know that your QML-based plugin will support this useful feature for free.

A QML-based virtual keyboard for Fremantle

MeeGo Keyboard Quick available in Maemo's devel-extras repo.

As part of the exercise I also packaged MeeGo Keyboard Quick for Fremantle - it's available from Maemo's extras-devel repository. Be warned though, this is a rough tech demo and as such, fairly useless. A nice advantage of it is that it allowed me to package a MeeGo Input Methods version for Fremantle that is completely free of any MTF/libmeegotouch components.

How to testdrive:

Instead of the test application, one should be able to use it in any other Qt application, as long as the MeeGo Input Methods service is running and the QT_IM_MODULE environment variable is set correctly.

To fight wrong expectations: I won't have the time to polish or to maintain this port, and from my first tests it would need undefined amounts of integration work to get it to run properly in Fremantle, starting with proper GTK+/Hildon support. If you are, however, interested in furthering this port, then please don't hesitate to contact us.

Suspend broken for X200 and X201 Thinkpad in Natty

In case you wonder why suspend is broken for you after upgrading to Natty: Good chances tpm_tis is bugging you. Quite some hits when searching in Launchpad. Seems LP#770272 lists a good workaround. Have to admit I did not try it. Not knowning what benefit a TPM gives me I just blacklisted the kernel module:

echo blacklist tpm_tis | sudo tee /etc/modprobe.d/blacklist-tpm_tis.conf

Qt Contributors' Summit

I am attending the Qt Contributors' Summit.

A bit odd for something with my background? Does that mean I am leaving the GNOME universe?

No. It just happens in Berlin, and I've just spent lots of time on letting QtContacts use some awesome GNOME technology (tracker). On the summit I'll try to convince some Qt core guys, that maybe UTF-8 would be a much better choice for the Linux port of Qt. It would improve interaction with kernel, DBus and GNOME libraries so much. Well, and maybe I can get them to consider more reasonable memory management for QObject: With Qt leaving the GUI corner it's simple parent-ownership model doesn't fit anymore. QtQuick already skips that obsolete model. Now let's also let C++ components benefit.

PS: If someone ever wants to modernize libebook, then looking at QtContacts API is a good exercise. It was designed to explicitly fix the issues we had with libebook during Fremantle. Actually even thought of making a GIR typelib for QtContacts - but that's a different story and maybe even doesn't make sense.

GFX performance pattern

In case you struggle with your GFX performance: Michael just nicely wrote down a receipe to dramatically boost performance in non-trivial canvas based applications. Old story for some, hot stuff for others.

Painting Strategy in MeeGo's Virtual Keyboard

I've never been happy with the conclusion in the influential blog post "Qt Graphics and Performance — The Cost of Convenience" by Gunnar Sletta: If you want performance, downgrade QGraphicsView to a mere canvas with a single QGraphicsItem. It defeats the whole purpose of decomposing the problem into many small QGraphicsItems and is therefore entirely counter-intuitive. One might be quick to ask what Qt Graphics View is good for (and one might find the answer here), but instead I would like to present an alternate solution to Gunnar's which reaches the same performance but embraces the very nature of Qt Graphics View.

Flashback

Gunnar took the example of a virtual keyboard (which doesn't really do anything, it just sits there and paints itself). And as it happens, I've been working on a real virtual keyboard. At one point, our painting strategy was exactly following the advice of Gunnar, but once new requirements piled up, it became increasingly harder to deal with that painting strategy.

The new requirements were custom key sizes, flexible layouts, custom font sizes for each label, custom font colours, custom key background and custom key labels. And all that dynamically during runtime please.

In MeeGo 1.3, applications will be able to make us of that. For example, when writing an email, the application can temporarily turn the enter key on the virtual keyboard into a "Send" button. Should the user enter an invalid email address, the application might choose to disable the "Send" button until the mistake has been fixed.

Reinventing QGraphicsItems - or not?

As the required logic for keys becomes more complex, so does the data structure representing keys. The paint method we used in the single keyboard item was about to become a real mess. Especially after I added geometry to keys, I knew that I was, in fact, reimplementing QGraphicsItems. But I could not reuse them because of performance reasons! Or was it possible that Gunnar's blog post pointed into the wrong direction?

Maliit's overlay item strategy

So I thought about my dilemma and figured out that the actual constraint is rather the amount of paint event calls per frame, and eventually, how much screen area needs updating. With the single QGraphicsItem approach, every change triggers a scheduled repaint for the whole keyboard - even if it's only the label position in one key.

I then realized that only pressed or customized keys would need to be actively painted, and there we had it: An alternate painting strategy that blits the idle keyboard view into a single QGraphicsItem but lets active keys paint themselves as overlays. The rest of our team was sceptical about the idea, so Viacheslav Sobolev - a colleague of mine during my 11 months in Helsinki - had to first improve the performance benchmarks before the team would accept the fact that indeed we managed to keep the same performance, and even improve it for Qt's raster engine.

The Strategy

Even if the general idea of overlay items is very simple, I summarized the following steps that one needs to take care of:

As a bonus, you can find the basics of the strategy in a modified benchmark of the original blog post. I added two new options "-overlayitems" and "-fullscreen". The latter is useful for N900 where it helps to get rid of system compositor noise.