marnanel: (Default)
Further to my previous post:

Here's a live AJAX-based version you can play with. It's not very fast unless the word is cached, and it only takes at most a word of context (unlike the real thing, where the context is everything you've ever typed), but it should serve to demonstrate the principle.

Now to release the code, and to look into patching existing VKB systems.
marnanel: (Default)
Funnily enough, someone was asking about virtual keyboards on gtk-list this morning.

Last week at the MeeGo Conference several people were talking about virtual keyboards, and the idea came up of doing predictive text, either by making more likely letters physically larger, or merely by increasing their sensitivity.

When I came home, I wrote a JavaScript mock-up based on a third-order Markov chain. It's quite fun to play with, especially on a touchscreen.

When I showed this to a few people at Collabora and elsewhere, Rob McQueen suggested avoiding reinventing the wheel by using the rather wonderful Dasher system as a back end. So, after a longish hacking session, here it is:



State of the keyboard after typing "FLO".

Click here to see a video of the keyboard in action


The front end shown here is just a custom GTK widget I threw together; in real life it would use an existing input method. I've exaggerated the differences between letter sizes for demonstration. (As I mentioned above, the physical letter sizes might not change at all.)

There is a wiki page about all this. Let me know if you'd be interested in helping work on this; I'll be releasing the code shortly, and adding a link on the wiki to it. (Odd thought: I wonder how useful another demonstration piece of JavaScript would be, pulling data from Dasher running as a CGI. Let me know.)

There is also an existing roughly similar system for Android, and, I hear, for the iPhone.

Update: An AJAX version you can play with.
marnanel: (Default)
Woke up quite early, packed up my suitcase, and hacked for a while. Got stuck on the correct (fairly complicated) commandline to pass to qemu; kudos to Ian who woke up, recited it to me, and then fell back asleep again. After that I went downstairs and had breakfast, then caught a taxi to the airport.

The driver asked us whether we wanted to take the toll tunnel, which costs €3. One of the other passengers said yes, but he only had sterling, and was that all right? The driver made the sign of the cross at him.

The plane ride was long, dull, and uneventful, so I spent as much time hacking as my battery and lack of network connection would allow. I took this photo of another plane out of the window, and wrote about clouds. And it was wonderful to be back with Fin, Alex, and Rio, as always.

It's been quite a conference. Thank you once again to all the organisers, and to everyone who made it what it was. Perhaps we'll all see one another in May in San Francisco.

The rest of my conference posts:
marnanel: (Default)


Today was a day of un-conferences, where people attending the conference made up the programme. It was a day of being given notebooks, as you've already heard. And it was a day of packing everything up.

After things were cleared away, some people went to watch the Ireland v Norway match, but we went to a brewpub called Porterhouse for beer and pies.

It's been a memorable and useful few days. Back to Philly tomorrow morning.
marnanel: (Default)
Intel and Nokia have decided to give us all free laptops, as long as we use them to go around and show people MeeGo, and for those of us who are coders, we write lots of cool stuff with them.

Here is the queue to get one:



And here is my new and as yet unnamed laptop, running MeeGo:



I have to say that installation was a piece of cake. Now everyone has three devices attached to the wifi: their ordinary computer, their phone— N900s are almost ubiquitous here— and their new laptop. So I hope this posts!

Oh, I have to mention: the screen on this swivels right round, and is a touchscreen, so you can just tap the screen instead of using the touchpad. It is a very, very shiny thing.
marnanel: (Default)


I think my talk went reasonably well in the end, though there were no questions. I'm not sure whether the audience were entirely satisfied or catatonic with hearing me drone on.

After the talk, I went off and hacked for a bit to clear my head until it was time to go to the brewery; I sat next to Michael Meeks on the way down there, and we talked about office suites.

When we arrived, they had a pretty comprehensive exhibition about how they make Guinness: I wandered around that, reading about hops and barley, until I noticed that people were vanishing upstairs. Well, toucan play at that game, so I followed. My goodness. There was a bar with people walking around handing out pints. There, Will gave me the useful information that Guinness Special Export is much more tasty than the ordinary kind. After a rather large number of pints of that, a rather good U2 cover band appeared, so we all danced. (Thank you for the whiskey, Chani.) I feel surprisingly well today, apart from a slight headache. I suppose Guinness is good for you.

Huge thanks to the organisers, who have certainly shown themselves capable of… no, that line is far too obvious.
marnanel: (Default)


Monday was mostly spent working on my presentation, though I spent quite a bit of time mingling with folk and talking. After hours, Collabora threw a party, and when that party eventually broke up we ended up at a brewpub called Messrs Maguire, who do good beer and good pies. I went back to the hotel and tried to do some more work, but fell asleep.

I also extemporised on some verse by Chesterton.

I am approximately in the centre of this picture, wearing orange with the yellow strap of a bag across my back.

The lights here are shaped like aerofoils, and glide forwards and back in air currents.
marnanel: (Default)


There are hundreds upon hundreds of fascinating people here at the MeeGo conference. I've been here less than a day and already had useful and memorable conversations. Even when I was waiting at the gate at Philadelphia, two people spotted me, realised I was going to the conference, and came over to talk about tech stuff.

The flight was fairly uneventful, and I slept much of the way. On arrival, we were told it was 34° outside, and I wondered why I'd brought my coat, until I found that US Airways seem to be unfamiliar with Celsius: it's a bit chilly. I took a taxi to the hotel, with a very chatty driver who gave me a brief overview of surviving in Dublin, told me where to find good beer, pointed out landmarks, and passed me a tissue when I sneezed. The hotel didn't have my room ready, so I wandered off.

I soon found myself at St Bartholomew's. A friendly usher (from Leicester, but who had lived in the area for years) welcomed me and told me that they were considered very Anglo-Catholic for the Church of Ireland, but middle-of-the-road by Church of England standards. I think he was mistaken: they would have been considered well within high-church anywhere I've ever been in England. Everyone was friendly, and gave me coffee and biscuits.

Back at the hotel, they were very rushed, with the whole conference taking over the place and then a football match on top of that, and they still didn't have my room. I went to lunch and came back later, and they gave me the key: I went upstairs, took a shower, and fell asleep. After a while Ian turned up, so we explored the hotel for a while, but then he had work to do. Soon after I ran into Rob, Will, Gary, Rodrigo, and others, and we went out for a drink in some random pub near Trinity College, which turned into dinner. And now I'm back at the hotel writing it all up.

I'm really looking forward to tomorrow.
marnanel: (Default)


In xzibit we have the concept of doppelganger pointers, which track the movement of "real" pointers on the other window. (Doppelganger pointers are usually shown with black insides, and "real" pointers with white insides, as in the diagram above.) The position of the doppelganger is transmitted over RFB in one direction, but since RFB sends this information only in that direction, the equivalent information in the other direction is sent over the xzibit control connection.

However, there is a problem.  In order to send the doppelganger information we have to track the movement of pointers, by receiving XMotionEvent events.  It turns out that XMotionEvent tracks the movement of doppelganger pointers as well as "real" ones.  This causes a feedback loop: the host user moves the pointer, the remote user sees their doppelganger, the remote user's doppelganger generates motion events, which moves the host user's doppelganger, which causes the remote user's doppelganger to move again, and so on for ever.

There are a few possible workarounds, but none seems very satisfactory.

We could stop using actual pointers for doppelgangers.  Instead, we could use shaped windows and move them about.  That would stop them generating XMotionEvent.  However, experimentation shows that this causes an unacceptable amount of flickering.  So it looks as though we're stuck with using actual pointers.

We could also discount motion events derived from doppelgangers.  This could be done by examining the event for some sort of flag (which doesn't seem to exist).  A better solution is to remember where we put the doppelganger, and ignore any motion event from those coordinates.  This is easily done for messages over the xzibit control connection.  However, doing this for messages using the RFB PointerEvent mechanism poses a greater problem.  We could:
  1. fork the gtk-vnc library and add a special case for doppelgangers.  This is the most efficient answer.  It is also a horrible bodge.
  2. ignore all RFB PointerEvents, and use the xzibit control connection in both ways, at the cost of sending all pointer events in one direction twice.  This is the simplest answer, but is horribly inefficient.
  3. run a simple filter over the RFB stream as we receive it, which removes any RFB PointerEvent messages at the place where we last put the doppelganger.  This is probably the cleanest solution, but is rather a hack.  It's also difficult to implement, since RFB messages are of variable length and require knowledge of each type in order to know where each one ends.  (A first approximation is simply to look for PointerEvent messages with the expected parameters anywhere in the stream, and hope that the five bytes we're looking for don't happen to occur embedded within another message.  It's unlikely that they will, but again, it's not an elegant solution.)
I don't much like any of these.  Tell me what you think.

Update: jojomojo points out xinput2's XIDeviceEvent which does have the mpx ID field.  This is very useful, but now I have to make gtk-vnc be aware of it.

marnanel: (Default)
I shall be presenting a talk called "Porting existing software to MeeGo" at 15:30 on November 16th in the 1872 Suite at the Aviva Stadium in Dublin. Huge thanks to Collabora for sponsoring me to go, and to Nokia and Intel for organising everything.

Here is the abstract:
Understanding the best ways to design for the screen of a portable device, with unusual widgets and limited space, can be a daunting prospect. In this talk I will discuss the ways in which the user interface of existing software can best be re-conceived under MeeGo. I'll examine the differences between MeeGo's widgets and those of desktop systems, as well as those used under Android, supported by some case studies. Finally, I'll talk about other, non-UI considerations needed in porting.
marnanel: (Default)
If you'd like to read the paper I presented to Durham Compsoc on Tuesday on xzibit, it's at the link.

Also, here is a photo of University College, Durham.

marnanel: (Default)
Here is a current problem I have with xzibit.

I am implementing what I call "doppelganger cursors". These are mouse pointers which appear on a shared window as the mouse pointer moves across the other window. They have the icon of the person on the other end appended. Here's a video of what they currently look like.

Currently, these are implemented by creating a new, extra pointer using XInput2, and then doing a pointer grab on that one pointer in order to change it into the new image. This is good as far as it goes, but if your client has a pointer grab then MotionNotify events aren't reported for any other pointer, which means that we can't detect movement on that window: so it's impossible to get doppelgangers working in each direction.

Possible workarounds:
  1. Find a way to do a grab and still get MotionNotify events.
  2. Don't do a grab.  Set the pointer per-window instead.
  3. Don't use an extra pointer.  Use an undecorated window and float it about in the right place.
Your thoughts are welcome.
marnanel: (Default)
Two things I'm considering about xzibit:

1) Permissions. At the moment there is no provision for the receiving side to reject the window. I could make a rule that you have to okay every new window, but then you would have a dialogue popping up for every dialogue, and even every menu. So I think I will make an accept dialogue that you have to reply to once, at the start of the connection, thus:



2) Doppelganger cursors. I already have a subsystem to make "doppelganger cursors" which track the mouse movements of the user on the other end of the connection. (Video of that here.) But RFB only has a way for the sending side to notify cursor movement, so we'll have to send the movements for the receiving side over the Xzibit protocol. I'm wondering whether I should just send all the movements over the Xzibit protocol and make things simpler. I'll probably accept RFB-based movement messages, either way.
marnanel: (Default)
xzibit 0.03 released, now with a usable UI. Here's a demonstration.

0.04 will have fixes to the way we use X extensions to fake keystrokes and mouse events.
marnanel: (Default)
xzibit 0.02 is released. This version uses Telepathy tubes for transfer, rather than TCP sockets. Next version will contain UI improvements; you may want to wait for the next version, which will be easier to use. If you do try this version, you will need a patched version of libvncserver. The patch hasn't been upstreamed but I'm hoping it will be.

Until 0.03 comes out with a better UI, the best program to use to test this is the supplied xzibit-autoshare, which is capable of setting the "source" and "target" properties on itself and sending itself to another xzibit system. Here is an example of sending a simple window from one account to another logged in to the same computer:

marnanel: (Default)
xzibit 0.01 is now available for download. xzibit is a per-window VNC system; its development is supported by Collabora Ltd. This version is just a taster of what's possible: there's a fair amount more in existence that you won't see unless you play around. More of it will be more easily visible in later releases.

Do read the README for instructions on setting it up: the checks in autotools for required programs are by no means perfect yet.

How this is different from X forwarding:
  • Most importantly, because it works (or will soon work) over Telepathy tubes, so you can share windows with your chat contacts.
  • Less importantly, it uses VNC so that it's platform-independent.  There is only an X implementation at the moment, but eventually I'm hoping there'll be Windows and OS X implementations as well.
Other details
  • I have proposed a talk on xzibit at LCA in January.  I don't yet know whether it will be accepted.
  • The protocol is explained here, and you can see a diagram explanation here.


You may clone the git repository from here.

I'm now working on 0.02, which will have proper working Telepathy support.
marnanel: (Default)


Surprisingly much work got done on xzibit this week, despite my being at Guadec.

Week 30 (of the year, not the project):
  • Metadata can be sent about windows. This is trickier than it sounds, because the window might not yet be mapped on the receiving side.
  • Demonstrated the system in a talk at Guadec on Friday.
  • This got Xzibit mentioned in a German magazine (seventh paragraph).
  • Discovered why menus don't get drawn: GDK doesn't receive expose events on override-redirect windows (at least in some circumstances).
  • Made a start on multiplexing the various streams so it will be easier to send them over Tubes, and wrote tests for the marshalling.
Plans for week 31:
  • Discover why expose events don't get received on override-redirect windows, and either fix it or use something else.
  • Finish multiplexer.
  • Make gtk-vnc be able to read from fifos (if it can't already) so that we can easily hook up the windows on the receiving end and still run them in separate processes, without having to mess around with TCP. May need to send a patch upstream for this.
  • Consider the ramifications of using a single gtk-vnc process for all received windows; maybe it could just eavesdrop on the received multiplex stream without any other interpretation.
  • Switch to sid so I can run Daniel's spiffy build of Xephyr, and use it to test the MPX work.
  • Send window icons in the metadata.
  • Send transiency information in the metadata (including Cartesian offset from the parent window, something I realised we need in this place last week).
  • Start thinking about how to implement the doppelganger cursor. Someone in the audience suggested decorating it with a hackergotchi, which I think is a wonderful idea.
marnanel: (Default)
I thought I would walk to the Collabora party from the conference. It was four miles, a pleasant walk. On the way I had to stop to write a sonnet.

If anything should happen to The Hague,
if someday they abandon Amsterdam,
philosophers will take these strange and vague
descriptions, and derive each tree and tram
by mathematical necessity:
should nations shake their fists across the seas
with words of war, it follows there must be
a middle ground, a people loving peace.
And is this scrap alone a netherland?
Not so: we spend our nights beneath the sky,
and every country's low for us, who stand
a thousand miles below the lights on high;
if only I could learn to live as such,
and count myself as kindly as the Dutch.


I passed the Palace of Justice on the way, which is very beautiful. Collabora's party was as impressive as always, with barbecues and beer. This morning I managed to pull myself out of the resulting hangover enough to give part of a talk on xzibit. (It was really Guillaume's talk, but he was kind enough to give me a timeslice.) The talk went well except that the demo failed, due to my having tried to fix something and breaking it further. There will presumably be video of it all at some point.

Many thanks to Collabora for organising the party, but still more for sending me here (and to Cambridge).

I have written a nautilus plugin to post photos online. I might tidy it up a little and package it.

The MeeGo book is fast approaching publication. It feels like levelling up.
marnanel: (Default)
I've been to some rather interesting talks, notably j5's talk on JavaScript. I would like to turn Gnusto into a JS-based GNOME desktop app.

The wifi infrastructure is solid:

...the gateway to the rest of the world less so.

I need to work harder on patch review in Metacity.

I shall be helping give a talk tomorrow morning at 0930 in the Copenhagen room; I'll be talking about xzibit.

xzibit currently has a bug where all menus are resized to 10x10. I need to build some test programs to find out why.

Someone from Canonical is doing work on notification icons in titlebars, which is a) something that xzibit needs that I'm currently hacking in rather badly and b) interestingly similar to the question of "why can't we have extra buttons in Metacity?", which I am frequently asked. I am therefore comparing notes with them, so as not to duplicate effort.


Canonical's party was last night; I went but didn't hang around; I was tired and couldn't get near the bar. I did take a photo outside the place, and was then approached by a large drunken Dutchman who wanted to know why I was taking photos of him and whether he could see my phone to see the pictures. I used my superpower of invisibility to escape. (It's a good job I packed a d20.)

Collabora's party is tonight, on the beach.
marnanel: (Default)


You and I spend our lives beneath the heavens, beneath the stars; in some sense we all live in a nether land, and we are all inhabitants of the Low Countries. I therefore surmise that I have arrived at the centre of the planet.

In England, you will sometimes see a bee drawn into the glaze on a toilet bowl. This is a Latin pun: the word for bee is apis. In the toilets at Schiphol I saw such a symbol, and had to rush outside and find a Dutch person to verify that the same pun works in Dutch. (It does.)

Last week, I heard that the house of my great-grandmother, Nono, had changed hands and name. This is unfortunate: it's in the village of Turvey, and Nono had the good sense to name it Topsy. Her postal address was therefore Topsy, Turvey, Bedfordshire.. The new owners do not share her sense of humour.

On Monday, very little happened, except that I broke my laptop. I was certainly in the right place to do so, because four or five people congregated to help me bring it back online. One person from Canonical said, "You need a bootable USB key. Just shout and ask for one." When I demurred, he shouted, and one was lying on the desk in front of me within ten seconds.

We went to Chinatown for lunch, and ate dim sum. The afternoon was spent working on xzibit, with red wine substituted for coffee, and I made a rather homesick call to Fin and Rio. I'm pleased to report that you can now share menus in xzibit. I would like to give a lightning talk, but I think I've left it a bit late to register.

After work I walked back to the hotel with Danielle, stopping at a supermarket to buy licorice, bread, and some of that hard, sour cheese the Dutch are so fond of. Then I went back to my room and ate cheese sandwiches, and hacked until I fell asleep.

Profile

marnanel: (Default)
Monument

March 2017

S M T W T F S
    1234
567891011
12131415161718
192021 22232425
262728293031 

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Mar. 26th, 2017 08:44 pm
Powered by Dreamwidth Studios