Yarrow and metadata

Note that at the moment rgtp.thurman.org.uk and marnanel.org are both down. I will write this as thought they weren't. If you are looking for something to replace rgtp.thurman.org.uk until it comes back, please go here.  I may be posting more about the downtime later.  But this is not that post.

I would like your thoughts on a question of protocol design.

There is a system which I maintain called Yarrow. It is one of several clients for a bulletin board protocol called RGTP. To my knowledge, there are three bulletin boards in the world that use RGTP:

• GROGGS;
• a private one known to me;
• my blog.  The reason my blog uses RGTP is that recently I realised that I know Yarrow like the back of my hand, and it does about 90% of what I want out of a content management system.  And it also means that I have more of an incentive to keep it maintained, since I'm using it every day.

As to RGTP servers: GROGGS uses an RGTP server called simply "rgtpd", written in C by Ian Jackson.  The other two use an RGTP server I wrote many years ago called Spurge.  This means that I maintain both the server and the client for my blog.  However, the server is only accessible from localhost, and thus my blog is a special case because it is only accessed using Yarrow.  The other instances are also accessed using other RGTP clients, none of which I maintain.

When I set out to fix up Yarrow to work as a blogging platform as well as a bulletin board client, there were two principles I tried to keep in mind:

• some new features would be necessary, and
• any new features would not be permitted to interfere with existing pure-RGTP features.  Ordinary groggers should not have to know about any of it.  Certainly there should be no chance of GROGGS appearing in HTML, for example.

The most important of these new features is metadata, which I added to Yarrow for several reasons, the three most important among them being:

• to allow the content type to be specified (basic RGTP is text only; my blog needed to be HTML), and
• to allow the charset to be specified (basic RGTP is iso-8859-1; my blog needed to be Unicode), and
• to allow for tags.

The metadata is written inline at the starts of entries.  At present, however, the server is an almost unmodified Spurge, and knows nothing of the existence of metadata.  Therefore, the server cannot be asked for metadata information.  This means that if Yarrow's cache of metadata information gets out of date, Yarrow has to request the entire corpus from the server in order to rebuild the cache.

The RGTP standards document says in §2.2 that extension commands may be added.  I am considering adding a "META" command which retrieves only the metadata.  This would make everything a whole lot simpler and faster.  The disadvantage would be that metadata couldn't be used with an unpatched server, but this isn't necessarily a problem since I, and anyone else who might be using Yarrow for blogging in the future, will either be running Spurge or will be running some new server whose authors can be made aware of the extension.

What do you think?  Sensible?

I also need to work out how Yarrow can tell whether a server supports metadata in the first place:

• At present this must be set in Yarrow's configuration for each RGTP server.  This works, but it's ugly.  I would like to make it so Yarrow could essentially configure itself.
• I could make a rule that Yarrow tries the META command the first time it connects to any server, and if it's not recognised, it will remember that that server doesn't know about metadata.
• I could also make a rule that servers which support metadata must put "(META)" in the connect string, or something like that.  That would allow a server that didn't support metadata to start doing so without confusing all the clients.

What do you think?

Future thoughts:  One day I'd like:

• Yarrow to use genshi for templating so its HTML output could be easily redesigned (this is actually half written);
• Yarrow/Spurge to be debianised (separately or together) so people could install them more easily;
• Yarrow to be able to auto-configure itself using a browser on first install (like WordPress or MediaWiki are able to).
• Yarrow to have a way of interrogating Spurge instances running on localhost in order to make this auto-configuration simple.
• Maybe adding virtual hosting support to RGTP.  I have ideas about how to do this, but that can be another post.

Nimyadelen

Am soc caril tiricoth yalad far a yoroden til. So rejilen majanwi, am fam inten olic Jinarec, am til inten inec. Yad inten tirilec, jetac if ofefen am lath inten tiricoth. Soc inten ofefen al orefen, a soc inten gathamaran al naratifil. Am yad inten, jetil if coli ifoth, dasanmil e mim ten te ojil, ac joc rejilen if a mititij carilec tiricoth. Al dro carilwi te Joril ac fam in te dasan e til te rejilen ten. Yad te Joril, sil jim caran far e yad tenen yalad til ten laracen, jim inten laracen tiricoth. Jeso carilwi if ac tij yalad intenti ac mijoc inten tirilec. Am tij caldacoth carilwi tiricoth te Joril te rejilen, am mitac inten te coli. Ac lac tacan inti Tifij, sel tij te Joril te yalad cariloth tiricoth caldacec. Tij caldacoth te Joril inten thajacwi tiricoth.

You may not know that I've been working on a conlang called Nimyad for the last twenty-odd years. Names like "tajasel" and "firinel" and "marnanel" are words from this conlang. I worry that I don't know enough linguistics-- that things I want to do in Nimyad aren't *really* how a language would work. Also, I haven't worked on it in the last couple of years as much as I should have.

I would like to rectify this. Five is a special number in Nimyad. By this time next year, I would like to have done five things:

1. Finished the conscript (the canonical way to write down Nimyad words);
2. Have a lexicon with at least a thousand words in it (the current one I'm working with has perhaps a hundred);
3. Have written a comprehensive grammar;
4. Have written a primer (i.e. a text to teach people Nimyad);
5. Have translated the entire Gospel of Mark as a demonstration text. But feel free to advocate for other texts.

If you'd like to help with this, and be part of the Nimyad community, please join the Nimyad community on LJ and we can discuss it further. Or would you rather I posted Nimyad discussion in my own journal? Or should we run a forum or a blog on nimyad.org?