{"id":561,"date":"2010-08-11T23:12:04","date_gmt":"2010-08-11T23:12:04","guid":{"rendered":"http:\/\/brej.org\/blog\/?p=561"},"modified":"2010-08-12T09:16:55","modified_gmt":"2010-08-12T09:16:55","slug":"tidbit-more-technical","status":"publish","type":"post","link":"https:\/\/brej.org\/blog\/?p=561","title":{"rendered":"Tidbit: More technical"},"content":{"rendered":"

In the previous post<\/a> I covered the ideas behind tidbit. In this post I will try and cover the technical aspects of the tidbit system. Currently the work is very exploratory, so everything may change.<\/p>\n

Tidbit record structure<\/h2>\n

This is a typical tidbit which was generated using the Rhythmbox plugin:<\/p>\n

TIDBIT\/0.1; libtidbit\/0.1; Rhythmbox Tidbit Plugin v0.1<\/span>\r\ntidbit_userkey==usePzEg4Cl4g1ASdzpssVHtQ1hJJilS+ryiBWjF...\r\ntidbit_table==audio\/track\r\ntidbit_created:=1281479640\r\ntidbit_expires:=1313037240<\/span>\r\nartist==Arcade Fire\r\ntitle==Keep The Car Running\r\nalbum:=Indie\/Rock Playlist: May (2007)\r\ngenre:=Indie\r\nyear:=2007\r\nplay_count:=34\r\nrating:=0.8\r\ntidbit_signed:=JyJ1fIwhRL5t3y9CACmshm\/UibYVhvInxh7XVx4...<\/span><\/pre>\n
The first line is the header. It states the version of the tidbit followed by the user agent. The rest of the record is composed of key-value pairs. The key has a strict format of lower-case letters and underscore. The value can contain any character above 0x1F, and is terminated by a new line. Other characters must be escaped. The first\u00c2\u00a0 four pairs are compulsory and they all contain “tidbit_” at the start to distinguish them from normal data. The userkey is a unique(ish) 1024 bit RSA key the user uses to identify themselves and also serves as the public portion of their signing key. It is base64 encoded and in the text above it is clipped but in reality it is over 100 characters long. Table is compulsory field which designates the subject matter. The created and expires values state when the record was created (must be in the past) and when it will expire. Expired records are no longer valid. These are currently using Unix time, but a more general format will be used in the future. This is followed by a number of values specific to the record type. Finally, the record is completed by a signature which signs the body up to that point (also base64 encoded). The signature is generated using the user key which signs an SHA512 hash of the record (up to that line). There is a hard limit of 2KB per record to prevent abuse.<\/p>\n
The separation between the key and the value is either ‘==’ or ‘:=’. These signify if to search for that value, or overwrite the value. When inserting a new record, a search is performed for any records which match all the key\/value pairs with the ‘==’ separator. These records are discarded as they are overwritten by the new record. To ensure the correct sequence in cases where an old record is re inserted into the database, the created date is checked. This allows a record to be updated by destroying an older version.<\/p>\n
Library<\/h2>\n
A library (libtidbit) handles most of the complexity of creating tidbits, key handling, communicating with databases and performing queries. Keys are stored in a gnome-keyring. There are also python bindings which make creating plugins simple. Here is partial mock-up of an example use in Rhythmbox:<\/p>\n
\"\"<\/a><\/p>\n
In this plugin, forming tidbits and passing the out is very easy. Presenting the data is the hard part.<\/p>\n
Databases<\/h2>\n
There are several database backends used in tidbit:<\/p>\n