Using TiddlyWeb as a content management system for

I saw Mike posting recently about how he is using TiddlyWeb with theTiddlyDocs and TiddlyGuv projects and I thought I would put something out about how we’ve been using TiddlyWeb with another Osmosoft project – the redesign of for the International Lesbian and Gay Association.

The ILGA project has taken our use of TiddlyWeb in unexplored directions. The essential problem that we’re using this system to solve is that we have a set of people around the world who wish to contribute content to a website, which is then stored and displayed to a visitor when they access the site. It is, simply, a CMS problem as solved a hundred times over by projects, public and private, around the world.

Chris Dent did not want to create a framework when creating TiddlyWeb , but with this is what he has done. With ILGA, we have been determined to push the limits of the system and expose new patterns of use and holes for further development.

The following describes how we’ve used TiddlyWeb to solve the problems of content-editing and publishing. The natural progression has been to work with a TiddlyWeb plugin, cheekily called “jinx”, when the existing toolset didn’t seem to be adequate. As you’ll see, revisions of our methods have used more and more of TiddlyWeb’s innards as we’ve gone along, showing what you can do with the system in its native form.

Accepting content from contributors

Where and how to publish

TiddlyWeb uses the concepts of “bags“, which are spheres of permissions, and “recipes“, which collect bags and pour out the tiddlers through filters. Tiddlers themselves do not have permissions assigned to them, which in a sense, helps you with your decision about where to let people put content. Each contributor gets their own bag, and anything that they contribute ends up in this bag.

In a finally enlightening cycle of development, we discovered that the connection between a recipe and the bags it references is implemented in a useful and elegant way: when you create a new tiddler and save it back to TiddlyWeb, it will end up in whichever is the first bag you have permission to PUT to, counting up from the bottom of the recipe. This means that if your own bag is the last line in the recipe, any new tiddlers will be assigned to that bag. A neat syntactical trick in creating the recipe can make it generic for any contributor:

   /bags/{{ user }}/tiddlers

This simple construction saved us large chunks of code that had been written as TiddlyWeb plugins before this mechanism was properly understood.

The TiddlyWiki as content manager

Although the pages that we wanted to show to site visitors had to behave like ordinary web pages, there is no such limitation on pages that the relatively small number of contributors use to edit and publish content, as they are effectively private. This allowed us to take advantage of the full power of TiddlyWiki as an editor and manager of the content in someone’s bag.

Draft vs. publish

One feature we added to the TiddlyWiki editor early on was a button to publish a piece of content. Initially, this involved a lot of custom code to handle PUTing an article, until the understanding of the bag and recipe concept outlined above encouraged us to use the default behaviour. Setting a flag on a tiddler that indicates which bag it comes from overrides the recipe-based bag selection and keeps publishing to different bags simple.

Displaying content to visitors

Now that we have content safely tucked up in bags, the question is how to get it out again for visitors to the site, outside of the shackles of a standard TiddlyWiki. This is because TiddlyWikis are not suitable as our ordinary web pages for reasons of accessibility, page size and search engine optimization. This has so far been the area most in need of attention and experimentation, as TiddlyWeb does not make content presentation as obvious as content saving.

Getting stuff out – recipes, filters and tiddler selection

[Update: this techniques described in this section continue to evolve, but I think it is worth posting this as it shows where we came from and does include some useful generic techniques.]

The complement to the content-storing bags mentioned above are the recipes, which deliver a filtered set of tiddlers that come from any number of bags. We’ve set each person up with their own bag to store their tiddlers. We found two difficulties with using recipes to deliver content:

  1. You can’t dynamically add the title of a tiddler you’re looking for to a recipe, unless you have a single recipe for e.g. each news article. This means that our recipes deliver a large number of tiddlers, which then have to be filtered further in some way, either in the template or in the jinx plugin;
  2. A further consequence is that the recipes get longer as you add more contributors, and have to be maintained. It would be easier to be able to point to all contributors’ bags in a recipe.

As mentioned above, further selection is required after a recipe has delivered a set of tiddlers. This is where the jinx plugin has been developed (and repeatedly re-worked!) the most. Because we can examine the URL requested in the plugin, there is an interdependency between the URL’s structure and the filtering applied to a set of tiddlers returned by a recipe. The URL structure itself is examined further down.

The scenarios we have for wanting to get content out of TiddlyWeb and the corresponding recipes and tiddler selection we’ve set up in the jinx plugin are:

  • Pages with one news article (a tiddler tagged ‘article’)

  • Pages with many news articles

selection (no further filtering)
  • Pages with country profile information (tiddlers tagged ‘countryprofile’ from a single contributor), news articles for that country, a tiddler defining themes for a map and a tiddler for the country itself

selection (no further filtering)

The final example, although still developing, illustrates how difficult it can be to get the tiddlers you need in a way that you can tell them apart. We added the second argument to the get_tiddlers_from_recipe function so that we could pass in tiddlers as named variables to the next step, HTML templating, and refer to them directly rather than iterating through a huge loop of tiddlers each time we wanted to find one.

Formatting stuff – templates

Right from the word go, we’ve wanted to “template” tiddler content through HTML templates, inserting bits and pieces of tiddlers into the relevant places in a mainly static page. The approach we’ve developed so far involves using the jinx plugin to map a particular URL path, say /articles/MyArticle, to a function that collects a set of tiddlers as described above (in this case probably “articles”) and pushes them through a templating process to generate the HTML pages.

A technique we’ve found ourselves using quite frequently is iterating through the delivered set of tiddlers to find the one we want. For example, on country page, we want to be able to isolate the tiddlers which are news articles. At the moment, we can do that like this:

{% for tiddler in tiddlers %}
   {% for tag in tiddler.tags %}
      {% if tag == 'article' %}
      {% endif %}
   {% endfor %}
{% endfor %}

This structure is less than ideal and it would be much easier to be able to access the groups of tiddlers directly, which is why the named_tiddlers_arrayparameter was created for get_tiddlers_from_recipe. However, this doesn’t feel very solid yet.

Accessing stuff – URL’s

The URL used to access a tiddler can be accessed and acted on by the jinx plugin, so we have used it to determine what is presented and how, through two methods: first, by mapping various URL structures to functions inside the jinx plugin; second, by using the sections of a URL’s path as variables in these functions, determining things like which template is used, or which tiddler is retrieved.

The mappings we are using are listed below. TiddlyWeb differentiates between/resource and /resource/, so there are double entries for most mappings:

config['selector'].add('/index.html', GET=get_index)
config['selector'].add('/myactivism', GET=get_myactivism)
config['selector'].add('/myactivism/', GET=get_myactivism)
config['selector'].add('/allies', GET=get_allies)
config['selector'].add('/allies/', GET=get_allies)
config['selector'].add('/article/{name:segment}', GET=get_article)
config['selector'].add('/countries/{country:segment}', GET=get_country_section)
config['selector'].add('/countries/{country:segment}/', GET=get_country_section)
config['selector'].add('/countries/{country:segment}/{section:segment}/', GET=get_country_section)
config['selector'].add('/countries/{country:segment}/{section:segment}', GET=get_country_section)

It seems that, in the pursuit of genericism, GET /path/to/resource should be understood as GET /path/to/recipe/tiddler – accessing a recipe, pushing the results through a template (perhaps with the same name as the recipe), and selecting the named tiddler.