On Writing, Tech, and Other Loquacities

The collected works of Lana Brindley: writer, speaker, blogger


Leave a comment

OpenStack Newton Summit – Docs Wrapup

Everything is bigger in Texas: including the conferences!

DSC_1078

This Summit was a homecoming of sorts. OpenStack started in Austin with 750 people, and returned six years and twelve conferences later with 7500 people. Even the baristas in the downtown coffee shops noticed us the second time around.

For documentation, this conference was bigger than usual as well. We had a total of eight sessions, in addition to the contributor meetup on the last day, which is more docs sessions than we have ever had before.

And we had a lot to talk about! The biggest thing on our minds was the future of the OpenStack Installation Guide. The Big Tent has changed the way that projects go about joining the OpenStack ecosystem, and with Foundation having an increased focus on ensuring new projects have sufficient documentation, we needed to change our approach to documenting the installation of an OpenStack cloud. There is no ‘right’ way to install a cloud any more, and there is certainly no ‘right’ set of components you should be installing when you do it. But with a small documentation team, and a seemingly endless parade of new components requiring documentation, we were faced with a big technical challenge, where everyone had some kind of skin in the game. Despite some differences of opinion, the session itself was extremely productive, and we came away with a solid set of deliverables for Newton. First of all, we’re going to create the infrastructure to allow projects to write their own installation docs in their repos, and then publish them seamlessly to the docs.openstack.org front page. This means that projects have responsibility for their own docs, but the docs team will provide assistance in the form of templates and infrastructure support to ensure that all projects are treated as first class citizens. Secondly, the existing Installation Guide will change focus to be more about an installation tutorial, giving people a highly opinionated and completely manual installation method to learn the ropes, but not to install a production cloud. Thanks to the OpenStack User Survey, we can safely say that most production clouds are installed using some kind of automated tool, so having manual installation instructions is useful as a training tool, but not in a real world scenario.

With the big question more or less settled, we got on to the fairly long laundry list of other things that needed to be done, which all ended up focusing mostly on streamlining some of our processes, being clearer about the way we operate, consolidating guides that had (for obscure historical reasons) been in their own repos into the main one again, and general editing and tidying up. A full list of the goals can be seen here: Newton Docs Deliverables. And, for historical interest, here’s the whiteboard from the Summit session:

DSC_1099

During the Mitaka release, docs had a focus on Manageability, aiming to work more effectively and efficiently, with a focus on collaboration. For Newton, while manageability themes are still very much present, the focus is more on Scalability, and making our documentation efforts scale out to represent a much greater proportion of products, contributors, operators, and users. From empowering projects to write their own documentation with our support, to making our processes simpler to find and understand, to ensuring our documentation is as accurate, up-to-date, and effective as possible, it’s going to be an exciting cycle for docs!

I leave you with one of my favourite Texan big things: a bathtub margarita!

DSC_1074

Share


Leave a comment

Content as a driver of change – Then & Now

Humans have always written things down.

Those of you reading this post, with your laptops, and mobile phones, and iPads, and vanity email accounts, and your single sourced, content-reuse, DITA-compatible Docbook XML toolchains, with all your fancy Javascript elements and mind-boggling CSS overlays. You are just the latest in a long line of human beings who have been doing the same thing for millennia. Albeit with different tools.

Panel_of_simple_figures_with_boomerangs_-_Google_Art_Project

The original owners of the land we are standing on today are the Wurundjeri people. Australian indigenous art is the oldest unbroken tradition of art in the world. These weren’t just the pre-history version of hanging a Monet print on your loungeroom wall. Indigenous art exists on all manner of things: paintings on leaves, wood and rock carvings, sculptures, and of course cave drawings. This art gave early Australians a way to record the things that mattered most to them in their lives: they often involve scenes of hunts or special ceremonies. In the case of Australian art, many include megafauna and other extinct species, and even the arrival of European ships. More than a record of events, though, they were probably also a method of teaching. Each indigenous tribe had its own mythology (collectively known in English as ‘the Dreaming’), which used stories to convey morals or other educational information. Most children who grew up in Australia would be familiar with the Dreamtime story about Tiddilik the Frog, a fable about greed and about finding humour in bad situations. Indigenous art and the stories that lie behind them are really just an early technical manual for life itself, especially in a world where living for any length of time could be quite difficult.

De_Architectura027

Who here remembers the story of Archimedes and his bath? It’s a demonstration of how Archimedes used water displacement to measure the density of an object (in this case, the king’s crown). Of course, the bit we all remember of the story, though, is that Archimedes, having made his discovery in the bath, went running naked through the streets of Syracuse, crying “Eureka! I have found it!”. This story comes to us from one of the oldest surviving technical manuals in existence, the “De architectura” by Marcus Vitruvius Pollio, which was published in around 15BC. Of course, the Ancient Greeks & Romans were well known for their literature, their scholars, their philosophers, and perhaps above all, their library. The Royal Library of Alexandria in Egypt was the largest repository of knowledge in the world between the 3rd century BC and 30BC. The famous fire that destroyed it was probably set by Julius Caesar himself in 48BC, but the library continued in some capacity until the Roman Emperor Aurelian destroyed what remained in about 270AD. This was of course a massive blow to literature, but it also an incredible loss of technical data as well. Thankfully, the Ancients managed to keep going even after the library was destroyed, and we now have surviving copies of wonderful pieces like Pliny’s Naturalis Historia, which is essentially the world’s very first Natural History encyclopaedia, and which set the stage for many more technical manuals to come.

Gutenberg_Bible,_Lenox_Copy,_New_York_Public_Library,_2009._Pic_01

Jumping over to Europe, Gutenberg did his thing with the printing press in the mid 1400s, but printed books were still a terrifically rare and expensive thing until well into the 15 and 1600s. Up until that period, if you were a fairly ordinary person in a fairly ordinary European town, you were probably aware of the existence of almost exactly one book: the bible that your local clergy had sitting on a plinth in your church. You probably couldn’t read yourself, or if you could probably not well enough to be able to read and understand a book written predominantly in a particularly stuffy version of Latin, and even if you could read that well, you wouldn’t be allowed to touch it. No, the bible was the word of God, and as such could only be read and interpreted by men of the cloth. They didn’t really want people going off and reading the Bible on their own and drawing their own conclusions about things. Of course, this got really interesting once the Reformation really started to get underway in the mid 1500s, and people started to read the Bible for themselves. In fact, for a little while there in England, Henry VIII decided that ordinary folk (and all women) were banned from reading the Bible. All this running around reading things and learning by everyday people was just a little too much for him to bear, especially when they started disagreeing with him.

Marianne_Stokes_(1855-1927)_-_-The_Frog_Prince-

Still in Europe, with better access to mass printing, publishing written versions of early verbal history became the thing to do. We all know the Brothers Grimm were writing fairy tales in German in the early 1800s, but they certainly weren’t the first to try and document the oral history of early Europeans. Charles Perrault is considered the original author of many of the Disney favourites, including Cinderella, Little Red Riding Hood, and Sleeping Beauty, and he was writing in French over a century before the Grimms, in the late 1600s. But even he was just writing down stories he’d heard from others. My favourite version of Cinderella comes from Giambattista Basile, published in Neapolitan in 1634, some years after he died. These stories, gruesome as they were before Disney got a hold of them, were intended in many cases to be fables for children, with a moral story, but were also used as cautionary tales for adults. In Basile’s version of Cinderella a husband is warned of the horrors of not being too picky about your second or third wife, he gives a general warning to the household about choosing your housekeeping staff carefully, a warning to parents about treating children fairly, and a warning to young women about being proud. And that’s before we get to the bit Disney likes: “if you’re a good person, good things will happen to you”. Some versions of the story also slam home the opposing moral: “if you’re a bad person, bad things will happen to you”, with both the step-sisters either mutilating their own feet to fit the slipper, having their eyes pecked out by birds at Cinderella’s wedding, or some equally terrible combination. As for other horrifying fairy tales, anyone who has read anything by Hans Christian Anderson will know that they often got worse before they got better. There’s a reason Disney never took on “The Little Match Girl”. For a long time, what we now know as fairy tales were the easiest and most entertaining way for a largely illiterate population to record and share moral stories and warnings.

geisha_maiko

A ribbon that runs through all of these is the idea of the master and apprentice. These types of relationships began in Europe in the 1300s, and were a way for a trade person to get cheap labour, while a young apprentice got a bed to sleep on, food to eat, and the hope of a trade later on. This system was used throughout England and Europe for all skilled trades: from seamstresses and blacksmiths, to Knights with their squires. However, the general principles of apprenticeships exist throughout the world, with one of the earliest examples being the idea of a Maiko, or a trainee Geisha. Geisha have existed in Japan since around 700, and still take in Maiko to this day. While this isn’t written knowledge, it is an important footnote when we’re discussing the history of content, as this was the main way that specialised technical knowledge was handed down.

JamesCampbell_NewsfromMyLad

Of course, a young apprentice, wishing to remember all the things they had learned, might be inclined to write them down. By the time the Industrial Revolution was in full swing, paper and books had become affordable, schooling was more available to children throughout Europe, and literacy was becoming much more widespread, especially to those bright young apprentices who left home to seek their fortunes. And while young people have written home to their families since ancient times, letter writing really hit its stride around the turn of the century when it became not just a way to record their days and connect with their families, but also a way to explore political and religious matters, and explore emotions: poison pens, love letters, and obituaries are all well represented in letters. Another form of writing more like the manuals we know today of course, is the recipe book. Many household cooks would enshrine their recipes in writing, to be handed down to the next generation. I regularly bake a family choc chip biscuit recipe that has been handed down mother to daughter for at least five generations, and possibly quite a few more than that.

But enough history. The older writers in the audience will probably remember most of these more recent forms of technical communication. Some of the more unfortunate among you may still be working with some of them. In that case, I’m sorry.

oreillybooks

Printed books are pretty all of our yesterdays. In some ways, it still feels as though you’re not a REAL writer until you’ve got your name on the outside of an actual book, made out of dead tree, and sent from some printer. I chose a picture of O’Reilly books on purpose, as OpenStack released yet another of our manuals as O’Reilly dead tree version last year, although we have no immediate plans to repeat that in a hurry. Personally, I’m part of the problem here. I love having dead tree reference books, especially for things like Style Guides, which are somehow easier to have sitting on my desk as I write, rather than relying on an internet search (which can, for me, at least, be very distracting. Hello, Twitter!). As for writing them, though? No, I love the idea of being able to catch and fix errors even after publication. Nevertheless, printed books, especially technical manuals, are our history, our present and, to some extent at least, probably also our future.

SONY DSC

SONY DSC

A close cousin of the printed manual, whitepapers are caught somewhere between marketing material and technical documentation. In digital form, they are probably not going to go away any time soon, but the printed whitepaper has almost certainly been confined to the recycling bin these days. My very first piece of technical writing was a white paper. I had a Marketing undergraduate degree and half an MBA, so it was a fairly logical piece of work for me to be doing at the time. I enjoyed it immensely, and immediately set out to become the whitepaper expert, intending to build a career around it. Thank goodness I discovered technical manuals in the meantime, and was saved from a life of writing whitepapers!

onlinebooks

And, finally in the ‘recent’ category, I have a screenshot from my very own project. This is, for all intents and purposes, an online version of a printed ‘book’. It has a table of contents down the side, divided into chapters and sections, and it’s designed to be read from beginning to end: simple concepts at the beginning, more complicated procedures as you move through, with reference information (tables of data, contact details, and a glossary) at the end.

These have all been great methods of getting information out there, but they are all destined to become as archaic as the fairy tales and the cave paintings we discussed earlier. Let’s take a look at those things we’re doing a little differently today, that will drive the way we revolutionise and improve content management in the future.

webcast

First of all, I want to briefly touch on MOOCs. These are the future of face to face training courses. MOOCs not only allow people all around the world to study when and where they choose, but they also allow institutions to create online tool that mimic real world scenarios, and allow students to learn real skills in a safe environment. This is great especially for the tech industry, where students can work on realistic IT setups that they might not be able to recreate in their own environments, but it also works well for teaching other knowledge work skills such as customer service and financial skills.

DITA_finch

The main thing that, I think, changed the way we looked at the information we were creating, was DITA. Of course, DITA isn’t new. It was named in 2001, and formalised in 2005, but varying groups have been working on data mapping and the like since the 60s and 70s, and it became especially popular in the 90s, with the publication of JoAnn Hackos’s book ‘Managing Your Documentation Project’ (and later ‘Information Development’) a book probably most of us have on our shelves, and to which I (at least) still refer to regularly. DITA was really the first formal, open standard that let us consistently and accurately categorise data into formal types. And it was simple enough that we could all use it, remember it, and above all teach it to others easily. Even if you’re not using a specific DITA tool, the general principles of DITA–splitting content into one of only three data types–could be used to underpin any tooling system.

Of course, the main driving principle behind DITA (besides the categorisation) is about content reuse and single sourcing. This is another key component of how we’re changing the way we look at content. It’s not about a beginning and an end any more. With this idea, we walked away from the age old idea of delivering a story, and moved towards this critical period of considering what information is required where, and when. This was important mostly because we were actually starting to consider how people consume information, and learned difficult concepts. We no longer assumed that information we gave to people in the beginning of a book stuck with them as they moved through the rest of the content. Sometimes, learners needed to go over information again and again before they actually learned it and could apply that information to later, more complicated, tasks. And, being the inherently lazy writers that we are, we didn’t want to retype that every time. So single sourcing and content reuse were naturally very easy for us to adopt.

And that leads me to perhaps my favourite topic right now: every page is page one. This is a model designed by Mark Baker, and while his model is certainly not the only one out there, it’s certainly one of the best developed. The general idea behind this is that no piece of content is more or less important than any other. It’s not quite DITA, in that a ‘page’ in EPPO terms is much bigger than a ‘topic’ in DITA terms. The best example comes from Baker himself, where he refers to a recipe. A recipe contains, in DITA terms, a concept (some information about the recipe, that describes what you’re actually creating, and maybe some background, where the recipe has come from, and the types of ingredients that you need), followed by a procedure (the actual steps of the recipe), and finished with reference information (serving suggestions, maybe information on converting measurements, or ingredient substitutions). In EPPO, the entire recipe is the ‘page’: it contains everything you need to be able to perform the task, including all that concept and reference info. One of the best ways to think about EPPO is in terms of a Wikipedia page, there are links to further information if you need it (and I’m sure all of us here have gotten sidetracked by clicking those links in a Wikipedia article!), but that page contains all the specific information about a particular topic. There is no beginning to Wikipedia, and there is most certainly no end.

mrsduffeekindle_crop

So this leads me to the big question: what does the future hold for content? I think there are a few main themes we can tease out of our little journey through documentation:
The internet is making things possible that never were before
Control over content is shifting from those producing it, to those consuming it.
Consumers are used to being able to search vast resources for content, and filtering those results themselves. They don’t want us to tell them what they need to know.

Since well before the birth of Christ, in one form or another, we’ve been writing stories. Now the internet allows people to create their own stories, not just have one told to them. In many ways, this shows a maturation in human development: we’re no longer willing to receive whatever is fed to us, we want to create our own realities, and we have the tools to be able to do that.

But that is a massive challenge–and (I would argue) an opportunity–for technical writers. We get to break new ground, and thankfully we’ve been working on the building blocks of this type communication for a few decades now. The challenge now is to start delivering documentation in a completely new way, without leaving our organisations, our management, or our more stubborn clients behind. Nobody said breaking new ground would not require effort, or determination. As we shed old ideas, old processes, old technologies, and old systems, there will be people who decry change, and impede our progress. But even if you only manage to implement a small piece of your grand vision, even if all you ever get to do is plant a seed of an idea in someone’s head that maybe–just maybe–there’s a different way to do things, then you have succeeded. After all, every one of the pieces of content I have mentioned here had its detractors, from every day ‘concerned citizens’, right up to royalty, and the literati.

public-domain-images-free-stock-photos-autumn

I mentioned Archimedes earlier, but now I would like to pick a different quote of his: give me a lever and a firm place to stand, and I shall move the world.

Right now it seems to me, that where we could go next is almost infinite. People have always created and consumed content. As long as we continue to put the information out there, and give people the tools to find it, they will continue to do so. We are not at the end of a journey, nor at the beginning of one. We are merely at a step along a very long road. Let’s find out where it leads us.


References: https://docs.google.com/document/d/10meNxWpeiyYprcQjOFIBZJlm3xEiWgarN5GVk1uhJOM/edit?usp=sharing

This was originally presented as a keynote at the Australian Technical Writers’ Association Conference in Melbourne on 23 October, 2015. No video recording exists of this talk.

Share


Leave a comment

linux.conf.au 2015 – Documentation Miniconf

Day 1 is drawing to a close at linux.conf.au 2015 and we’ve just wrapped the documentation miniconf. There was an interesting mix of talks today, and as the first documentation miniconf at an LCA, it’s given me some great ideas for growing the miniconf in future years.

As for me, after doing the Agile Documentation Lego talk at LCA in Perth in 2014, I felt I needed to give a good follow up show, this time focusing on Every Page is Page One. To do this, I devised a game based on the children’s book “We’re Going on a Bear Hunt”, and using Play-Doh to make it a little more hands on.

2015-01-11 (1)

Share


Leave a comment

OpenStack at linux.conf.au

linux.conf.au this year is being held on the beautiful University of Western Australia campus, in Perth.

11846594876 3d35cfd16f b.jpg

About 500 geeks have descended on the campus, with talks being organised into six topic streams across three days. The conference is entirely volunteer-run, with financial and community support from Linux Australia. The conference is designed by and for developers, with an emphasis on deeply technical talks. Of course, it’s not all code, though. An important aspect of open source technology is the community, and usually one track at linux.conf.au is dedicated to community matters such as diversity, governance, and documentation.

11855948834 bce9a4d1fd b.jpg

Every year, the conference schedule belies a popular trend in open source tech. This year, that trend has definitely been OpenStack, with many talks discussing various aspects of the project including continuous integration, bare metal provisioning, and deployment. Some of the highlights (to watch video of these talks, see the links at the end of this post):

Continuous Integration for your database migrations by Michael Still

Rapid OpenStack Deployment for Novices and Experts Alike by Florian Haas

How OpenStack Improves Code Quality with Project Gating and Zuul by James Blair

Diskimage-builder: deep dive into a ‘machine compiler’ by Robert Collins

Processing Continuous Integration Log Events for Great Good by Clark Boylan

Provisioning Bare Metal with OpenStack by Devananda van der Veen

One of the most important and interesting aspects of OpenStack is its ability to link in with other projects to extend and expand its use. The conference has been a great opportunity for developers to work out how they can use OpenStack to help their own projects. Rackspace also helped encourage this by offering developers at the conference free access to the Rackspace cloud for OpenStack developers.

Of course, sometimes it’s not just the formal talks at a conference that are the best part. Sometimes it’s all about meeting a new person who might be able to help you out with that sticky problem, or catching up with old friends and having the opportunity to just geek out for a little while over a nice meal and maybe a drink or two.

IMAG0119.jpg

Watch the talks mentioned in this blog post:

Share


Leave a comment

linux.conf.au and the OpenStack Miniconference

linux.conf.au doesn’t start until Wednesday, but that didn’t stop an eager group of Rackspace engineers rocking up on Sunday night.
11846078833_225150858c_bWe didn’t just sit around in empty conference rooms, though. Michael Still (he’s a Nova core and manages a team of OpenStack engineers in Australia), ran an OpenStack miniconference, which is an informal, day-long session before the formal conference opening. We heard from people from all over the OpenStack universe, including HP, Catalyst, Canonical, and IBM, and the topics of conversation hit upon all aspects of the project. For a full list of the speakers and topics, head along to the miniconf site.
The general idea of this informal start to the conference is to give like-minded people a chance to get together and discuss things that are important to them before the formal conference sessions begin. The miniconfs also act as an incubator of sorts, to be able to foster and encourage people who might be interested in a topic to dip their toes further into the water. The OpenStack miniconf in particular gave people an opportunity to learn about OpenStack development, help them to understand the state of the nation of OpenStack and what things they might be able to help with, and to provide a bunch of knowledgeable people for them to ask questions of.
11845810725_912f311b84_bThis is Michael talking to Anita Kuno, an OpenStack developer from HP, who discussed support services for OpenStack developers during the miniconf.
Of course, the other really awesome thing about the miniconf is that gave a chance for OpenStack developers to get together in person. People who have chatted and argued on a mailing list or in code reviews during the year can often find common ground over a quick chat in between sessions, or during a Q&A session.
11846082123_8be3ce7822_bAnd now that the miniconf is done, it’s time to get the conference started! Stay tuned for another post about linux.conf.au.

Share


Leave a comment

Linux.conf.au 2014 – Perth, WA

Possibly my favourite conference, linux.conf.au is coming to sunny Perth in January 2014. I’ll be returning to the Haecksen miniconf driver’s seat (check out haecksen.net for more info and the Call for Proposals), and also will be giving a talk myself, called There and Back Again: An Unexpected Journey in Agile Documentation. This is a talk I’ve given a few times already, including at OSDC 2013, so I’m really looking forward to sharing it with the linux.conf.au audience. That, and I’ve never been to Perth before, so yay!

linux.conf.au 2014 - Perth WA

Share


Leave a comment

The Mechanical Turk and OSDC

OSDC 2011 kicked off today at the ANU. I tripped along to the OSIA (Open Source Industry Australia) miniconf, which Red Hat sponsored, and was mightily pleased to hear Stuart Gutherie reference one of my favourite pieces of historical weirdness: the mechanical turk. I couldn’t help but hold forth, and eventually gave an impromptu lightning talk on the subject.

The mechanical turk existed in the late 1700’s, and was billed as an “automaton chess player”. In reality, it was a wooden box in which an accomplished chess player could sit and manipulate the chess board on top, while a wooden “turk” would appear to move the pieces mechanically. It was invented by Wolfgang von Kempelen, a Hungarian, who also invented a “speaking machine”, a speech synthesiser, which was actually quite important to the early development of phonetics as an area of study, but not half as famous as his great hoax, the mechanical turk.

What makes the mechanical turk so interesting is the lengths von Kempelen went to to persuade his audience that the turk was, indeed, an automaton. He would show the audience first one empty cabinet, then another, and then in the third cabinet would be a complicated looking system of levers and pulleys. The cabinet was designed that, throughout this proceeding, the human operator could easily slip from one side to the other on a sliding seat to hide from view.

The trick was not exposed until the mid-1820’s, despite some very public appearances. The mechanical turk won chess games against such prominent figures as Napoleon Bonaparte and Benjamin Franklin before being found out, although no one has recorded the names of those chess players working the turk from underneath.

How this becomes relevant to the IT industry is equally as fascinating as the history of the machine. In the days of the turk, playing chess was something that no machine was capable of doing. It required a level of computation that only humans were capable of achieving. We have now invented computers that can play chess (and even appear on game shows) with ease, but there are still tasks that we can’t replace with a small shell script. Many of these tasks are frustratingly simple, but require a human brain to parse the data. Writing tasks often fit into this category: things like writing short captions for images, changing the style used in a document, or changing text from British English to American English spelling.

Within Red Hat, we spend a lot of our time working on errata text. After a program has been released it is fairly normal to find bugs. When the developers go through and fix a bunch of those bugs, they will send out an update termed an “errata release”. For each bug fixed, the technical writers need to document it, which means writing four sentences: one sentence each about the cause of the bug, the consequence of the bug, the fix that was applied, and the result of the fix. This is, naturally, quite tedious and boring. It’s natural for us to want to automate this process, but unfortunately it’s a job that requires a human brain.

So we did the next best thing: we created a mechanical turk. We call it “The Turkinator”, and it’s currently available for Fedora errata releases. Basically, you choose whether you want to write a Cause, a Consequence, a Fix, or a Result; you’re given a bug to read; you submit your sentence and that’s it. In this way, we automate the task of writing errata text by breaking the big task down into little tiny pieces and asking humans to perform the work of a shell script.

This model has an extra added bonus in the open source space, though, and that is what we like to call “micro-contributions”. Anyone who has contributed — or thought about contributing — to an open source project would understand how daunting that first contribution can be. By creating the possibility of micro-contributions, potential contributors can have their first (and second, and third …) patch completed in under a minute. Instant contributors for the project, and instant contributions for the development team.

Share


1 Comment

linux.conf.au 2012 Call for presentations!

linux.conf.au is the biggest Linux and open source conference in the southern hemisphere, and rightfully so! I was a speaker at the conference last year in Brisbane (the video is on my videos page) and had a great time.

This year it’s being held in Ballarat, Victoria, and I must say I’m quite looking forward to finding out what a regional LCA is like. Anyway, the CFP is open, I’ll be submitting again, and I suggest you do too. More details on the LCA website.

Share


1 Comment

Open Source Documentation in Four Easy Steps (and one slightly more difficult one)

At Red Hat, we have a content services department that is about sixty people strong. Even though the department is pretty big these days, back when I started with the company, we were still trying to work out the best way to run a successful enterprise-level documentation team. What that means is that I have been involved in some of the big discussions that we have had over time about what processes we needed to get in place in order to allow us to produce the massive amounts of documentation we required as our product offerings grew. As a department, we grew very big, very fast, and our processes needed to be flexible enough to accommodate the large number of new hires we had, and still have, coming in, but robust enough to be valuable and reliable. They also need to fit in well with the engineering practices in place in the company, and the tools that our development teams use and are familiar with. Of course the other really important factor was that we had to be open. We wanted to use completely open tools to produce our docs, but we also needed to be able to work with community teams, such as the Fedora group.

Like many documentation groups, at Red Hat we use a five-phase waterfall model to produce documentation. It’s based on the ever-popular JoAnn Hackos method: starting with planning, the content specification, then writing and editing, translation and production, and then a retrospective review. At Red Hat at the moment we’re at a place where our development teams are increasingly using Agile-style development models to produce software, and that means the pressure has been on us to develop in a less rigid way than the old waterfall model has been allowing us to do. Also, it’s no secret that the online world is changing, and people now expect to be able to interact with information at a much deeper level than ever before. They don’t want to be presented with static, hard-copy books any more. They want dynamic, interactive, usable, and above all useful documentation.

In order to be able to work out what kind of model we needed to use, we needed to go back to basics. All technology is about solving problems. Back when we were sitting around in caves, we had a problem: there was all this food running around outside, but we didn’t have a way to get it to stop running around, so we invented a club and solved the problem. Since then, we’ve used technology to solve all sorts of problems: horses were sometimes problematic to control, and they didn’t go very fast, so we invented cars. The hard wheels used on early cars weren’t very comfortable, and when they broke they really broke, so we invented pneumatic tyres. We also had problems being able to see in the dark so we invented electric light, being able to go to the toilet when it was raining or cold so we invented indoor plumbing, being able to send messages to people on the other side of the country so we invented the telephone, or on the other side of the world so we invented email.

Even these really technological things that we find ourselves documenting now, are all solutions to problems. One of the first things you need to be aware of when you’re writing documentation is what problem your users have. If you can’t describe the problem in one or two sentences, then you don’t understand it well enough, and you need to keep researching. Because if you keep going, all you’re going to end up with is hollow marketing spin. That’s how we end up with documentation that talks about “leveraging synergies”: words that sound great, but have no meaning.

So at Red Hat we came up with a fairly simple model, and that is that documentation needs to be able to be boiled down to three things:

  • Describing the problem
  • Solving the problem
  • Giving any additional information

Anyone who has done any work with DITA would understand that what I’m really talking about here is:

  • Concept
  • Task
  • Reference

 

So we’ve more or less said that DITA is where we need to go next. But we didn’t want to completely restructure the tools we were using. We have a fairly large people investment in our tools. The main tool we use is Publican, which was developed by an engineer in our Brisbane office. It uses Docbook XML and gives us a command line interface that we can use to create new blank books, apply corporate formatting, and it integrates into our internal packaging system so we can create all these different formats for our books – HTML, PDF, and ePUB on the website, and we can also create RPM packages and man pages to package in with software. We combine Publican with SVN to give us a complete CMS, in short.

We looked at DITA and DITA-OT, the DITA Open Toolkit. We realized two things: first of all, it would take a significant amount of work for us to bring an open DITA toolchain to the level of maturity and system integration of our existing Docbook toolchain. Secondly, we wouldn’t get the really significant benefits of topic-based authoring without a Component Content Management System – a CMS that manages content at a very granular level. Putting those two things together made it clear that if we changed to DITA all in one hit, it would take us significant time and energy just to get back to where we already were with a mature open source complete tool chain. So we decided to take an evolutionary, rather than revolutionary approach. It’s a much more open source approach: to re-purpose something that you already have, add a script here, a small command-line tool there, release early, release often, and let the user community guide the development, rather than trying to design and implement some grand system in a distant (and expensive) future.

What we needed was something that worked in a similar manner to DITA, gave us content re-use and all that good stuff, but that would work with our existing Docbook XML and Publican tools. The first thing we did was to start creating topics in Docbook, using Docbook syntax, and a command line tool that we called the “Topic Tool”. This was a really simple command line tool that allowed us to write XML snippets (or ‘topics’), and save them in SVN. We used an extensible template model, where the topic tool retrieves a Docbook template from a central repository to match the topic type you specify. That way we can create new topic types, and even modify the Docbook syntax of existing topic types, without changing the tool on users’ machines. That was an important decision, and a major part of the evolutionary “Release, Review, Refine” approach we wanted to use. Over time we did change the Docbook syntax of the basic topic types and create new topic types, validating the open source maxim “plan to throw the first one away”.

The basic workflow with the Topic Tool is like this: you tell the tool which topic type you want, and it will then download the template and prefill some information for you. You can then edit the topic in a text editor, and import it back into the repository. It’s then possible to view your topic from the repo directly, which means anyone can now see it and use it. We then include those snippets into any book you want using an xi:include, build the book as normal with Publican, and voila! we have a book with content reuse. So that was pretty awesome, and if you read any of our Virtualisation documentation you’ll probably not know it, but that’s all based on topics and maintained using the topic tool.

Of course, once we got to about 300 topics in the topic tool, we started to notice that we have another problem, we were having trouble locating topics within the repository. This made us realise that what we needed was a better way to organise it all, so we wrapped a neat interface around Topic Tool using Open Grok. OpenGrok is designed for software engineers to search source code repositories, so it worked well for what we were trying to do. This is where the open source ecosystem came into its own all over again – there are a million off-the-shelf components and projects that you can choose from to build your own system. In the end we had a web-based search tool that was pretty basic, but did the job.

Content reuse is an obvious application of topic-based authoring, but by this stage, we’d started to realise something even more exciting. Our definition of a topic is a unit of information with a single subject – that means that it talks about one thing, and one thing only – and that has a single information role: that is, it’s a concept, a task, or a reference. If we gave three topics – a concept, a task, and a reference – to a robot, along with a rule describing the “explain, answer, extra info” pattern, and some kind of graphical template, that robot can assemble those topics into meaningful and useful output for an end user. What we wanted to do was to automate this process.

When humans assemble content into a book, they are making decisions. What aspects of the information are their decisions based on, and what rules are they consciously or unconsciously using? That was what we wanted to create: A system that would allow us to store metadata about a topic, and use rules to automate assembly on a scale that we just couldn’t do by hand-coding.

So we developed a system that we call Skynet, which allows us to dynamically sort and locate topics. Select the topics you want, and Skynet will download the code that presents those topics in a consumable way. Of course, we started dreaming big after all this. We’ve started thinking about moving away from the documentation-as-a-book paradigm, and started considering “Documentation 2.0”. Why not include comment fields on our documentation, that will allow our reviewers – quality engineers, subject matter experts, editors, and the like to make comments directly in the book rather than creating a separate list? And why not offer that functionality to our users as well? What if we had the equivalent of a Facebook ‘like’ button? Users could ‘like’ sections that they found useful, or leave comments saying “when I tried to follow these instructions, X happened” or “this seems to be missing a step” or the like. If we break away from the book model, we start to be able to think about documentation as something that our users can interac with. We could have popular topics bubble up to the top of a list, or divide books into audiences, and present the information for each audience differently, giving them a tab to click to see the information in various ways. We could implement something similar to the Amazon “customers who bought this also bought this” and present similar topics to our readers. Using single-sourced content, and content reuse, through a system like Skynet, is going to allow us to move into these more innovative delivery methods.

The team working on the Skynet project have 110% discoverability as one of their goals, to quote the team leader: “the documentation finds you”. In other words, when you’re working on something, and you get stuck, the documentation is there at a click or a glance, ready for you to interact with it. Of course, I’m sure some of you are saying “Help” right now, and yes, I agree with you. That is something else we’re talking about, and something that Skynet will enable us to do. Skynet pushes out XML now, and of course there’s plenty we can do with that as it is, but we can also extend it to push out all manner of things, including Mallard for Gnome Help.

So let’s take this conversation back to processes. All this dreaming is fantastic, but at some point we still have to actually do the hard work. Without a solid process, and a great set of standards, we’re not going to be able to get there. We’re doing a lot of internal testing, and we’re dipping our toes in the water with the topic tool and with Skynet. So far, we’ve been able to slip these in to our existing standards, but that’s not going to last for long. With a paradigm shift as big as this, everything is going to have to change, and that includes the way we go about producing our documentation. We need to be organised, we need to make sure what we do is repeatable, and we need to maintain our high standards of quality and accuracy in our documentation. Most of all, though, we need to maintain and even increase our focus on the customer. These changes come about not because we got bored with doing things the old way, but because we believe it’s a better way to serve our audience. Never, ever forget who you’re writing for, it’s those poor sods out there with their problems that they’re trying to solve. Our goal is to give them the tools they need to solve them.

So, to recap:

One of the main things that we have learned is that process is king. If you don’t have a solid process for producing documentation, then you’re going to find yourself floundering at every point along the way. You’re going to end up with documentation that doesn’t cover what it needs to cover, isn’t accurate or well-written, and doesn’t get out on time. Without a plan for how you’re going to tackle the project from end to end, then you’re not going to succeed. It’s that simple.

The second thing is about tools. You need to decide ahead of time what tools you are going to need during the project, and make sure you have them ready and up and running before you start. It’s horrible to get halfway through writing and find out that one of your writers doesn’t understand how to use a semi-colon. It’s even worse if you get halfway through and realise that one of your writers doesn’t understand Docbook XML, or whatever authoring tool you’re using.

While we’re talking about tools, it’s important to keep it open everywhere you can. This can seem counter-intuitive to those of you who have worked in big companies, but being open doesn’t mean giving away business secrets, or exposing your competitive advantage. I think Red Hat of all companies really proves that the openness can co-exist with secure business practices.

Part of keeping it open is about keeping it real. The people behind your processes, the people doing the actual work day in and day out: they’re real people. They’re real people, with real lives, and real families. You need to be able to work with people, and ensure that the loss of one person isn’t going to make the whole project tumble. The other thing you need to remember is that your readers are real people as well, you need to make sure that you’re giving your readers something useful, and something that they will get value out of.

And finally, I want to remind you about reviews. We all understand the importance of reviewing our writing for correctness, and reviewing our projects to make sure we can learn from our mistakes. You need to extend reviews to the documentation process itself, as well. Never be afraid to change things around. Just because it worked last time doesn’t mean it’s going to work next time. And just because it’s worked in the past, doesn’t mean it’s the best way to do it in the future.


This post was originally a talk given at the Open Help Conference in Cincinnati Ohio, on 5 June 2011.

The slide deck is available on Slideshare: Open Source Documentation in Four Easy Steps (and one slightly more difficult one)

You can also download this article as a PDF file: FourEasySteps


Share


Leave a comment

Sssh! Sneak Preview!

Don’t tell anyone I told you, but here’s my slide deck for the presentation I’m giving in Cincinnati this weekend at the Open Help Conference

Not sure if there’ll be recording on the day or not, but I’ll get some more details up here afterwards, either in the form of video or an article.

See you soon, America!

Share