On Writing, Tech, and Other Loquacities

The collected works of Lana Brindley: writer, speaker, blogger


1 Comment

Open Source Documentation in Four Easy Steps (and one slightly more difficult one)

At Red Hat, we have a content services department that is about sixty people strong. Even though the department is pretty big these days, back when I started with the company, we were still trying to work out the best way to run a successful enterprise-level documentation team. What that means is that I have been involved in some of the big discussions that we have had over time about what processes we needed to get in place in order to allow us to produce the massive amounts of documentation we required as our product offerings grew. As a department, we grew very big, very fast, and our processes needed to be flexible enough to accommodate the large number of new hires we had, and still have, coming in, but robust enough to be valuable and reliable. They also need to fit in well with the engineering practices in place in the company, and the tools that our development teams use and are familiar with. Of course the other really important factor was that we had to be open. We wanted to use completely open tools to produce our docs, but we also needed to be able to work with community teams, such as the Fedora group.

Like many documentation groups, at Red Hat we use a five-phase waterfall model to produce documentation. It’s based on the ever-popular JoAnn Hackos method: starting with planning, the content specification, then writing and editing, translation and production, and then a retrospective review. At Red Hat at the moment we’re at a place where our development teams are increasingly using Agile-style development models to produce software, and that means the pressure has been on us to develop in a less rigid way than the old waterfall model has been allowing us to do. Also, it’s no secret that the online world is changing, and people now expect to be able to interact with information at a much deeper level than ever before. They don’t want to be presented with static, hard-copy books any more. They want dynamic, interactive, usable, and above all useful documentation.

In order to be able to work out what kind of model we needed to use, we needed to go back to basics. All technology is about solving problems. Back when we were sitting around in caves, we had a problem: there was all this food running around outside, but we didn’t have a way to get it to stop running around, so we invented a club and solved the problem. Since then, we’ve used technology to solve all sorts of problems: horses were sometimes problematic to control, and they didn’t go very fast, so we invented cars. The hard wheels used on early cars weren’t very comfortable, and when they broke they really broke, so we invented pneumatic tyres. We also had problems being able to see in the dark so we invented electric light, being able to go to the toilet when it was raining or cold so we invented indoor plumbing, being able to send messages to people on the other side of the country so we invented the telephone, or on the other side of the world so we invented email.

Even these really technological things that we find ourselves documenting now, are all solutions to problems. One of the first things you need to be aware of when you’re writing documentation is what problem your users have. If you can’t describe the problem in one or two sentences, then you don’t understand it well enough, and you need to keep researching. Because if you keep going, all you’re going to end up with is hollow marketing spin. That’s how we end up with documentation that talks about “leveraging synergies”: words that sound great, but have no meaning.

So at Red Hat we came up with a fairly simple model, and that is that documentation needs to be able to be boiled down to three things:

  • Describing the problem
  • Solving the problem
  • Giving any additional information

Anyone who has done any work with DITA would understand that what I’m really talking about here is:

  • Concept
  • Task
  • Reference

 

So we’ve more or less said that DITA is where we need to go next. But we didn’t want to completely restructure the tools we were using. We have a fairly large people investment in our tools. The main tool we use is Publican, which was developed by an engineer in our Brisbane office. It uses Docbook XML and gives us a command line interface that we can use to create new blank books, apply corporate formatting, and it integrates into our internal packaging system so we can create all these different formats for our books – HTML, PDF, and ePUB on the website, and we can also create RPM packages and man pages to package in with software. We combine Publican with SVN to give us a complete CMS, in short.

We looked at DITA and DITA-OT, the DITA Open Toolkit. We realized two things: first of all, it would take a significant amount of work for us to bring an open DITA toolchain to the level of maturity and system integration of our existing Docbook toolchain. Secondly, we wouldn’t get the really significant benefits of topic-based authoring without a Component Content Management System – a CMS that manages content at a very granular level. Putting those two things together made it clear that if we changed to DITA all in one hit, it would take us significant time and energy just to get back to where we already were with a mature open source complete tool chain. So we decided to take an evolutionary, rather than revolutionary approach. It’s a much more open source approach: to re-purpose something that you already have, add a script here, a small command-line tool there, release early, release often, and let the user community guide the development, rather than trying to design and implement some grand system in a distant (and expensive) future.

What we needed was something that worked in a similar manner to DITA, gave us content re-use and all that good stuff, but that would work with our existing Docbook XML and Publican tools. The first thing we did was to start creating topics in Docbook, using Docbook syntax, and a command line tool that we called the “Topic Tool”. This was a really simple command line tool that allowed us to write XML snippets (or ‘topics’), and save them in SVN. We used an extensible template model, where the topic tool retrieves a Docbook template from a central repository to match the topic type you specify. That way we can create new topic types, and even modify the Docbook syntax of existing topic types, without changing the tool on users’ machines. That was an important decision, and a major part of the evolutionary “Release, Review, Refine” approach we wanted to use. Over time we did change the Docbook syntax of the basic topic types and create new topic types, validating the open source maxim “plan to throw the first one away”.

The basic workflow with the Topic Tool is like this: you tell the tool which topic type you want, and it will then download the template and prefill some information for you. You can then edit the topic in a text editor, and import it back into the repository. It’s then possible to view your topic from the repo directly, which means anyone can now see it and use it. We then include those snippets into any book you want using an xi:include, build the book as normal with Publican, and voila! we have a book with content reuse. So that was pretty awesome, and if you read any of our Virtualisation documentation you’ll probably not know it, but that’s all based on topics and maintained using the topic tool.

Of course, once we got to about 300 topics in the topic tool, we started to notice that we have another problem, we were having trouble locating topics within the repository. This made us realise that what we needed was a better way to organise it all, so we wrapped a neat interface around Topic Tool using Open Grok. OpenGrok is designed for software engineers to search source code repositories, so it worked well for what we were trying to do. This is where the open source ecosystem came into its own all over again – there are a million off-the-shelf components and projects that you can choose from to build your own system. In the end we had a web-based search tool that was pretty basic, but did the job.

Content reuse is an obvious application of topic-based authoring, but by this stage, we’d started to realise something even more exciting. Our definition of a topic is a unit of information with a single subject – that means that it talks about one thing, and one thing only – and that has a single information role: that is, it’s a concept, a task, or a reference. If we gave three topics – a concept, a task, and a reference – to a robot, along with a rule describing the “explain, answer, extra info” pattern, and some kind of graphical template, that robot can assemble those topics into meaningful and useful output for an end user. What we wanted to do was to automate this process.

When humans assemble content into a book, they are making decisions. What aspects of the information are their decisions based on, and what rules are they consciously or unconsciously using? That was what we wanted to create: A system that would allow us to store metadata about a topic, and use rules to automate assembly on a scale that we just couldn’t do by hand-coding.

So we developed a system that we call Skynet, which allows us to dynamically sort and locate topics. Select the topics you want, and Skynet will download the code that presents those topics in a consumable way. Of course, we started dreaming big after all this. We’ve started thinking about moving away from the documentation-as-a-book paradigm, and started considering “Documentation 2.0”. Why not include comment fields on our documentation, that will allow our reviewers – quality engineers, subject matter experts, editors, and the like to make comments directly in the book rather than creating a separate list? And why not offer that functionality to our users as well? What if we had the equivalent of a Facebook ‘like’ button? Users could ‘like’ sections that they found useful, or leave comments saying “when I tried to follow these instructions, X happened” or “this seems to be missing a step” or the like. If we break away from the book model, we start to be able to think about documentation as something that our users can interac with. We could have popular topics bubble up to the top of a list, or divide books into audiences, and present the information for each audience differently, giving them a tab to click to see the information in various ways. We could implement something similar to the Amazon “customers who bought this also bought this” and present similar topics to our readers. Using single-sourced content, and content reuse, through a system like Skynet, is going to allow us to move into these more innovative delivery methods.

The team working on the Skynet project have 110% discoverability as one of their goals, to quote the team leader: “the documentation finds you”. In other words, when you’re working on something, and you get stuck, the documentation is there at a click or a glance, ready for you to interact with it. Of course, I’m sure some of you are saying “Help” right now, and yes, I agree with you. That is something else we’re talking about, and something that Skynet will enable us to do. Skynet pushes out XML now, and of course there’s plenty we can do with that as it is, but we can also extend it to push out all manner of things, including Mallard for Gnome Help.

So let’s take this conversation back to processes. All this dreaming is fantastic, but at some point we still have to actually do the hard work. Without a solid process, and a great set of standards, we’re not going to be able to get there. We’re doing a lot of internal testing, and we’re dipping our toes in the water with the topic tool and with Skynet. So far, we’ve been able to slip these in to our existing standards, but that’s not going to last for long. With a paradigm shift as big as this, everything is going to have to change, and that includes the way we go about producing our documentation. We need to be organised, we need to make sure what we do is repeatable, and we need to maintain our high standards of quality and accuracy in our documentation. Most of all, though, we need to maintain and even increase our focus on the customer. These changes come about not because we got bored with doing things the old way, but because we believe it’s a better way to serve our audience. Never, ever forget who you’re writing for, it’s those poor sods out there with their problems that they’re trying to solve. Our goal is to give them the tools they need to solve them.

So, to recap:

One of the main things that we have learned is that process is king. If you don’t have a solid process for producing documentation, then you’re going to find yourself floundering at every point along the way. You’re going to end up with documentation that doesn’t cover what it needs to cover, isn’t accurate or well-written, and doesn’t get out on time. Without a plan for how you’re going to tackle the project from end to end, then you’re not going to succeed. It’s that simple.

The second thing is about tools. You need to decide ahead of time what tools you are going to need during the project, and make sure you have them ready and up and running before you start. It’s horrible to get halfway through writing and find out that one of your writers doesn’t understand how to use a semi-colon. It’s even worse if you get halfway through and realise that one of your writers doesn’t understand Docbook XML, or whatever authoring tool you’re using.

While we’re talking about tools, it’s important to keep it open everywhere you can. This can seem counter-intuitive to those of you who have worked in big companies, but being open doesn’t mean giving away business secrets, or exposing your competitive advantage. I think Red Hat of all companies really proves that the openness can co-exist with secure business practices.

Part of keeping it open is about keeping it real. The people behind your processes, the people doing the actual work day in and day out: they’re real people. They’re real people, with real lives, and real families. You need to be able to work with people, and ensure that the loss of one person isn’t going to make the whole project tumble. The other thing you need to remember is that your readers are real people as well, you need to make sure that you’re giving your readers something useful, and something that they will get value out of.

And finally, I want to remind you about reviews. We all understand the importance of reviewing our writing for correctness, and reviewing our projects to make sure we can learn from our mistakes. You need to extend reviews to the documentation process itself, as well. Never be afraid to change things around. Just because it worked last time doesn’t mean it’s going to work next time. And just because it’s worked in the past, doesn’t mean it’s the best way to do it in the future.


This post was originally a talk given at the Open Help Conference in Cincinnati Ohio, on 5 June 2011.

The slide deck is available on Slideshare: Open Source Documentation in Four Easy Steps (and one slightly more difficult one)

You can also download this article as a PDF file: FourEasySteps


Share


Leave a comment

Sssh! Sneak Preview!

Don’t tell anyone I told you, but here’s my slide deck for the presentation I’m giving in Cincinnati this weekend at the Open Help Conference

Not sure if there’ll be recording on the day or not, but I’ll get some more details up here afterwards, either in the form of video or an article.

See you soon, America!

Share


Leave a comment

Open Source Developer’s Conference 2011 – Call for Papers!

The Open Source Developer’s Conference (OSDC to its friends) is being held in Canberra in November this year, which is a little bit exciting, and they have just opened their call for papers.

Also this year, for the first time, they’re asking for miniconf proposals too. I would love to do a whole miniconf on open source documentation, but I’m not sure I have that kind of stamina. Of course, If you’re interested in helping me out, let me know!

I spoke at OSDC last year, when it was in Melbourne, and the footage is on my videos page. I thoroughly enjoyed the experience, so it will be interesting to see what kind of event Canberra can put on this year.

Share


Leave a comment

Taking a Walk on the Open Side

Tonight, I gave a talk about Open Source for the Canberra Society of Editors (who invite the ACT branch of the Society of Technical Communicators, of which I am a member). There was a reasonable crowd, who all seemed interested and who asked lots of questions. It is always satisfying to give a talk and to see a light go on in people’s eyes, see that spark of something flash in their faces as they learn something new, think about something in a different way, or experience a different perspective. That, my friends, is what makes it all worth it.

This talk wasn’t recorded, unfortunately, so I won’t be providing video, but I can tempt you with my slide deck. Unfortunately, my slide decks don’t tend to have many words on them, so you might not learn much, but there are lots of pretty pictures. And if you were present at the discussion tonight, thanks for coming, and I hope you found some inspiration. Please feel free to drop a note in the comments if you’re after more information about anything.

If you do want to know more, there is a fairly severely shortened version of it in this month’s edition of Words. I’ll put the full text of that up here in the next few days, or you could head over to their website and check it out there.

Share


Leave a comment

Society of Editors Playing in the Grass

Some time ago, I gave a web seminar for the Society of Technical Communicators (STC) in the US, which was an interesting experience. Considering it occurred at 4am local time, I’m not entirely certain I was at my best, but luckily I’ve been given the opportunity to repeat the talk at a more reasonable time. The Canberra Society of Editors also welcomes the ACT branch of the Australian Society of Technical Communicators (ASTC) to their meetings every month, and through this happy alliance, I will be presenting The Grass is Greener on the Open Side on 27 April at 6:30pm, at the Australian National University.

I’m constantly surprised with how closed the technical communicator’s world can be. In an industry where Adobe and Madcap rule the roost, if you mention that you work in open source, or use open source tools, people tend to nod mutely. Either they have no idea what ‘open source’ actually is, or they have no idea what working with open source tools might actually entail. And so this talk was born. It is intended to be treated as Open Source 101 for technical communicators, essentially.

And of course, if you can make it along, I’d love to see you!

Share


Leave a comment

Louisa Lawson and The Digitise The Dawn Project

Louisa Albury was born in 1848 in Mudgee, the second of twelve children. Although she was offered a position as ‘pupil teacher’ at school, she was encouraged by her parents to leave school in order to look after her younger siblings. It was a fairly common thing to happen to eldest girls at the time, but judging by Louisa’s later life, it seems that she regretted it most severely. And who can blame her? She married Niels Hertzberg Larsen (who called himself Peter) in 1866 at the tender age of eighteen, and they later Anglicised their surname to Lawson. Peter, for good or bad, spent much of his time away at the goldfields, and left Louisa at home to look after their brood of five children alone. Eventually, his absences became longer and more frequent, and by the time Louisa moved with her children to Sydney in 1883, the marriage was all but over. Left alone with five children to support, and with very little and sporadic financial assistance from Peter, she turned her hand first to sewing and washing to earn money. She also took in boarders from time to time. In 1887, she took the opportunity to purchase The Republican newspaper, a paper about which I’ve been almost completely unable to find information on, sadly. The one thing I have learned, though, is that it (apparently) “called for all Australians to unite under ‘the flag of a Federated Australia, the Great Republic of the Southern Seas'”[0]. By all accounts, it didn’t last long though, and ceased production the following year, in 1888. But Louisa’a political leanings were very much beginning to show.

Apparently bitten by the publishing bug, and probably eager to continue publishing her own essays and works of poetry, she started publishing a magazine called The Dawn in 1888. It was printed as “A Journal for Australian Women” and “publicize women’s wrongs, fight their battles and sue for their suffrage”[1]. It was the first newspaper printed in Australia that dealt with issues of feminism and suffrage, and is considered perhaps the single most important factor in the beginning of the suffragette movement in Australia. Shortly after The Dawn‘s inception, Louisa’s husband Peter died, leaving her with a large inheritance, which was immediately spent on improving the printing press and increasing the circulation of the magazine. She also hired ten staff, all of whom were women. The NSW Typographical Association did not accept female members at the time, and took exception to the fact that a magazine could be edited, printed, and circulated only by women. They took up arms against Louisa and the magazine and encouraged advertisers to boycott The Dawn and reportedly harassed the women on site.

As evidence of Louisa’s strength, she did not let this discourage her, and in 1889, she began running meetings at the Dawn offices which became known as The Dawn Club. The Club discussed issues relating to the “evil laws” made by men, and encouraged women to infiltrate male-dominated arenas such as debating clubs, and Louisa herself became the first female member of the board of management of the Sydney Mechanics’ School of Arts.
Rubbish Paragraph
The Womanhood Suffrage League of NSW began in 1891 and, hardly surprisingly, Louisa was elected to its council. She offered the Dawn offices and printing press for the League to use for meetings and pamphlets free of charge, and this remained the case until the League’s demise, despite the fact that Louisa herself withdrew from the council in 1893 after an ill-documented dispute.
Rubbish Paragraph
By the time women were given the vote in 1902, Louisa was starting to slow down. In 1900, she had a fall from a tram and was badly injured, although she was politically active again in 1902 itself, when she was introduced to the Australia parliament as “The Mother of Suffrage in New South Wales”[2]. During the early 1900’s she took several extended ‘rest’ periods from her campaigning and the magazine. She was 54, not old by our modern standards, but perfectly elderly by the standards of the day, and she had worked hard both physically and mentally all her life.

With the coming of the women’s vote, Louisa aged and so, sadly, did The Dawn. The columns grew fewer and less fervent, the advertisers gradually departed, and in 1905 the newspaper printed its last edition.

Louisa continued to write for several Sydney-based publications, and she also produced an extensive volume of poetry.

I have been unable to find out what mental ailment troubled her in her final days, but dementia appears to be the most likely. She died in the Gladesville Mental Hospital aged 72, in 1920. The fight gets to even the strongest of us in the end.

Unfortunately, The Dawn has so far not been included in the National Library’s ‘Trove’ Digitisation Project, despite it’s great historical significance in gaining Australian women the vote, and despite Louisa’s passion and fervour in promoting women’s rights of all description. Do you feel it’s an important part of Australian history? If you do, why not contribute to the project? It’s being run by the lovely Donna Benjamin and she needs your help to raise the funds to make the digitisation a reality. You might also like to follow @digitisethedawn on Twitter to keep up with progress, and to help spread the word.
Rubbish Paragraph
Oh, and as a postscript: yes, Louisa did have a very famous son, but her story is so much more interesting than that, don’t you agree?

[0] http://www.nla.gov.au/guides/federation/people/lawsonl.html

[1] http://adbonline.anu.edu.au/biogs/A100019b.htm

[2] http://en.wikipedia.org/wiki/Louisa_Lawson

Share


Leave a comment

FOSS Training

I was privileged enough to be able to attend linux.conf.au in Wellington in January. While there, I caught Bob Edwards’ and Andrew Tridgell’s talk on “Teaching FOSS at Universities” (video of which can be found here). It intrigued me.

Open source software development is very different to developing software in a more traditional, closed source environment. The aim of the course is to teach students how to go about working within the open source community. It covers the practical aspects of checking out code from a repository, submitting patches, and undergoing code approvals and reviews. It also looks at some of the less tangible aspects, like what’s accepted and expected within the community, the motivation behind project development, and governance. The course also goes into some detail about documentation.

Documentation for open source projects is not quite the known quantity that it can be in many proprietary software environments. I once had a developer I was working with describe it as “we live in the Wild West out here”, and – at least to an extent – he makes a good point. While writing for an open source project may not be as wild and exciting as that sentence makes it sound, it can sometimes be unpredictable and, at times, incredibly frustrating. Frequently, a book has been written and reviewed in preparation for a release, only to find at the last minute that a feature has been pulled from the version, a component has suddenly been renamed, or the graphical interface has had some kind of redesign. All of these things happen to open source writers on a regular basis, and frequently the only solution is to pull an all-nighter, get the changes in, and have the document released on schedule. And that’s only if you were lucky enough to find out about the change with enough time to spare before release!

So how does a writer plan for and write a documentation suite when there’s so much unknown in a project? The answer is – perhaps ironically – to plan ahead. You can’t plan for every contingency, nor should you. But if you have a plan of any description, you’re going to be better off when things start to go wrong. Pin down the details as best you can as far ahead as possible. But don’t leave it there, continue to review and adapt your plan. Keep your ear to the ground, and constantly tweak your schedule and your book to suit. If something comes up in a mailing list about a feature you’ve never heard of, don’t be afraid to ask the question – “Does this need to be documented? Will it be in the next version? Where can I get more source information?”. Another trick is to make sure you build in ‘wiggle room’ to your schedule, in case you suddenly discover a new chapter that needs adding, or a whole section that needs to be changed. If you’re consistently a few days or a week ahead of schedule, then even a substantial change should not throw you too far off balance.

Just like a ballet dancer, technical writers need to be disciplined, structured, and organised. But you also need to have grace, poise, tact, and – most importantly – flexibility.

Thanks to Bob and Tridge, I’ll be lecturing the 2010 FOSS course students at the Australian National University later this week. I’ll also be contributing the textbook that is being developed for the course. True to form, it is being built by and for the open source community, using open source tools (including Publican which has been developed in-house by some of my esteemed colleagues). Watch this space for more information.

Cross-posted to FOSS Docs
Share