Addressing process fragility in legal work

Graeme Johnston / 10 October 2023

In this piece, I’m going to discuss

• fragile, robust and antifragile approaches to the process – as opposed to the substance and documents – of legal work
• why it’s worth paying attention to
• a realistic way to achieve robustness and antifragility more systematically within an organisation, rather than it being mainly something for individuals

Introduction

Let’s start with the long-recognised distinction in legal work between

• Predictable situations which can satisfactorily be handled by quite precise rules and templated documents or parts of documents.
• More complex situations which can best be handled by adapting to that situation, case-by-case. Sometimes this involves applying principles but often they’re ones derived from experience, which aren’t well articulated.

Rule-based approaches

The rule-based approach can be highly efficient where feasible and properly executed. One problem, however, is that simple rules aren’t enough for the legal and factual realities of many situations and the number of rules required tends to mushroom surprisingly when faced with a situation of even moderate intricacy. Anyone who has built non-trivial software, or automated a non-trivial process or document, will know about this.

For me, that understanding has developed from legal work on regulations and contracts, from software development and from work defining and automating processes. 

In the context of legal document automation (not my speciality), Catherine Bamford gave an outstandingly candid presentation earlier this month about this mushrooming effect:

“I recently worked on a suite of Set Up Resolutions. Once finished we had 19 different structural pathways and 2,680 moving parts. That’s 2,680 different pieces of linked and connected text that could be changed. Once automated, however, the questionnaire took the lawyers less than 10 minutes to complete. The benefits of having these documents automated for the lawyers therefore were huge and this suite was a massive success story for the firm BUT the effort on the part of the lawyer working with me cannot be played down. It took us several months of hard work, weekly project calls and iterations, before piloting with testers, to get the suite finished and made live for use. This is typical and just one type of document, in one department, in one law firm… think, therefore, of the effort needed to automate everything lawyers are drafting in all firms today.”

Such an automation also involves a significant overhead to maintain the rules, for example (in the legal world) when the relevant law and desired practices change. Someone has to understand the existing rules, then change them in a way that achieves the desired outcome without breaking anything. And if this is done often, an investment in what developers call refactoring is advisable. It all takes time and therefore money.

In short, there are some significant limitations and cost / risk trade-offs with a rule-based approach. So it tends to have been adopted in legal work in only three types of situation, or a combination of them:

• First, where the process is predictable and simple, so that the cost isn’t huge. A contract in which the only automation is to insert party names and some basic commercial terms is an example of this.
• Second, where the process is more intricate but still predictable and encountered so often it is worth investing in developing and maintaining the rules. A conveyancing process and high volumes of similar claims (e.g. debts, insurance) are examples of the kinds of legal matter which can benefit from this. Systems for managing data subject access requests or contract lifecycle are examples from legal operations / BAU (business-as-usual). 
• Third, and dysfunctionally, where the owner of the process either doesn’t care enough, or doesn’t have sufficient budget, to ensure the reliability of the rule-assisted outcomes. It’s all too easy to end up with rule-based software which leads to significant errors. The Post Office Horizon system in the UK and the Robodebt system in Australia have each recently been officially recognised to have been disastrous in this sense. Many other examples could be given.

Adaptive approaches

For the reasons just mentioned, a lot of legal process isn’t handled by rule-based systems.

Now, we should avoid simple binaries, recognising that many processes involve hand-offs large and small. For instance:

• You assemble the first draft of a contract using a rule-based automation then negotiate the final terms manually.
• You run a document disclosure process via a highly-defined process with rules to guide the team and some workflow automation in an ediscovery tool, then review the potentially relevant documents in a more manual way.

However, what’s noticeable about legal work is that a process often tends to one extreme or another.

• Highly structured and rule-bound, with human judgment and discretion being constrained to certain stages.
• Manual and adaptive, typically organised with documents and email.

For that second, adaptive, sort of process, it’s worth thinking about a distinction emphasised by Nicolas Nassim Taleb between:

• Robust: systems and processes which adapt effectively to disorder
• Antifragile: those which become stronger as a result of disorder

Effective law firms and legal departments mostly do a reasonable job at robustness, and successful lawyers tend over time to develop some level of antifragility as a result of reflecting on failures – one’s own and those of others.

However, antifragility of legal process has been a difficult, time-consuming and therefore expensive thing to develop at an organisational level, as opposed to an individual or small team level.

The signal is to be found amid a noise of words, actions and decisions. You have to be there, day after day, to catch it. And even then, it’s hard. And not readily scaleable.

To make the point clear, let’s use an example of how the absence of effective organisational learning about process can play out badly:

• I won’t embarrass anyone with names, but I recently witnessed the cross-examination of some medium-experienced professionals from reputable firms whose process had failed to address the relevant problem effectively, with significant adverse consequences.
• They’d handled it in ways which I’m sure others would not have. How did that happen? My strong impression is that there was inadequate institutional knowledge-sharing of how to tackle that particular process properly, exacerbated by communication breakdowns in the particular matter, and by missing the wood for the trees in what was undoubtedly a complex, challenging matter.
• It was excruciating to watch. My sense is that, whatever individual errors were made, the failings were primarily organisational: a failure to share process knowledge effectively, combined with the fact that you can serve quite a few years in a large firm these days while only working on relatively few large matters, each raising complex issues which you may not have encountered before.

This sort of thing presents a real problem for risk, quality, service and efficiency. Serious stuff. And it isn’t going away, given the ever-increasing complexity of laws, structures, materials and issues-to-be-addressed.

What might a less fragile approach to legal process look like at an organisational level?

The established ways that large law firms seek to share knowledge involve things like

1. Internally, expensively produced documents (guidance, templates, precedents, flowcharts…) and spoken presentations (lectures, seminars, videos…)
2. Commercially available or published versions of these, for instance subscriptions to the likes of Practical Law, professional education events – now supplemented by interactive quizzes and even games

These are, of course, important, but the first is expensive and the second is undifferentiated. Also, they don’t cover process in a sophisticated way which can easily be

• adopted for this matter
• abstracted from this matter into really good templates
• systematically improved by reference to experience other than one’s own

The friction, and therefore the effort, time and cost required, are really high with a word-centric approach:

• adoption requires effort to read documents – amid the 101 other things people have to do when starting a matter, it often just doesn’t happen
• abstracting from matter to template is hard if process is just captured in bespoke text and in people’s heads
• challenges in adoption and sharing combine to create challenges in improving – a virtuous circle is hard to get moving

I suggest that a more antifragile approach to legal process is likely to look like this:

1. Find a way to capture process in complex matters that is as easy and visual to digest and adopt, at all three of the stages above.
2. Unless you’re in the kind of situation where automated rule-based approaches can work well and justify the investment, don’t try too hard to capture logic: it risks increasing fragility. Instead, think ‘minimum viable logic.’ It’s helpful to capture a few milestones to provide focus and perhaps capture that certain things should be done before others if that’s not obvious. But generally speaking, just break down the work into large buckets (e.g. phases – though call them what you will) and then break those down into component tasks. You employ smart people: they’ll figure it out. And if you don’t, you have bigger problems.
3. Don’t make the breakdown too complicated or clever. Just enough to show the important steps. Don’t over-specify. Let people use their heads to apply it in a way best adapted to the situation. Less is more.
4. Get the virtuous circle going by using the greater ease of understanding and adoption to feed the learnings from the concrete (matters) into the abstract (templates) and vice versa. 
5. Link to more wordy things – detailed guidance, contracts, laws, hand-offs to rule-based systems, links to generative AI tools and all the rest – within those phases and tasks. Make the action – the forward motion – of the work your main organising principle instead of some more abstract legal or business categorisation.
6. Take the topic seriously by assigning appropriate people to capture and maintain the relevant types of process template. This may require stepping outside the usual comfort zones of legal knowledge teams (documents) and  process improvement teams (rules), but the value of doing so is huge.

The beauty of doing it this way is that it helps with communication in two crucial senses:

• communication within the particular matter – which is key for robustness
• communication of experience from one matter to the next, even if different people were involved – which contributes not only to robustness, but also to antifragility at an organisational rather than just individual level

Doing this effectively will still be imperfect. And it will still require dedication and skill. It’s not simply a matter of better tools. But it’s feasible in a way that you can’t achieve with rule-based or text-/voice-based approaches alone (even with AI).

Software for helping make this a reality is exactly what we offer at Juralio.

Image generated with Bing Create