Can governance evolve to hard-wire public interest into the way things are done?

The use of data and digital traces for urban services raises new challenges for guarding the public interest.

To help address this, my colleagues and I have been developing new forms of what we call governance clinics: strategy workshops to explore and resolve novel challenges for data governance in public sector service projects. It’s been a pleasure to work with Linnet Taylor and Merel Noorman of Tilburg University, and Thijs Turèl of Amsterdam Metropolitan Solutions. After having run a couple governance clinics, it is all the more clear that the key challenges for more just & ethical data projects are structural.

The agile development teams in city municipalities I’ve spoken with recognize their unique position: they have the potential to develop viable alternatives in digital service provision that are not based on extractive data practices. There is a clear vision of the potential to be a legitimate alternative to Big Tech. In itself, this is super exciting and I want to support this. To do so, I am writing this blog to reflect and improve on the processes for digital public sector projects.

Challenges for good agile governance practice

The way to provide viable alternatives to Big Tech is framed as simple: the tech solution needs to be effective. It needs to be done well, and fast. Otherwise, the opportunity window closes. To move fast, remove the obstacles. Development teams choose processes, strategies and project management philosophies like agile and scrum because they have proven to work well, fast.

Thing is, the open-ended, iterative, often experimental nature of agile working can be a challenge for the ways we think about “doing good”.

Here I focus on agile as a loose philosophical grouping. I think the same points apply to scrum, lean, and the general trend of software development currently.

If you are reading this and you are not familiar with the processes of software development wondering what is agile or scrum — fear not! This is the most useful (and hilarious) graphic I have been shown and I highly recommend it. It gives you the main ideas needed for this conversation: software projects need clear goals, the fat is trimmed, and process is sacred. The ‘actually could we’-type requests that come halfway through a project life cycle — which could be where a lot of “ethical”-type requests come in — are to be actively defended against.

There are two major challenges for agile governance in the literature:

1 : Modularity challenges accountability.

The modularity of current development practice means there is rarely just one actor responsible for a particular service. A product or service draws on many different input streams, of both data and actors. Different B2B service arrangements, a whole host of APIs, cloud providers, third party services, the list goes on. What if the data used for analytics was bought by a broker who collects data from many different places? This panoply of actors means it is not always clear who is the addressee for norms of privacy or accountability. Who do you pin down at the end of the day to check if you agree or not, when the product or service is a motley mix of many different elements?

For those of you more intimate with the European Data Protection regime, this is why identifying the precise ‘controller’ in the GDPR can be difficult. This argument on modularity is put forward very well by Gurses and van Hoboken in their 2017 paper, ‘Privacy After the Agile Turn’.

2: Iterative development needs iterative oversight.

The adaptive nature of agile development means there is a certain open-endedness. This is its strength: as circumstances change, production can adapt and shift directions. This strength becomes a challenge if that’s all there is.

In some lean practices of development focused only on the minimum viable product, governance and bigger picture questions are out of scope. Worse, they can come to be seen as distractions that developers need to be actively defended from. There are instead many smaller teams working on many smaller short term goals, that will quickly move beyond any preliminary agreements made about intentions at the start of the project. If the project got an OK from whoever is overviewing it when it started but then the shifts gears, is that still valid? For a really good insight into how these challenges work in practice, check out Noorman, Swierstra and Zandbergen’s 2017 chapter on Responsible Innovation.

Existing procedures can be unwieldy. For instance, if the only check is an ethics reviews before projects start, these will often bloat and become too bureaucratic. There is a tipping point where review processes will get sidelined. Nobody wants to do the impossible bureaucratic demand. The process loses its functionality and there is nothing left.

Software development is adaptive. This means questions of ethics and governance need to evolve with the project. It is not a software developer’s job to fix issues of justice and ethics, this is what review, reflection and oversight does. This is governance.

Look to governance, not ethics

You may wonder why I stress governance, not ethics.

‘Ethics’ means too many different things to different people. Moss & Metcalf wrote a great piece illustrating the many different meanings of ‘ethics’. As a term, it’s vague, and without further clarification, that vagueness is unhelpful.

In some cases, ethics is at best, a rubber stamp of approval to be able to continue the project anyway. There is rarely the option of no, this shouldn’t be done. In a 2020 open-access paper, Taylor and Dencik argued that the ethics industry is parasitic because, much like corporate social responsibility in the environmental domain before it, it legitimises existing projects that self-perpetuate.

For the sake of this argument, I am going to state loud and clear that city engineers really do want to do good. (In most cases, there are much better salaries available for good engineers elsewhere.) Thing is, as Dunn’s recent post highlights, “tech for good” isn’t good enough, because we all know what the road to hell is paved with.

For public interest to be preserved, what we value needs to be baked into the way things are done.

For public interest to be preserved, what we value needs to be baked into the way things are done.Designing systems need to have governance at the heart of its thinking. The solution requires that you “hard code your values into your governance so it doesn’t matter if you get hit by a bus or get acquired”.

Ways forward: Governance design

How can we make governance interventions at the heart of agile engineering systems to move towards more just uses of technology in our society?

Governance processes need to be intentionally designed in order to respond to evolving development. One be-all-and-end-all procedure isn’t going to cut it. As we’ve seen above, it will become huge and lead to increasingly being sidelined. We need several different feedback loops in multiple places to govern project development. From the start of a project lifecycle, the scope of digital public service provision projects would need to be reformulated so that a check on public interest and legitimacy is part of the project’s DNA. Then, iterative oversight can be designed to check in at multiple points of digital service development.

Here is one concrete example:

Reflection is already at the heart of the agile development process. These are the regular retrospectives and sprint reviews. Reflection and review is what makes iteration and adaptation possible: we learn through feedback and then course-correct. These reviews are proceduralised; there are formal requirements of things to check. In scrum, one set focuses on content, what the team is building. Another set of reviews focuses on procedure, how the building is done. What if a third layer of ‘public interest reflection’ was part of these regular reviews?

The details would need to be adapted to the circumstances; this is why my colleagues and I run governance clinics. There are ways to change and define the scope of projects in the public interest. Public interest is not out of scope, we have to create the space to be able to discuss it. This is one way of designing systems for auditability, which is one of the ten simple rules for big data research.

In this particular context of municipal service provision, there is an added layer of legitimacy built in. If we are talking about the tech sector more broadly, a lot of work shows how internal governance mechanisms are not enough. For instance, McDonald writes about Data Review Boards as a concrete example of civic data trusts in the wake of the Cambridge Analytica scandal.

No single suggestion is not going to solve all the major tech and data governance challenges we have right now. This will, however, do two things:

1. Create the process to hard-wire public interest reflection into engineering projects

2. Prime you to keep your eyes open for other process-based governance solutions