Dr James McKeone, Principal Data Scientist, Modo25
This started as a collection of thoughts on one of the hot-topic challenges in my discipline – ethics in data science, machine learning (ML) and artificial intelligence (AI). This is the must-have topic in every analytics conference in 2020. Frequently a forum at conferences, delegates grapple with bias in automated decisioning, the societal impact of automation, privacy concerns and inequality.
It appears to me that we are not making progress in this area and one more observation on the case for moral judgement in AI and ML applications is of little benefit. Instead, I want to present a near future – an adjacent future – not the one where robots have replaced our jobs and upended society but one rather more insidious and closer to today’s reality.
Particularly topical at present, what services do we rely on in times of crisis? Access to doctors, news, transport, cashflow and immediate access to medical, food and living supplies. I would argue that the poorest, most disadvantaged members of society rely on these services not only in times of crisis, but much more frequently.
Contemplate how much of the key crisis services are already automated: Medical triage, public transport networks, welfare and banking networks, logistics and supply networks. The adjacent future I would like to consider is where at each key decision in these areas we place an algorithm or AI system.
Now, take into consideration what we know about the state of AI applications in 2020. There is the potential for bias, sexism, racism, inequality and sometimes the potential for widespread harm. None of these problems are intended by design at the outset but rather by omission or oversight, a sort of extreme or catch-22 example of George Box’s famous statement: “All models are wrong, but some are useful”.
If we drop in place AI solutions as they have currently been deployed, with entrenched bias, racist, sexist and inequitable decision making, it’s very easy to imagine those most disadvantaged members of society whom rely most on the key crisis services above to develop deep and lasting distrust of algorithms, ML, AI and data science generally.
As an industry, what we have achieved over the last 20 years is incredible, but where we are headed in the next 5 requires active thought. The needle of public acceptance of automated decisioning is shifting and distrust and suspicion will cloud my field in years to come.
There is another adjacent future, however, one where AI can break down exceedingly complex problems, drive economic growth, fulfil human rights and reduce systemic inequality globally.
These might seem like lofty goals beyond the reach of a Leeds-based start-up like Modo25. But we are not spectators. We are not in the business of building automated welfare systems, state sponsored surveillance programs, or automated medical triage.
We are committed to building models, tools and software that increase inclusion and access to products and services in the digital world and consider very carefully the impact of what we deploy.
