The fourth in a series of articles by Carl Rowlands on the impact of technology on society and politics.

Palantir, in many ways, have become a bogeyman, a corporation which popularly represents the second Trump era. Clive Lewis MP has perfectly encapsulated the public disgust at their growing influence and seemingly unstoppable growth across the British state, from the Metropolitan Police to the NHS and Department of Work and Pensions and beyond.

Palantir is a company, incubated by some of the most right-wing characters who struck big in Silicon Valley in the 1990s and 2000s, among them Peter Thiel. It has geared its future, successfully, around forging political connections. It uses its financial and lobbying firepower to get access to a government office near you, and therefore gets deep access to personal data. The fact is, if we are talking about surveillance and privacy, we are also talking about the military. And Palantir‘s ties with the US military, ICE and the Israeli military are obvious and the implications are stark.  

Possibly, the new world of data and IT wasn‘t always meant to be like this. For the first few years, the internet in the 1990s was fun. Websites represented new possibilities for different media. Sometimes anonymity even allowed people to express themselves more easily. Even now, the internet offers different ways of connecting with interest groups and other activists. But things changed, perhaps with the smartphones that we carry around and voluntarily use to damage our own privacy ourselves. Together with the increasingly intrusive data harvesting and segmentation of the digital advertising world, a real race to collect data has been developing.

London and other cities in the UK have subjected citizens to a regime of CCTV surveillance for a long time, dating back to the mid-1990s. But the intensity is growing. One of the first uses for machine learning, based upon neural networks, was to recognise patterns – for example, to identify tanks in imagery. Developing facial recognition ran parallel to this, as documented by Kate Crawford, among others. It was trained on thousands upon thousands of different human faces, using data harvested from sites such as Flickr among others, as expressions, emotions and different identities could be inferred (not always accurately) from huge sets of data, pictures voluntarily uploaded by mostly unwitting users of ‘free’ online services. The use of mobile Live Facial Recognition (LFR) systems in policing is built primarily on these advances in facial recognition, reflecting expanding real-world databases. 

The early trials of LFR cameras by the Metropolitan Police, with static cameras located at Kings Cross, were considered a success, and the arrest of a few crime suspects was cited. It is probably the data collected at Kings Cross which determined that LFR was successful, and that it could be matched and linked to various other forms of personal data. And this combination effectively opens the door to a form of ‘full spectrum surveillance’, where, if necessary, information can be harvested easily from specific locations. That the Metropolitan Police has also signed contracts with Palantir shows the extent to which policing and military surveillance and operations could potentially blur, given situations of civil disturbance.

Observers have specifically noted that LFR tends to be used around areas and events with a large black participation – for example, Notting Hill Carnival. Another example would be political demonstrations, with pro-Palestinian activists already targeted by LFR systems. Before recent demonstrations in support of the proscribed Palestine Action group, the Metropolitan Police even warned participants that their presence could affect their ability to become teachers at a later stage in their lives.

Needless to say, there is a pressing need for a clear set of guidelines on how surveillance data should be used and deployed – something that even some of the police themselves are asking for. The lax and ill-defined regulatory environment provided by the Data (Usage and Access) Act, also means that there is a possibility that data resulting from facial recognition cameras could be passed onto third parties, and even potentially used commercially. The main risk, however, is undoubtedly that of ‘doxing’ – the publication of individual data containing personal details of participants. This could be used to flatten and suppress valid, legal forms of political dissent, whether left- or right-wing. 

Other questions are raised by the Ministry of Justice’s plans for AI: trials are being conducted on prisoners to establish the predicted likelihood of repeat offences, especially of murder. Glowingly hailed by the Labour Home Secretary Shabana Mahmood in true Benthamite fashion, the monitoring of prisoners offers the possibility of a true Panopticon, feeding in data from just about every aspect of an individual’s life, and their interactions with others.

In the meantime, the Department for Work and Pensions is set to introduce machine learning to identify fraud, claw back overpayments, and manage benefits sanctions, in combination with the access to individuals’ bank accounts enabled by the Public Authorities (Fraud Error and Recovery) Bill. The implications of this initiative, given the growth in both in-work and out-of-work poverty in the UK since the 2008 financial crisis, could be profound. It is part of an increasing tendency to ‘digitally police’ those in poverty, through processes which have limited accountability, and in which there is virtually no transparency on how algorithms are weighted and trained. It is possible to see the amalgamation of facial recognition, phone, tax and social security data being used for a variety of purposes – not least of which is political repression.

At the very minimum, we should ask for a moratorium on facial recognition technology, pending a full enquiry and binding report upon the ethics and potential abuses and biases of machine learning across all government departments, police and local authorities. This would report fully upon the price, performance and privacy implications of existing programmes of facial recognition. To counter the growing ubiquity of smart cameras, government should establish a Privacy Inspectorate to enforce compliance with an enhanced set of rules on monitoring and surveillance, including on domestic ‘doorbell’ cameras, and ‘wearable’ tech, with the power to audit the usage of personal data.

If we are to introduce more surveillance, then I suggest we should change the subject. The best place to start is possibly a Civil Security Clearance system for all companies providing services to government, which has the power to investigate all corporate-related financial activity, including consideration of their tax commitment to the UK and the company’s adherence to democratic values at the highest levels. The spaces that this would leave would allow more room for insourcing, alternative strategies based on Community Wealth Building and approaches which represent a clearer set of public ethics. Ultimately, the NHS needs to be helped to build its own data analysis infrastructure and high-powered computing capabilities.

It should also be noted that the potential for workplace surveillance is also greatly enhanced by the use of AI; and, again, this relates to the ubiquity of Microsoft 365 as a package of utilities. Microsoft 365 can already provide management with a number of reports on all sorts of aspects relating to employees: the number of files accessed and created, and how many messages are being sent and received across different channels. AI, it is clear, will be able to provide scores for each employee on ‘projected productivity’, according to Key Performance Indicators (KPIs) provided by management. Ultimately, for many companies, the temptation to address behavioural aspects with tech-centred, ‘data-driven’ analysis is likely to prove too great a temptation to resist. Trade unions need to be given a legal right to examine and influence the implementation of such measurement schemes. 

But here, technology is not necessarily the cause of the abuses: it reflects the world as it is, and underlines the need for a deeper socialist transformation. We will look further into how tech policy could relate to this in the following article, which will conclude this series.

Carl Rowlands works as a learning resource creator, teacher and writer. He is based in central Europe and the UK. Earlier articles in the series can be found here and here and here.

Image: “CCTV camera” by Mike_fleming is licensed under CC BY-SA 2.0.