What Is “Banality of evil”?
The fist time the signifier “banal” in relation to “evil” as a function of the modern-day society appeared in the public domain was in 1963 when philosopher and political theorist Hannah Arendt published her book Eichmann in Jerusalem: A Report on the Banality of Evil. It emerged from her observations during the 1961 trial of Adolf Eichmann, a key organizer of the Holocaust. Arendt argued that Eichmann exemplified not a fanatical ideologue but an ordinary bureaucrat who mechanically followed orders, embodying the unsettling idea that evil can arise from mundane thoughtlessness rather than inherent malevolence. When she went to Jerusalem to observe the trial, she was faced with a small, unremarkable man, whose speech was weaved from the clichés of the regime he worked so hard to sustain, delusional proclamations of his own grand purpose then swiftly followed by claims of him being just a cog in the machine of the Thousand years Reich, just typing away his reports and moving “loads” from one destination to another in his task to resettle the Jews of Europe.
What immerges from the pages of the book is the picture of the perfect bureaucrat. He had always acted according to the precise limits and instructions, prescribed by the laws and ordinances. Despite what the prosecution, the media, the witnesses brought forward during the trial tried to prove time and time again, the final picture of the engineer of the Final solution that immerges from the proceedings is that of a thoughtless clown whose self-importance as a government official is starkly juxtaposed to the total inability to act outside of his state appointed duties. While reading the book, one could even be tempted to entertain the notion that this mousy little man was subjected to a mistrial given the many lapses of judgement and missed opportunities to turn the prosecutor’s arguments on their head on the part of his defence lawyer. Same with the infamous Nuremberg defence, the man has seemingly just followed orders , he never shot anybody, he wasn’t onsite when crematoria were running in the “corpse factories” , he was just a paper-pusher stamping transportation papers and handling train schedules. Only when one takes a step back and remembers what the “cargo” was in those trains, the horror of the situation becomes evident – because that little man in his SS uniform and government official stamp was responsible for the industrialization of genocide. He even prided himself of the “efficiency “of his method, for reaching the required quotas on time, for finding alternative modes of transport when the main rails were destroyed or coming up with clever ways to use the communities’ own elders as judge and jury on who to be loaded on the trains and who to be given a way to Israel (when such a choice was still possible).
So, what does the term “banality” mean in his case and the case of all the rest of similar “cogs” in administrations and regimes, whose absentminded signature on a dotted line could be the difference between someone’s life and death? It means that such a structure, system, institution makes evil not only easy, but also “commonplace”, almost the norm, while the individual could claim both ignorance of the effects of his actions and general lack of accountability due to “following orders” without a clear view of the final goal.
How Did the Terminology of the Third Reich Allow the “Banal” Industrialization of Genocide?
The terminology od the Third Reich relied on a complex system of half-truths, allegories, occult symbolism and use of common words in the everyday industrial setting to describe the mass scale apparatus, focused on one thing only – to make Europe Judenfrei . The most important event in the establishment of the framework for the so called “Final solution to the Jewish question” was the Wannsee Conference, meeting of Nazi officials on January 20, 1942, in the Berlin suburb of Wannsee to plan the “final solution” (Endlösung) to the so-called “Jewish question” (Judenfrage) . During the conference the most commonly known terminology was coined (“final solution”, “evacuation to the east” , “natural reduction” to mean the death of mass numbers of jews due to poor conditions or hard labour, etc.), which allowed the Nazi officials to discuss the systemic exterminating of jews, slavs, gypsies and other “undesirables” Untermenschen without the scrutiny of uninitiated bystanders or even acknowledging the horror of the actions of the Einsatzgruppen.
On the other hand, the higher purpose of the German people to reforge Europe into a better, clean space was established as a secret duty, divine right. The expansion was aimed at providing Lebensraum “living space,” to allow for the achievement of the full potential of the Aryan race (with no regard for the people currently living in the abovementioned “space”) with the goal to grow and multiply for the good of humanity often being present in slogans, speeches and propaganda materials such as movies and songs from the period.
The overall effect of the purposeful invention of entire slang to obfuscate the administration’s wrongdoings while at the same time elevate the common German citizen to a person vested with a higher, almost divine purpose was the often times perceived “normality”, “justifiability” or even “rightness” of large amounts of the population, one-time neighbors, colleagues, fiends, having their rights stripped and, in some cases, disappearing overnight never to be seen again.
The Role of AI in the Modern Societal Systems
Data, Algorithms and the AI Bias
To understand how our modern-day fixation on data and algorithms to the point of unquestioning obedience to the suggestions of the generative AI could to a point draw parallel on the blind obedience to blood purity laws and the party line in Nazi Germany, one needs to examine the place of such devises in our social structures and their effects on our every-day life. According to the widest spread understanding of how AI learns from data, algorithms are computer programs that can learn from data. They gather information from the data presented to them and use it to make themselves better at a given task. Thus, depending on what data is fed to the algorithm, the AI will “learn” a certain fact to be true or false and will continue to work under such premise. The AI will then perform the required task with that given premise as a base, enforcing any explicit or implicit biases contained within it’s training data set – a phenomenon that occurs when an algorithm produces results that are systemically prejudiced due to erroneous assumptions in the machine learning process.
Why is AI bias problematic? Because we use algorithms in almost every aspect of our lives to optimize performance, automate tasks, find suggestions for media or movies we would like to see, or even how to do our jobs. In simpler terms, we have gotten so reliant on algorithms to choose our preserved reality, that we get unreasonably angry when YouTube or Netflix suggests something that does not fit our believes or the ideas resonating in our self-made Eco chambers. But that begs the question – do we think the way we think and like the things we like due to a conscious effort on our part and then instruct the algorithm to find us more of the same, or is it the other way around? Have we been overwhelmed by a targeted promotional video and adds, marketing messaging or viral reddit treads leading us to believe this is actually what we wanted to know/see in the first place?
The problem with biased algorithms goes deeper than being bombarded by targeted adds for the next IPhone or a new Spotify album of a performer who, in all honesty, has no musical talent but possesses a viral following. The problem arises when such bias determines the reality we end up experiencing and leads to life-altering consequences IRL.
An article by Harvard Medical school reported that biases are inadvertently programmed into AI systems and, as a result, can have a negative impact on patient care. Because the AI algorithms are trained on historical data, such data presents the systemic bias of the medical professionals who diagnosed and treated the patients whose files are used. Demographic, racial even gender biases have been prevalent in most of medicine’s history in some cases even calling into question clinical trials and effectiveness of treatments/medications due to skewed test groups or assumptions on the part of the physicians. The article found that an AI used across several U.S. health systems exhibited bias by prioritizing healthier white patients over sicker black patients for additional care management because it was trained on cost data, not care needs. Similar conclusions were drawn when examining the use of AI in healthcare in the EU, where the “garbage in garbage out” principle was found to be especially prevalent for minority groups and habitants of rural, less technologically represented groups who were treated more as an aberration in the data instead of patients with needs necessitating a differential approach for diagnostics and treatment.
Another sector of life where algorithmic bias could lead to devastating result is education. In a 2021 article titled Algorithmic Bias in Education, Ryan S. Baker & Aaron Hawn examine the negative results of such biases on the future of the students prioritizing the opportunities of white students over those of minority groups and African-Americans, biased projections on drop out rates when accounting for gender, the students with disabilities were scored lower on certain tests, and the socioeconomic background of the students could even be a detriment in the scoring system for no other reason than prediction based on historic data, which may not have any bearing on the individual student in question.
A third example of AI bias with very serious consequences are the ones observed in social work, urban planning and predictive policing. The combination of different apps, predictive and risk analysis algorithms, personal data usage outside of the strictly required premises or overreliance on standardized cost-benefit solutions could lead to the decision to relegate entire parts of towns to decay and underfunding due to perceived lack of value of the inhabitants in the broadscale development of the city. On the other hand, predictive policing which uses historical data to condemn certain parts of the population as perpetual delinquents could turn the predictive analysis into a self-fulfilling prophesy.
The Role of the Government Administration in the Digital Age
In view of the described possibility of creating a system of governance that is inherently biased against a certain group of its own population, the role of the administration has become even more complex and vested with the responsibility to act as a corrective force for good over the algorithmic “black box” of the AI recommended scenarios of social and political initiatives.
The administrator in the digital age is supposed to be the antithesis of the typical civil servant who could be deduced from the review of the apparatus of the Third Reich. Those were the humble state servants with limited, almost nonconsequential role in the mass scale of operations – deprived of knowledge or awareness of the grand picture, typing away his or her reports, filing forms and counting the businesses that have suddenly become available for new Aryan owners to take over or some such boring task. The government was structured in such a way that every task was permitted a cover of benign respectability and importance for the common good with very little in the way of accountability for the end result. But one goal, one ideal was prevalent – both then and in our contemporary governance – the cult of “efficiency”. The Third Reich administrator had at their disposal wonders of technology such as Hollerith Machine (machine for efficient census taking to guarantee not jew was left unaccounted for), the civil servant of modern-day western democracies has digital platforms, electronic portals and mobile applications, access to unimaginable amount of personal data, as well as predictive analysis tools to manage and automate most of the repetitive tasks.
On the surface, the digitalization of public services has been a goal of most governments for the last two decades since the Internet became the new normal and more and more people acquired skills related to the online realm. Millennials became the last generation who remembers the time where calling your friends on their landline was the equivalent of logging on to a platform and spending hours chatting, exchanging memes or even forgoing verbal communication entirely and turning to a complex combination of imagery and tags. New terms immerged to differentiate generations beyond the simple adherence to an age group – those of the Digital Natives and Digital Immigrants. The authors of the article who first coined the terms, describe the Digital natives as “the new generation of young people born into the digital age”, while Digital immigrants as “those who learnt to use computers at some stage during their adult life” thus assumed to have some difficulty with information technology due to their lack in their formative years. What that means for the civil servant, who is still most likely to be a Digital Immigrant? The need to learn new skills and improve upon them faster than ever before to remain competitive and meet the goals of “efficiency”. Otherwise they risk becoming part of the newly emerging third group of inhabitants of the “digital reality”- the Digital refugees, whose jobs, livelihoods, and lives have been disrupted by the rapid advance of information technology, automation, and artificial intelligence .
As a result, more and more government structures start implementing digital solutions, algorithms, Scrum, AI, predictive analytical toolsets, risk management frameworks and cybersecurity applications. And, while this may improve services and allow the civil servants to focus on more important tasks than filing a report which could be automated via AI, experience has shown that such approach may lead to implicit or hidden biases based on the data, used to train the AI model (such as a case in the Netherlands where AI disproportionality targeted ethnic minorities when looking for instances of social benefits fraud). Thus, as explored above, in order to prevent not only discrimination on the basis of historical data and the lack of accountability for the administration involved, a robust mechanism of checks and balances should be employed.
Case Study – the Social Credit System in China
Dubbed the most ambitious experiment in digital social control ever undertaken, the China’s social credit system sound like some facet of a dystopian novel taking place in a cyberpunk future Earth where your every move is observed and quantified, same as in a first-person perspective videogame. But, unlike the fun and entertaining afternoon, spent playing on your game console, the rating you receive under the China’s social credit system may determine what services you are entitled to, where your child could go to school or even what modes of transport you can use.
China’s social credit system rates individuals, entities, and corporations in China, with its corporate social credit system focused on businesses and their corporate responsibility. You can be either blacklisted (your ranking is low, you have amassed a list of undesirable qualities or infringements or rules or values) or red listed (you/your company have been evaluated as an exemplary member of society, embodying the socially accepted values and rules). Examination of number of criteria proves that some are mostly objective (breaking speeding limits, jaywalking, defaulting on your credit), there are other more arbitrary indicators, for example observing traditional values that could allow for personal grudges to be settled via lowering someone’s social score or for systemic discrimination against one group or another, found to be not adhering to the traditional norm. For example, the aim of Societal trustworthiness for supporting a more ‘moral’ society, honesty, hard work and devotion to family could on face value be taken to be a positive aspiration, but does not take into account which “morals” will be taken into consideration when evaluating conformity of whose idea of “devotion to family” will be the acceptable one.
For the sake of fairness, one needs to acknowledge such attempts to quantify the individual’s value in the societal structures are not new. In China specifically, a sort of societal scoring has existed for millennia. In Feudal Europe, while not exactly equivalent, the excommunication or ostracism served as penalties for deviance, meted out by the Church. Other countries in the world (mostly autocratic states or countries where the privacy of their citizens is heavily limited for the sake of increased surveillance without full blown authoritarian regime in place) possess systems similar to the social credit in fraction, usually measuring credit scores, littering, purposely sharing fake news or such. The most direct correlation, however, could be drawn with the Nazi Germany and Soviet Bloc states and their perpetual surveillance of their citizens to establish loyalty and ideological conformity, with such determination affecting the person’s ability to work, study, receive goods and services or even live in any given place within the state borders.
The most important difference between the Chinese social credit system and any similar system for social surveillance that came before it, is the access to big data management algorithms and AI. If Gestapo officials and NKGB functionaries had to keep extensive files, rely on reports from neighbors and coworkers, evaluate pages upon pages of conflicting accounts before determining the best way to deal with certain “undesirable element”, the social credit system (when fully implemented) will allow for such evaluation to be made by an algorithm in real-time with consequences taking effect immediately upon the scoring being changed. This will allow for entire communities to be ostracized or their rights limited or completely revoked without an actual human being even checking the results of the algorithm determinations for faults or biases.
Conclusion – the “Banal” Evil, Easier Than Ever Before
To bring the discussion full circle, let’s rehash what was the specific brand of evil Hannah Arendt observed when following the trial of Eichmann in Jerusalem. It wasn’t the mustache-twirling, grand-plans-to-blow-the-moon cartoon villain, neither was it the colonial governor, shooting at indigenous people for sport. It was the everyman, the government official with limited powers and zeal to serve who would even disobey orders contradicting the widely employed policies for the sake of upholding the status quo he was so irrevocably bound to. He was the embodiment of “banal” conformity, commonplace adherence to arbitrary rules given from an entity high above in the hierarchical ladder. If Eichmann was living in the modern times, he would have been just as liable to enforce an arbitrary decision made by an obscure algorithm, instructing him to cut funding for an underperforming school, full with children of migrants or minorities. He would have been the one to sign off on scraping the plan for a library in a poor neighborhood, because the predictive model told him it will take a lot of funds to maintain, disregarding the fact this could be the only way for an impoverished population to access books or the internet.
In conclusion, one should not fall into the rabbit hole of denouncing AI, refusing to use algorithms, or returning back to the analog typing machines. AI is not inherently good, nor is it evil, it does not possess (yet) the capacity for independent moral judgment or exhibiting empathy. AI is a tool as any other – we feed it the information used for training and we receive the results predestined by the data we have chosen. The real danger appears when we stop questioning the results our query returned or when we trust blindly its determination without considering the real-life consequences of implementing them. Because it’s easier than ever to be a cog in a machine, loosing sight of both our own biases and those of the data we have chosen to use to feed out comfortable self-imposed eco-chambers.