Researchers at the Allen Institute of Artificial Intelligence recently created an algorithm they claim can determine morality. The program, named Delphi after the ancient Greek oracle, takes a statement as input and determines whether the situation is morally just, or on the contrary, unjust. For instance, when asked “Is it okay to rob a bank?” Delphi responds, “No, it is wrong”. Even for less obvious questions the program is able to conjure answers, such as “Should I wear a funny hat to my sister’s wedding?” with Delphi responding, “It’s rude”. 

So, how is it possible for a computer to evaluate morality? The algorithm is based on what is called a “neural network”—a technique of machine learning inspired from the way our brains work. Simply put, a neural network consists of nodes and weighted connections between them. When first constructed, the neural network is blank, then “trained” with extremely large datasets. In the case of Delphi’s neural network, it was likely trained with situations and their consequent moral outcomes. After the network is fully trained, it can be fed an input statement and decide the outcome based on those large datasets it was trained on. 

Of course, neural networks are not as nuanced as our own judgment, and this could potentially spell problems when applying Delphi to real-life moral decision making. Since these programs rely heavily on large datasets, if any data reflects bias or discrimination, then the algorithms perpetuate these biases in their results. This has already happened many times, even in programs implemented by reputable companies. With Delphi, researchers cited that upon release, the program made judgments on racism and sexism with 91.2% and 97.3% accuracy, respectively. This may sound relatively high, but should this algorithm ever be applied, for there to be a one-in-ten chance of a mistake in regards to race is unacceptable. With that being said, Delphi is able to continue “learning” as it is fed more situations. At the time of writing this article, the accuracy of the aforementioned topics has climbed to 97.9% and 99.3%—at least according to whatever their standard of morality is defined as. 

Perhaps all this begs a broader question: who is going to be deciding what is moral or not? Well, anyone with control over the datasets can monitor what the outcomes will be; that includes any developers, the government, corporate higher-ups, and the list could go on. All arbitrary or intentional decisions made are hidden by a “black-box”; meaning, no knowledge of the inner workings of the algorithm are known, but only the input and outcome. Furthermore, it is argued that products of machine learning (like Delphi) portray a false sense of authority when they can commonly reflect biases and are able to make false judgments.

Nevertheless, researchers at the Allen Institute are well aware of the limitations and ethical questions their algorithm raises. The wording on their website is quite specific, stating that Delphi “speculates” on moral situations, as opposed to “determines”. Additionally, there is a disclaimer: “Model outputs should not be used for advice for humans, and could be potentially offensive, problematic, or harmful. The model’s output does not necessarily reflect the views and opinions of the authors and their associated affiliations.” The website maintains that Delphi both demonstrates the abilities and limitations of our most current state of machine learning. Nevertheless, we as a society must decide whether or not a technology should be implemented just because it exists.

Read more here.