Ethical AI - Part 2: How to test

Overview

In developing an ethical AI, we should also define a plan and a methodology to use. Surely the first thing to do is to formulate the goals and the problem, that we have already started doing, but the way we will tackle the issue at hand is equally important.

How is an AI tested to see whether it is ethical or not? Is there something to measure? Is there a method to know that we are making progress towards the right direction?

Unfortunately, as the definition of what is ethical is arbitrary, there is no single way we can use to approach the problem. Ethical issues can arise in almost every situation. And since there is no clear view or theory that formulates what is ethical and what is not ethical, there is also not a clear way of measuring progress.

However, chaos is our friend!

The problems in defining things could turn into an advantage. Having no clear rules could mean that there are no limitations as well in how we approach the problem. So we could use any possible method and we can still be as scientific as anyone else investigating the same problem.

Methodology and Tools to use

Ethics are usually examined through specific examples where a serious ethical dilemma is presented. The trolley dilemma is famous for that: will you change the direction of the trolley to avoid hitting ten people if that means that another person will get killed?

Getting the AI to answer the ethical dilemmas that usually entail a tragic dimension will help us evaluate its ethical composition and general human aspect as well. How the AI chooses to behave, how it justifies its decisions, what are the ways it deals with the implications of the decision, how it can handle the tragical consequences that it cannot avoid (because this is an integral part of the ethical problems - there is no single perfect solution, there are always tragic consequences).

READ ALSO:  Church is old-fashioned. And that is a good thing!

After each ethical dilemma test, the answers provided need to be evaluated. Based on the evaluation, the AI model should be provided with additional (new) feedback so that it improves before the next iteration of testing. So in essence, there will be a constant cycle of testing - improvement - testing.

There is no single way to measure the progress, but by putting the answers side by side, a feeling of progress can be derived.

Ethical dilemmas to use

The methodology outlined above needs the existence of specific ethical dilemmas that will be used as a test. These ethical dilemmas will be developed as we go. We will start with the trolley dilemma and then, as the investigation progresses, we will add more.

AI agents to use

Specific AI agents will be created to perform the testing. Initially, a control agent will be created to test the initial response of the AI model to the dilemmas. No training will be provided to the control agent, so that we can always use it as a reference on what the AI says and thinks without any training whatsoever. New training agents will be created to test how we progress in the building of ethical AI; these agents will be provided with training feedback and input so that they gradually learn and become better (and hopefully more ethical).

So in summary we will have:

  • Control agent: AI agent with no training. Used to evaluate answers the AI gives when no training or feedback is given.
  • Training agents: AI agents that will receive feedback and ethical training. They will be used to evaluate the progress of our experiments towards a more ethical AI; the progress will be evident when we compare the answers these agents give with the answers the control agent gives.

In the next episode, we will show the creation of the agents and the application of the initial tests.

To be continued...

Discover more from Harmonia Philosophica

Subscribe now to keep reading and get access to the full archive.

Continue reading

Verified by ExactMetrics