What are our moral obligations (if any) toward an AI system that is far more intelligent than ourselves? If it is morally justified for humans to eat animals, is it morally justified for AI to harvest resources from humans or otherwise exploit us?

Assignment Question

Write an essay of 3-4 double-spaced pages in length (or 1.5-2 single-spaced pages) that answers one of the prompts below.

Two factors will contribute to your grade:

(A) Demonstration of engagement with assigned readings from AT LEAST some of units 6- 10. I recommend drawing on readings from units 1-5, as well. This requires the use of in-text references to specific pages in these texts. A paper with no textual references will not receive full credit. You should aim to write a paper that demonstrates careful and precise understanding of arguments and ideas from these texts. In short, your paper should prove that you have spent considerable time working on understanding the readings. That is a large part of the workload for this course.

(B) Cogency of argument and organization. A poorly organized paper that does not include an attempt at original argument, or that includes only very disorganized attempts at argument will not receive full credit. You should aim to write a paper that explains key ideas from the texts (see (A)) and then builds your own argument by responding to, or further developing, these ideas.

Prompts:

1. Some philosophers think that even if AI is not (or cannot be) conscious, the fact that it exhibits agency is enough for us to owe it moral consideration. Do you think a being with agency alone (i.e., without sentience/consciousness) deserves our moral consideration? Why or why not?

2. Can a machine (namely, an AI system) be conscious? Can it feel pain? If so—or, if not—what are our moral obligations (if any) toward it? (I.e., how should we treat it?)

3. What are our moral obligations (if any) toward an AI system that is far more intelligent than ourselves?

4. If it is morally justified for humans to eat animals, is it morally justified for AI to harvest resources from humans or otherwise exploit us? Why or why not? (*Note that the question here is more about whether you agree that the two cases are analogous than whether you think any one position on the use of animals/humans is right or wrong).

5. Describe and justify your view of the optimal way to share our world with both animals and machines.

6. Is it morally right or wrong for developers to create AI that is far more intelligent than humans? Why or why not?