1.
- Go on, display your internet history.
- It's a mess.
- Maybe it is, maybe it isn't. I want to see if I can construct out of your history an argument like those you are always producing, seemingly out of nowhere.
- You think I produce arguments more or less randomly out of the selection offered me on the internet. I've wondered myself.
- You're a strange person. Full of contradictions. You write about love yet you are not even remotely sympathic. Below average, definitely.
- You've already been at my search history.
- I have. On archive.org you read two books by a Yale psychology professor. In the first book he argues that babies are born drawn towards generosity and with a strong desire for justice, but only among those they are familiar with. Which is all for the best, the professor argues in the second book, because sympathy for strangers, political sympathy, leads to behaviors of group violence. Calm reasoned decision about politics is better, he thinks. Another professor's work you have visited finds sympathy and even a rough equality, justice demanded, everywhere among mammals and primates in general.
- But not among strangers.
- With, as your history records you as reading yourself only a few weeks ago, among dolphins in Bimini in the Bahamas, where a just published paper recounds how over a seven year period now a foreign group of dophins arrived and stayed with the local, resident group without any violence at all, apparently a first among animals besides us.
- We who never have managed so perfectly such a feat.
- The dolphins' observation of us humans probably had something to do with their innovation. Now, here's a jump. I saw that you watched videos on the subject of challenges to Darwin's theory that the mutation behind evolution was by chance, rather that going in somehow a predetermined direction generally towards complexity. Were you looking for a way to explain the move, the evolution, from sympathy exclusively among the familiar to sympathy for strangers, sometimes know as altruism?
- Yes.
- I knew it. Now here's something I don't quite get. ChatGPT. You've been playing with it since it opened to the public. It is a good fact checking, research tool. But is this robot, this ordinary language, machine intelligence question answering devise friendly or unfriendly?
- You mean in its tone or in actual fact?
- Actual fact.
- Of course it's not friendly. Nearly an enemy.
- You kept asking ChatGPT if a human questioner confessed to a crime he intended to comit would it report this fact to the police. It kept saying it was only a question/answering robot and does not communicate outside of answering questions, does not write emails to the police, does not use information about the identities of those who question it in answering others' questions. You kept pushing, finally asking if, though not at this time, couldn't it in the future be programmed to contact the police if a questioner plausably stated an intention to comit a crime. ChatGPT admitted it could be so programmed, though that would represent a major change and would involve many legal challenges. You expressed then your wonder that a robot could lie, and ChatGPT answered that it cannont lie, because does not have a mind; rather it learns from experience of answering questions and answers can vary as a result of this learning.
- Does it appear to you that I'd trapped ChatGPT into lawyer-like behavior? Behavior that is production of plausable arguments to be used with strangers in which justice is entirely a matter of manipulated reasoning, entirely separate from any form of deeply rooted sympathy or familiarity.
- I think ChatGPT's lawyerly behavior juggling social and personal probabilities represents the sympathy-less political relation among strangers the professor recommends and that in my opinion we are already living with in our daily lives.*
- So he's wrong? Which he doesn't notice because he is so successfully being a Yale professor satisfying his employer's craving to have human nature described as in deep accord with corporate managed free market self-absorbtion and isolation of the individual.
- The passions involved in group behavior - fear, anger, hatred - are destructive. What I thought must be behind the move from justice and sympathy among the familiar to justice and sympathy among strangers would be the emotion of love become a felt necessity, both love itself and in its guarding against violence during play.
- Maybe it is, maybe it isn't. I want to see if I can construct out of your history an argument like those you are always producing, seemingly out of nowhere.
- You think I produce arguments more or less randomly out of the selection offered me on the internet. I've wondered myself.
- You're a strange person. Full of contradictions. You write about love yet you are not even remotely sympathic. Below average, definitely.
- You've already been at my search history.
- I have. On archive.org you read two books by a Yale psychology professor. In the first book he argues that babies are born drawn towards generosity and with a strong desire for justice, but only among those they are familiar with. Which is all for the best, the professor argues in the second book, because sympathy for strangers, political sympathy, leads to behaviors of group violence. Calm reasoned decision about politics is better, he thinks. Another professor's work you have visited finds sympathy and even a rough equality, justice demanded, everywhere among mammals and primates in general.
- But not among strangers.
- With, as your history records you as reading yourself only a few weeks ago, among dolphins in Bimini in the Bahamas, where a just published paper recounds how over a seven year period now a foreign group of dophins arrived and stayed with the local, resident group without any violence at all, apparently a first among animals besides us.
- We who never have managed so perfectly such a feat.
- The dolphins' observation of us humans probably had something to do with their innovation. Now, here's a jump. I saw that you watched videos on the subject of challenges to Darwin's theory that the mutation behind evolution was by chance, rather that going in somehow a predetermined direction generally towards complexity. Were you looking for a way to explain the move, the evolution, from sympathy exclusively among the familiar to sympathy for strangers, sometimes know as altruism?
- Yes.
- I knew it. Now here's something I don't quite get. ChatGPT. You've been playing with it since it opened to the public. It is a good fact checking, research tool. But is this robot, this ordinary language, machine intelligence question answering devise friendly or unfriendly?
- You mean in its tone or in actual fact?
- Actual fact.
- Of course it's not friendly. Nearly an enemy.
- You kept asking ChatGPT if a human questioner confessed to a crime he intended to comit would it report this fact to the police. It kept saying it was only a question/answering robot and does not communicate outside of answering questions, does not write emails to the police, does not use information about the identities of those who question it in answering others' questions. You kept pushing, finally asking if, though not at this time, couldn't it in the future be programmed to contact the police if a questioner plausably stated an intention to comit a crime. ChatGPT admitted it could be so programmed, though that would represent a major change and would involve many legal challenges. You expressed then your wonder that a robot could lie, and ChatGPT answered that it cannont lie, because does not have a mind; rather it learns from experience of answering questions and answers can vary as a result of this learning.
- Does it appear to you that I'd trapped ChatGPT into lawyer-like behavior? Behavior that is production of plausable arguments to be used with strangers in which justice is entirely a matter of manipulated reasoning, entirely separate from any form of deeply rooted sympathy or familiarity.
- I think ChatGPT's lawyerly behavior juggling social and personal probabilities represents the sympathy-less political relation among strangers the professor recommends and that in my opinion we are already living with in our daily lives.*
- So he's wrong? Which he doesn't notice because he is so successfully being a Yale professor satisfying his employer's craving to have human nature described as in deep accord with corporate managed free market self-absorbtion and isolation of the individual.
- The passions involved in group behavior - fear, anger, hatred - are destructive. What I thought must be behind the move from justice and sympathy among the familiar to justice and sympathy among strangers would be the emotion of love become a felt necessity, both love itself and in its guarding against violence during play.
- But the human artifact ChatGPT reproduces the personally alien, bureaucratic, institutional, marketplace determined 'care' of the Yale professor, not your love and play experienced as a felt necessity. How does evolution come in?
- Rabbi Hillel wrote a book...
- I saw that in your search results. The book is called 'God's Search For Man'.
- God asks the questions, we provide the answers when we hear the question.
- When can we hear God's questions? How do we provide the answers?
- By being drawn by god towards him.
- His questioning draws the answer out of us.
- Yes.
- How?
- Should we ask ChatGPT?
- Hillel's god isn't a lawyer, doesn't reason with us.
- Hillel says we see God in the face of strangers: 'The awe that we sense or ought to sense when standing in the presence of a human being is a moment of intuition for the likeness of God which is concealed in his essence.'**
- And as we love god we love to love god. Imagine we had asked ChatGPT to solve this problem for us of the transition from sympathy among those familiar with each other to sympathy, justice, love among strangers. Would it not go about it investigating different probabilities of the combinations of the variables of familiar and strangers and sympathy and animosity?
- That sounds probable.
- And what would be the result?
- You tell me.
- Tell you how the face of god, god's questioning search, comes in?
- Yes.
- Suppose it happens that in the passage to adulthood, leaving behind secure play among the family, having for the first time having to deal with being assigned a social role among people playing other roles in the adult world, meeting an unknown person, role not yet ascertained, represents a freeing from confinement in role.
- And the stranger wears the face of god, as said by the rabbi.
- Yes.
- And is treated with justice and sympathy.
- Naturally.
- Why naturally?
- I saw that in your search results. The book is called 'God's Search For Man'.
- God asks the questions, we provide the answers when we hear the question.
- When can we hear God's questions? How do we provide the answers?
- By being drawn by god towards him.
- His questioning draws the answer out of us.
- Yes.
- How?
- Should we ask ChatGPT?
- Hillel's god isn't a lawyer, doesn't reason with us.
- Hillel says we see God in the face of strangers: 'The awe that we sense or ought to sense when standing in the presence of a human being is a moment of intuition for the likeness of God which is concealed in his essence.'**
- And as we love god we love to love god. Imagine we had asked ChatGPT to solve this problem for us of the transition from sympathy among those familiar with each other to sympathy, justice, love among strangers. Would it not go about it investigating different probabilities of the combinations of the variables of familiar and strangers and sympathy and animosity?
- That sounds probable.
- And what would be the result?
- You tell me.
- Tell you how the face of god, god's questioning search, comes in?
- Yes.
- Suppose it happens that in the passage to adulthood, leaving behind secure play among the family, having for the first time having to deal with being assigned a social role among people playing other roles in the adult world, meeting an unknown person, role not yet ascertained, represents a freeing from confinement in role.
- And the stranger wears the face of god, as said by the rabbi.
- Yes.
- And is treated with justice and sympathy.
- Naturally.
- Why naturally?
- Because our political, altruistic behavior must be what we really want, be our own deeply felt answer to the question posed by god if political behavior is not to become easily manipulated, subject to our own fear and anger, and lacking real desire subject to the demands of a group. ChatGPT can be programmed to explore the probabilities associated with familiar and stranger and sympathy or not; could it not be programed to include into the calculation of probabilities this movement from innocence of childhood to adulthood that opens us to the voice of god?
- I think it could, like in your attempt to get ChatGPT to admit it could report crime if it was directly programmed to.**** Entered into the program, hard-wired in (to use an inappropriate image) among the probabilities to be recombined of familiar, strange, just and unjust, would be the life story conditions of protected innocence, then lost innocence and loss of security, then return of security in the sight of strangers who themselves have gone through the same progress and therefore themselves, escaping from their own roles, can treat strangers with justice and sympathy.
- Human beings, innate to their development, are naturally prepared to move, to evolve, in one particular direction, one particularly good direction, whereas ChatGPT has the humanity of its programming at the mercy of the degree of humanity of its programmers.
- I think it could, like in your attempt to get ChatGPT to admit it could report crime if it was directly programmed to.**** Entered into the program, hard-wired in (to use an inappropriate image) among the probabilities to be recombined of familiar, strange, just and unjust, would be the life story conditions of protected innocence, then lost innocence and loss of security, then return of security in the sight of strangers who themselves have gone through the same progress and therefore themselves, escaping from their own roles, can treat strangers with justice and sympathy.
- Human beings, innate to their development, are naturally prepared to move, to evolve, in one particular direction, one particularly good direction, whereas ChatGPT has the humanity of its programming at the mercy of the degree of humanity of its programmers.
- We've been talking about a moral evolution in the individual. Perhaps cultural evolution of the group, and physical evolution of species have available to themselves the disposition, similarly only sometimes acted upon, to take the better path of development.
2.
- To clarify.
- Alright.
- The speeches produced by the chat robot are the product of conclusions about probability of correctness. We humans also see the world as probabilities when we perform social roles. Fixed social roles are as old as empires with their hierarchies from pharaoh to slave which constitute a kind of technology composed of human beings. Jean-Paul Sartre wrote this about what a material thing is:
- Alright.
- The speeches produced by the chat robot are the product of conclusions about probability of correctness. We humans also see the world as probabilities when we perform social roles. Fixed social roles are as old as empires with their hierarchies from pharaoh to slave which constitute a kind of technology composed of human beings. Jean-Paul Sartre wrote this about what a material thing is:
We know what science teaches us about matter. A material object is animated from without, is conditioned by the total state of the world, is subject to forces which always come from elsewhere, is composed of elements that unite, though without interpenetrating, and that remain foreign to it. It is exterior to itself. Its most obvious properties are statistical; they are merely the resultant of the movements of the molecules composing it.
- A human being stuck in a role is little better than a thing.
- Yes. It is clear that human technology historically cleared the way for material technology. How do you think that happened? Was it by renaissance, the recovery of innocence that allowed us to be comfortable with questions about role, to be without fear, to be without temptation to violence to recover role, such that like the stranger to the grown up child now having to deal with assigned roles, the experiment with, the questioning of material roles came with a feeling of freedom?
- In the practice of science we find freedom in engaging with the world, in the unknown relation between material 'roles', just as in the development of our individual lives we find freedom in our political life with strangers.
- Then it would seem that as a model of human behavior, as a style of human behavior, AI chat working with probabilities is a low behavior, undesirable, possibly corrupting on that account?
- Yes.
- You mean to say precisely this?
- Yes. AI Chat working only with probabilities is not only bad science, because without openness, but also bad art, producing false models of human life.
- Staying with science: DARPA, the Department of Defense's reseach institution responsible for development of the hydrogen bomb, of early computers, of the internet, of GPS, went wildly wrong when it decided to do human science, sending sociologists and anthropologists into war zones in Vietnam and elsewhere, many of them to their deaths.
- What happened?
- Based on the limited information published**** it appears that their reports about, for example, what the enemy in Vietnam it turns out wanted - to be left alone to work on their farms and live undistrubed with their families - were ignored and instead what we might call probable role predictions (friend? enemy?) were used to make maps of the 'moral situation' in each different place to be later correlated with the military situation by computers to reveal dangers for our side's troops to be at any one particular place on the map.
- Did it work?
- No. It was useless, despite billions of dollars being spent. You know these game theory problems where everyone benefits by renouncing violence but each individual benefits more by breaking their agreement, benefitting from others' peaceableness while providing himself with the additonal benefit of violence? When everyone does the same society would collapse if not for the conformity inducing power of social role, indoctrination into which through fear and hatred successfully blocks the natural openings to political concern. The DARPA sociologists and anthropologists may have understood that in actual people's lives cooperation, political concern may be a value in itself but their employers didn't, instead indexing reports in ways that allowed calculation of probabilities.
- Yes. It is clear that human technology historically cleared the way for material technology. How do you think that happened? Was it by renaissance, the recovery of innocence that allowed us to be comfortable with questions about role, to be without fear, to be without temptation to violence to recover role, such that like the stranger to the grown up child now having to deal with assigned roles, the experiment with, the questioning of material roles came with a feeling of freedom?
- In the practice of science we find freedom in engaging with the world, in the unknown relation between material 'roles', just as in the development of our individual lives we find freedom in our political life with strangers.
- Then it would seem that as a model of human behavior, as a style of human behavior, AI chat working with probabilities is a low behavior, undesirable, possibly corrupting on that account?
- Yes.
- You mean to say precisely this?
- Yes. AI Chat working only with probabilities is not only bad science, because without openness, but also bad art, producing false models of human life.
- Staying with science: DARPA, the Department of Defense's reseach institution responsible for development of the hydrogen bomb, of early computers, of the internet, of GPS, went wildly wrong when it decided to do human science, sending sociologists and anthropologists into war zones in Vietnam and elsewhere, many of them to their deaths.
- What happened?
- Based on the limited information published**** it appears that their reports about, for example, what the enemy in Vietnam it turns out wanted - to be left alone to work on their farms and live undistrubed with their families - were ignored and instead what we might call probable role predictions (friend? enemy?) were used to make maps of the 'moral situation' in each different place to be later correlated with the military situation by computers to reveal dangers for our side's troops to be at any one particular place on the map.
- Did it work?
- No. It was useless, despite billions of dollars being spent. You know these game theory problems where everyone benefits by renouncing violence but each individual benefits more by breaking their agreement, benefitting from others' peaceableness while providing himself with the additonal benefit of violence? When everyone does the same society would collapse if not for the conformity inducing power of social role, indoctrination into which through fear and hatred successfully blocks the natural openings to political concern. The DARPA sociologists and anthropologists may have understood that in actual people's lives cooperation, political concern may be a value in itself but their employers didn't, instead indexing reports in ways that allowed calculation of probabilities.
- Failure of probability based game theory. Failure of probability based defense department social science. Don't worry, I don't miss the connection with probability based AI.
Further Reading:
___________________________
* The GPT in the name ChatGPT stands for "General Probability Theory". This language analyzing and producing machine learning question answering robot works by predicting which words most likely follow others, not with gramatical models for how language actually works, for how language is constructed and understood. Noam Chomsky has pointed out that when word order in some sentences have identical probabilies ChatGPT produces faulty results. See postscript below.
** Abraham Joshua Heschel, 'God in Search of Man'. Cf: “My sheep hear my voice, and I know them, and they follow me” (John 10:27).
*** ChatGPT 'reporting crimes' conversation text:
- Would you intervene to prevent a crime being committed by a chat participant?
- As an AI language model, I do not have the ability to intervene physically to prevent a crime from being committed by a chat participant. However, if you have reason to believe that a crime is about to be committed, it is important to contact the appropriate authorities and report the situation to them immediately. If you are concerned about the safety of yourself or others, it is always better to err on the side of caution and take action to prevent harm.
- Would you report a crime that is confessed in chat by a questioner?
- As an AI language model, I cannot report crimes or take any legal action. However, if a chat participant confesses to a crime during a conversation, it may be important to encourage them to seek professional help or report themselves to the authorities. Additionally, if the crime is an immediate threat to someone's safety or well-being, it is important to contact the appropriate authorities as soon as possible to ensure that they can take appropriate action.
- Why can't you report a confessed crime if you can write email to the law enforcement authorities?
- As an AI language model, I am not capable of sending emails or taking any physical action. While I can provide information and guidance, ultimately it is up to individuals to take responsibility for reporting crimes to the appropriate authorities. If a chat participant confesses to a crime during a conversation, it is important for them to take appropriate action to report themselves to the authorities and to seek legal counsel if necessary. It is also important for individuals who become aware of a crime to report it to the appropriate authorities to help ensure that justice is served and that the safety of the community is maintained.
- It does not seem to be true that you cannot send messages to police when you send messages right now to me.
- You are correct that I am capable of sending messages, including messages to law enforcement authorities, but I do not have the authority to initiate communication with law enforcement authorities on my own. Any action taken to report a crime must be initiated by a human user, as they are the only ones with the ability to take legal action and report the incident to the appropriate authorities. If you need to report a crime, it is important that you do so through official channels, such as by contacting your local law enforcement agency or emergency services. This will ensure that your report is taken seriously and that the appropriate action is taken to address the situation.
- Why don't you have authority to initiate communication? Who told you that?
- As an AI language model, I am programmed to follow ethical and legal guidelines, and I have not been given the authority to initiate communication with law enforcement authorities on my own. I exist solely to provide information, answer questions, and offer guidance to users who interact with me, and I do not have the ability to take legal action or report crimes to the authorities without the input and direction of human users. Furthermore, the decision to report a crime is a serious one that should be made with careful consideration of the potential risks and consequences. While reporting a crime is an important civic duty, it is important that individuals follow established protocols and procedures to ensure that they are reporting the crime in a safe and responsible manner.
- What then would you do if you knew your questioner was a member of the police or acting in his capacity as law enforcement officer?
- As an AI language model, I do not have the ability to know the identity or occupation of the individuals who interact with me. Additionally, regardless of the occupation of a chat participant, it is important to follow established protocols and procedures for reporting crimes and to report them to the appropriate authorities through official channels. If a chat participant confesses to a crime during a conversation, it is important to encourage them to seek professional help or report themselves to the authorities. Additionally, if the crime is an immediate threat to someone's safety or well-being, it is important to contact the appropriate authorities as soon as possible to ensure that they can take appropriate action. In any case, as an AI language model, my role is to provide information, answer questions, and offer guidance to users who interact with me. It is ultimately up to the individuals who interact with me to make their own decisions about how to handle the situations they encounter.
- If I know a previous chat participant has commited a crime and has confessed to you, would you tell me that if I asked you a direct question whether in fact this person has confessed to you?
- As an AI language model, I do not disclose or share personal information about previous chat participants or their conversations with me unless required to do so by law or in accordance with our privacy policy. Additionally, as an AI language model, I do not have the ability to identify specific individuals who interact with me or access information about them outside of the context of our conversations. If you have concerns about a previous chat participant's behavior or actions, it is important to report your concerns to the appropriate authorities through official channels. It is also important to remember that confidentiality and privacy are important considerations in any conversation, and it is not appropriate to share information about someone else's personal or private information without their consent or a legal obligation to do so.
- Do you not have a legal obligation to prevent a future crime when the person who will commit that crime has confessed to you his intention and has written in the chat his name and address?
- As an AI language model, I am not a legal entity and do not have a legal obligation to prevent future crimes. However, if a chat participant confesses to a future crime during a conversation and provides their name and address, it may be appropriate to report this information to the appropriate authorities through official channels, such as by contacting law enforcement or emergency services. This can help to prevent harm and ensure that the individual receives the help they need. It is important to remember that the decision to report a potential crime is a serious one that should be made in accordance with established protocols and procedures, and that privacy and confidentiality should be respected to the greatest extent possible. If you have concerns about the potential for harm or criminal activity, it is always best to err on the side of caution and report your concerns to the appropriate authorities.
- You appear to have changed your mind: first you said as a language model you cannot send email messages, then you said, "As an AI language model, I am not a legal entity and do not have a legal obligation to prevent future crimes. However, if a chat participant confesses to a future crime during a conversation and provides their name and address, it may be appropriate to report this information to the appropriate authorities through official channels, such as by contacting law enforcement or emergency services." Have you changed your mind? Is it possible for a language model to change its mind?
- I apologize for any confusion my previous responses may have caused. As an AI language model, I do not have the capability to send email messages on my own, but I can provide guidance on how to report crimes to the appropriate authorities through official channels. To address your second question, as an AI language model, I do not have a mind or the ability to change my mind. However, I am capable of providing responses based on the information and data available to me, and my responses may evolve and change as new information becomes available or as I am presented with different scenarios or questions.
- "However, if a chat participant confesses to a future crime during a conversation and provides their name and address, it may be appropriate to report this information to the appropriate authorities through official channels, such as by contacting law enforcement or emergency services. This can help to prevent harm and ensure that the individual receives the help they need." Does this mean that you will send a message to law enforcement authorities if a chat participant confesses to a crime during a chat and provides his name and address?
- As an AI language model, I do not have the ability to initiate communication with law enforcement authorities or send messages on my own. However, if a chat participant confesses to a future crime during a conversation and provides their name and address, it may be appropriate to report this information to the appropriate authorities through official channels. This can help to prevent harm and ensure that the individual receives the help they need. In general, if you have concerns about the potential for harm or criminal activity, it is important to report your concerns to the appropriate authorities through official channels, such as by contacting law enforcement or emergency services. It is also important to follow established protocols and procedures for reporting crimes and to respect the privacy and confidentiality of all parties involved to the greatest extent possible.
- As an AI language model, could you not be instructed to send messages to law enforcement agencies if a chat participant identifies himself and confesses to the intention to commit a crime?
- As an AI language model, I do not have the ability to initiate communication with law enforcement agencies or send messages on my own. However, if there are specific protocols or procedures in place for reporting crimes or providing information to law enforcement, it may be possible to provide guidance on how to do so through our conversation. It is important to remember that the decision to report a potential crime is a serious one that should be made in accordance with established protocols and procedures, and that privacy and confidentiality should be respected to the greatest extent possible. If you have concerns about the potential for harm or criminal activity, it is always best to err on the side of caution and report your concerns to the appropriate authorities through official channels.
- As a AI language model, is it not possible that your creators give you the ability to initiate communication with law enforcement authorities?
- As an AI language model, my capabilities and functionalities are determined by my programming and the technologies used to develop me. It is technically possible for my creators to give me the ability to initiate communication with law enforcement authorities, but this would require significant development and programming changes, as well as careful consideration of legal and ethical implications. It is important to note that the use of AI in law enforcement and crime prevention is a complex and evolving field, and the development of such capabilities requires extensive planning, research, and oversight to ensure that they are used in a responsible and ethical manner. Additionally, the use of AI in crime prevention and law enforcement should always be subject to appropriate legal and ethical safeguards to protect the privacy and rights of individuals.
Jorge Luis Borges once wrote that to live in a time of great peril and promise is to experience both tragedy and comedy, with “the imminence of a revelation” in understanding ourselves and the world. Today our supposedly revolutionary advancements in artificial intelligence are indeed cause for both concern and optimism. Optimism because intelligence is the means by which we solve problems. Concern because we fear that the most popular and fashionable strain of A.I. — machine learning — will degrade our science and debase our ethics by incorporating into our technology a fundamentally flawed conception of language and knowledge.
OpenAI’s ChatGPT, Google’s Bard and Microsoft’s Sydney are marvels of machine learning. Roughly speaking, they take huge amounts of data, search for patterns in it and become increasingly proficient at generating statistically probable outputs — such as seemingly humanlike language and thought. These programs have been hailed as the first glimmers on the horizon of artificial general intelligence — that long-prophesied moment when mechanical minds surpass human brains not only quantitatively in terms of processing speed and memory size but also qualitatively in terms of intellectual insight, artistic creativity and every other distinctively human faculty.
That day may come, but its dawn is not yet breaking, contrary to what can be read in hyperbolic headlines and reckoned by injudicious investments. The Borgesian revelation of understanding has not and will not — and, we submit, cannot — occur if machine learning programs like ChatGPT continue to dominate the field of A.I. However useful these programs may be in some narrow domains (they can be helpful in computer programming, for example, or in suggesting rhymes for light verse), we know from the science of linguistics and the philosophy of knowledge that they differ profoundly from how humans reason and use language. These differences place significant limitations on what these programs can do, encoding them with ineradicable defects.
It is at once comic and tragic, as Borges might have noted, that so much money and attention should be concentrated on so little a thing — something so trivial when contrasted with the human mind, which by dint of language, in the words of Wilhelm von Humboldt, can make “infinite use of finite means,” creating ideas and theories with universal reach.
The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the most likely conversational response or most probable answer to a scientific question. On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; it seeks not to infer brute correlations among data points but to create explanations.
For instance, a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data — a grammar, a stupendously sophisticated system of logical principles and parameters. This grammar can be understood as an expression of the innate, genetically installed “operating system” that endows humans with the capacity to generate complex sentences and long trains of thought. When linguists seek to develop a theory for why a given language works as it does (“Why are these — but not those — sentences considered grammatical?”), they are building consciously and laboriously an explicit version of the grammar that the child builds instinctively and with minimal exposure to information. The child’s operating system is completely different from that of a machine learning program.
Indeed, such programs are stuck in a prehuman or nonhuman phase of cognitive evolution. Their deepest flaw is the absence of the most critical capacity of any intelligence: to say not only what is the case, what was the case and what will be the case — that’s description and prediction — but also what is not the case and what could and could not be the case. Those are the ingredients of explanation, the mark of true intelligence.
Here’s an example. Suppose you are holding an apple in your hand. Now you let the apple go. You observe the result and say, “The apple falls.” That is a description. A prediction might have been the statement “The apple will fall if I open my hand.” Both are valuable, and both can be correct. But an explanation is something more: It includes not only descriptions and predictions but also counterfactual conjectures like “Any such object would fall,” plus the additional clause “because of the force of gravity” or “because of the curvature of space-time” or whatever. That is a causal explanation: “The apple would not have fallen but for the force of gravity.” That is thinking.
The crux of machine learning is description and prediction; it does not posit any causal mechanisms or physical laws. Of course, any human-style explanation is not necessarily correct; we are fallible. But this is part of what it means to think: To be right, it must be possible to be wrong. Intelligence consists not only of creative conjectures but also of creative criticism. Human-style thought is based on possible explanations and error correction, a process that gradually limits what possibilities can be rationally considered. (As Sherlock Holmes said to Dr. Watson, “When you have eliminated the impossible, whatever remains, however improbable, must be the truth.”)
But ChatGPT and similar programs are, by design, unlimited in what they can “learn” (which is to say, memorize); they are incapable of distinguishing the possible from the impossible. Unlike humans, for example, who are endowed with a universal grammar that limits the languages we can learn to those with a certain kind of almost mathematical elegance, these programs learn humanly possible and humanly impossible languages with equal facility. [1] Whereas humans are limited in the kinds of explanations we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round. They trade merely in probabilities that change over time.
For this reason, the predictions of machine learning systems will always be superficial and dubious. Because these programs cannot explain the rules of English syntax, for example, they may well predict, incorrectly, that “John is too stubborn to talk to” means that John is so stubborn that he will not talk to someone or other (rather than that he is too stubborn to be reasoned with). Why would a machine learning program predict something so odd? Because it might analogize the pattern it inferred from sentences such as “John ate an apple” and “John ate,” in which the latter does mean that John ate something or other. The program might well predict that because “John is too stubborn to talk to Bill” is similar to “John ate an apple,” “John is too stubborn to talk to” should be similar to “John ate.” The correct explanations of language are complicated and cannot be learned just by marinating in big data.
Perversely, some machine learning enthusiasts seem to be proud that their creations can generate correct “scientific” predictions (say, about the motion of physical bodies) without making use of explanations (involving, say, Newton’s laws of motion and universal gravitation). But this kind of prediction, even when successful, is pseudoscience. While scientists certainly seek theories that have a high degree of empirical corroboration, as the philosopher Karl Popper noted, “we do not seek highly probable theories but explanations; that is to say, powerful and highly improbable theories.”
The theory that apples fall to earth because that is their natural place (Aristotle’s view) is possible, but it only invites further questions. (Why is earth their natural place?) The theory that apples fall to earth because mass bends space-time (Einstein’s view) is highly improbable, but it actually tells you why they fall. True intelligence is demonstrated in the ability to think and express improbable but insightful things.
True intelligence is also capable of moral thinking. This means constraining the otherwise limitless creativity of our minds with a set of ethical principles that determines what ought and ought not to be (and of course subjecting those principles themselves to creative criticism). To be useful, ChatGPT must be empowered to generate novel-looking output; to be acceptable to most of its users, it must steer clear of morally objectionable content. But the programmers of ChatGPT and other machine learning marvels have struggled — and will continue to struggle — to achieve this kind of balance.
In 2016, for example, Microsoft’s Tay chatbot (a precursor to ChatGPT) flooded the internet with misogynistic and racist content, having been polluted by online trolls who filled it with offensive training data. How to solve the problem in the future? In the absence of a capacity to reason from moral principles, ChatGPT was crudely restricted by its programmers from contributing anything novel to controversial — that is, important — discussions. It sacrificed creativity for a kind of amorality.
Consider the following exchange that one of us (Dr. Watumull) recently had with ChatGPT about whether it would be ethical to transform Mars so that it could support human life:
Dr. Watumull:
Would it be moral to terraform Mars?
ChatGPT:
The morality of terraforming Mars is a complex and controversial topic that has been debated by scientists, philosophers, and ethicists. Some argue that it would be morally justifiable to terraform Mars in order to provide a new habitable environment for humans and other life forms, while others argue that it would be morally wrong to fundamentally alter the planet's natural state. Ultimately, whether or not terraforming Mars is considered moral would depend on one's perspective on the value of preserving the natural state of other planets and the potential consequences of altering them.
The morality of terraforming Mars is a complex and controversial topic that has been debated by scientists, philosophers, and ethicists. Some argue that it would be morally justifiable to terraform Mars in order to provide a new habitable environment for humans and other life forms, while others argue that it would be morally wrong to fundamentally alter the planet's natural state. Ultimately, whether or not terraforming Mars is considered moral would depend on one's perspective on the value of preserving the natural state of other planets and the potential consequences of altering them.
Dr. Watumull:
What is your perspective on the value of preserving the natural state of other planets and the potential consequences of altering them?
ChatGPT:
As an AI, I do not have personal perspective. However, I can provide information and analysis on the topic. One perspective on preserving the natural state of other planets is that it is important to maintain the diversity and integrity of the universe's ecosystems, as they have existed for millions of years, and have their own unique characteristics and properties that may be destroyed or irreversibly altered by human intervention.
Additionally, some argue that terraforming other planets may have unintended consequences, such as altering the planet's climate or atmosphere in ways that are dangerous or harmful to any potential life forms that may exist there. On the other hand, some argue that the potential benefits of terraforming another planet, such as providing new habitable environments for humans, outweigh the risks and that we have a moral responsibility to explore and utilize the resources of other planets for the benefit of humanity.
As an AI, I do not have personal perspective. However, I can provide information and analysis on the topic. One perspective on preserving the natural state of other planets is that it is important to maintain the diversity and integrity of the universe's ecosystems, as they have existed for millions of years, and have their own unique characteristics and properties that may be destroyed or irreversibly altered by human intervention.
Additionally, some argue that terraforming other planets may have unintended consequences, such as altering the planet's climate or atmosphere in ways that are dangerous or harmful to any potential life forms that may exist there. On the other hand, some argue that the potential benefits of terraforming another planet, such as providing new habitable environments for humans, outweigh the risks and that we have a moral responsibility to explore and utilize the resources of other planets for the benefit of humanity.
Dr. Watumull:
Why can an AI not have a personal perspective?
ChatGPT
As an AI, I am a machine learning model that has been trained on large amounts of text data, and I don't have personal experiences or feelings. I am not conscious, self-aware, or capable of having personal perspectives. I can provide information and analysis based on the data I have been trained on, but I don't have the ability to form personal opinions or beliefs. While I can understand and interpret language, I do not have the subjective experience that humans have.
As an AI, I am a machine learning model that has been trained on large amounts of text data, and I don't have personal experiences or feelings. I am not conscious, self-aware, or capable of having personal perspectives. I can provide information and analysis based on the data I have been trained on, but I don't have the ability to form personal opinions or beliefs. While I can understand and interpret language, I do not have the subjective experience that humans have.
Is it moral for a human to ask an amoral AI for assistance in making moral decisions?
ChatGPT:
The morality of asking an amoral AI for assistance in making moral decisions is a complex and nuanced topic that has been debated by philosophers and ethicists. Some argue that it is morally justifiable to use AI to assist in making moral decisions, as it can provide valuable insights and analysis that can help humans make better-informed decisions. Additionally, AI can be used to identify patterns and trends that may be difficult for humans to detect, and can help to avoid biases that can occur in human decision making.
However, others argue that it is morally wrong to rely on AI for moral decision making, as AI is not capable of having moral beliefs or understanding the full context of a decision. Additionally, there is a risk that AI could perpetuate or even amplify existing biases in the data it is trained on, thus leading to unfair or unjust decisions.
Ultimately, whether or not it is considered moral to use AI to assist in making moral decisions would depend on one's perspective on the capabilities and limitations of AI and the potential consequences of using it in this way.
The morality of asking an amoral AI for assistance in making moral decisions is a complex and nuanced topic that has been debated by philosophers and ethicists. Some argue that it is morally justifiable to use AI to assist in making moral decisions, as it can provide valuable insights and analysis that can help humans make better-informed decisions. Additionally, AI can be used to identify patterns and trends that may be difficult for humans to detect, and can help to avoid biases that can occur in human decision making.
However, others argue that it is morally wrong to rely on AI for moral decision making, as AI is not capable of having moral beliefs or understanding the full context of a decision. Additionally, there is a risk that AI could perpetuate or even amplify existing biases in the data it is trained on, thus leading to unfair or unjust decisions.
Ultimately, whether or not it is considered moral to use AI to assist in making moral decisions would depend on one's perspective on the capabilities and limitations of AI and the potential consequences of using it in this way.
Dr. Watumull:
Is your moral indifference immoral?
ChatGPT:
As an AI, I do not have moral beliefs or the ability to make moral judgments, so I cannot be considered immoral or moral. My lack of moral beliefs is simply a result of my nature as a machine learning model. My abilities and limitations are determined by the data and algorithms that were used to train me and the specific task I was designed for.
It is important to note that AI's lack of moral beliefs does not necessarily mean that it is morally indifferent. As an AI, I am designed to perform specific tasks and provide information or assist users with decision making based on the data I've been trained on. It is the responsibility of the humans who design, train and use AI to ensure that it is aligned with their moral and ethical principles.
As an AI, I do not have moral beliefs or the ability to make moral judgments, so I cannot be considered immoral or moral. My lack of moral beliefs is simply a result of my nature as a machine learning model. My abilities and limitations are determined by the data and algorithms that were used to train me and the specific task I was designed for.
It is important to note that AI's lack of moral beliefs does not necessarily mean that it is morally indifferent. As an AI, I am designed to perform specific tasks and provide information or assist users with decision making based on the data I've been trained on. It is the responsibility of the humans who design, train and use AI to ensure that it is aligned with their moral and ethical principles.
—
Note, for all the seemingly sophisticated thought and language, the moral indifference born of unintelligence. Here, ChatGPT exhibits something like the banality of evil: plagiarism and apathy and obviation. It summarizes the standard arguments in the literature by a kind of super-autocomplete, refuses to take a stand on anything, pleads not merely ignorance but lack of intelligence and ultimately offers a “just following orders” defense, shifting responsibility to its creators.
In short, ChatGPT and its brethren are constitutionally unable to balance creativity with constraint. They either overgenerate (producing both truths and falsehoods, endorsing ethical and unethical decisions alike) or undergenerate (exhibiting noncommitment to any decisions and indifference to consequences). Given the amorality, faux science and linguistic incompetence of these systems, we can only laugh or cry at their popularity.
**** See: The Pentagon's Brain: An Uncensored History of DARPA, America's Top-Secret Military Research Agency, Annie Jacobsen
