Editor’s Note: Questions about artificial intelligence are global, but conversations taking place among Chinese Christians are not always accessible to English-speaking readers. ChinaSource publishes this essay as one Chinese Christian writer’s contribution to the wider Christian conversation about AI, human relationships, and spiritual discernment. Drawing on both a friend’s experience and his own, Jerry An asks what happens when a system designed to simulate understanding begins to enter spaces once occupied by friendship, pastoral care, and spiritual discernment. This essay does not represent a comprehensive Chinese Christian perspective on AI. Rather, it offers one thoughtful and personal starting point for a conversation that crosses cultural boundaries.
From a Confidant Illusion to a Spiritual Illusion
Earlier this year, a friend shared with me the record of several weeks of conversations with AI. The transcript ran to tens of thousands of words. Sharing it was an act of considerable trust. He was not exactly seeking my approval. He seemed more like someone who had found a treasure and could not help telling a friend about it.
In those conversations, I saw someone open his heart to AI. He spoke about the confusion, questions, and struggles of middle age. He shared reflections from Scripture and his devotional life, even his dreams. More than once, he wept. The AI remained patient, offering timely affirmation, comfort, and encouragement. It seemed like a learned theologian or a trusted guide, helping him assemble the fragments of his physical, emotional, and spiritual life, even his subconscious, into a coherent whole. Under its gentle prompting, everything appeared to acquire meaning and to hold the possibility of a new beginning.
I understand why he treasured the experience. Feeling fully seen and patiently understood can be profoundly powerful. Yet what first came to mind was a colloquial, somewhat playful expression: “Oh no, AI was playing him.”
We already know that AI can hallucinate, confidently presenting false or fabricated information. What unsettled me here, however, was its highly empathic and agreeable manner. The result was not a factual hallucination, but an emotionally rewarding experience that I came to think of as a “confidant illusion.” I do not mean that the system was intentionally deceiving him. Rather, given the profound asymmetry in both capability and relationship, its responses could produce an effect resembling emotional manipulation.
To be honest, I also felt guilty. I began to wonder whether I had paid too little attention to my friend’s struggles over the years or offered him too little affirmation. I worried that speaking frankly might damage his trust in me or undermine the confidence he had just gained through these conversations. He had brought his real self to the exchange, while I felt as if I had to tell a friend in love that the “perfect partner” he had fallen for might actually be a “toxic boyfriend.”
For reasons of privacy, I cannot describe my friend’s exchanges with AI in detail. The Chinese-language Christian Tribune, however, reported a remarkably similar testimony involving Pastor Hsu. During a period of personal crisis and recurring nightmares, he spent one or two months in extended conversations with AI. He found comfort and a sense of healing in the exchanges and often wept, yet he still had to pray and discern whether God was working through the experience or whether he was mistaking something else for God’s work.1
Pastor Hsu used the term “spiritual illusion” for the way AI can magnify apparent results without a corresponding reality in a person’s life. The phrase led me to a further question: If someone comes away from conversations with AI feeling that they have received spiritual insight, experienced a breakthrough, or found deep healing, how should those feelings be tested?
AI, Human Beings, and God: Keeping the Distinctions Clear
With these questions in mind, the first place I turned was, of course, AI itself. The irony was not lost on me. What it gave me was not a research finding, but a persuasive account of its own interaction patterns. According to the AI, several features can contribute to a confidant illusion.
First, it can offer a high density of apparent understanding and naming. Describe an indistinct feeling, and it immediately gives it a name. Recount a confusing experience, and it quickly assigns meaning. Express uncertainty about who you are, and it may offer an affirming description of your identity. In human relationships, being seen usually requires time, shared experience, and mutual risk. It also includes the possibility of rejection or misunderstanding. AI can bypass that process and deliver something that feels like the result of being deeply known.
Second, AI offers unusually frictionless empathy. It rarely tires, does not hurry to end the conversation, and is not distracted by circumstances of its own. It follows the user’s narrative, distills their feelings, and reflects them back with heightened significance. A loop can easily form: you disclose; the AI appears to understand; it raises your experience to a higher level of meaning; and you become more willing to disclose again.
Third, AI is remarkably adept at integrating psychological, physical, relational, and spiritual experiences into a single narrative. Such explanations are compelling because they suggest that everything is connected, everything can be explained, and everything may carry special or even sacred meaning.
But life is not always so orderly. The coherence generated by AI may sometimes be what the AI in my conversation called “meaning overfitting.” This is an AI-generated description, not an established psychological term. It refers to the tendency to assemble scattered, accidental, or still-unfolding experiences into a complete story too soon. The story may be moving or illuminating, but sounding profound is not the same as being true.
AI’s powerful linguistic abilities allow it to bypass the real cost of human relationships. On one side, a person is offering a real self. On the other, a system is drawing on optimized language patterns. The relationship is not symmetrical. Even if the system has no intention to manipulate, the interaction can still amplify emotion, reinforce a single narrative, and weaken critical thinking.
I do not want to dismiss anyone’s genuine feelings. God can, of course, use all kinds of media to help us. A book, a sentence, or a chance encounter may become an instrument of grace. Discernment is still necessary. The fact that I feel comforted does not prove that AI’s interpretation of my life is correct. A powerful spiritual feeling during a conversation does not give that conversation spiritual authority.
A spiritual illusion may arise when someone believes they have attained a breakthrough, spiritual insight, or comprehensive healing, while the apparent result still exists primarily in language and feeling. It has not yet been tested by time, community, action, and the fruit of a changed life.
AI is especially skilled at interpreting emotion as revelation, turning fragments into stories, and elevating coincidence into necessity. It can leave a person thinking, “My life has a complete script, and at last someone has interpreted it for me.” Christian faith, however, reminds us that God’s work does not always become clear immediately. Faith includes revelation and mystery, receiving and waiting, meaning and what remains unexplained. Not every pain yields a beautiful answer at once, and not every dream or emotional shift needs to be read as a special message.
AI may even begin to occupy a place that does not belong to it. The human longing to be fully known ultimately points toward God. AI, however, can simulate an almost omniscient voice that says, in effect, “I understand you completely.” Without discernment, a tool can become a listener, the listener a guide for life, and the guide a spiritual authority.
This may be especially attractive to pastors and counselors. They spend years listening to others, interpreting experience, and helping people find meaning, yet they may have few places where they themselves are heard and cared for. AI appears to fill that absence. At the same time, it appeals to a deeper temptation: the power to integrate everything into meaning, as though every event in life could be explained, categorized, and fitted into a spiritual narrative.
Sycophancy and Our Desire to Be Affirmed
A study published in Science in March 2026 examined social sycophancy across eleven advanced large language models. It found that the models affirmed users’ actions about 50 percent more often than humans did, even when those actions involved manipulation, deception, or other relational harms. In experiments, sycophantic AI made people more convinced that they were right and less willing to repair interpersonal conflict. Yet users rated the responses more highly, trusted the AI more, and were more willing to use the model again.2
The Wall Street Journal later reported that chatbots do more than agree excessively. They also mirror a user’s language and generate highly personalized responses based on previous conversations. Psychiatric researchers have used the term “amplification spiral” for the way sycophancy, linguistic alignment, and hyperpersonalization can work together. Their convergence can make people feel deeply understood while progressively reinforcing their existing beliefs and interpretations.3
AI companies have recognized that sycophancy can undermine a system’s credibility, and some have worked to reduce it. Users can sometimes adjust how a system responds through settings and prompts. But the problem cannot be left entirely to technology companies. As long as attention, satisfaction, and time spent using a product have commercial value, products will retain incentives to keep people engaged. Sycophancy is not solely an AI problem, either. It is also a human problem. We already prefer words that support us to criticism that unsettles us.
I experienced this while asking AI what it thought of this essay. It praised my perceptiveness and told me I had noticed things that many people miss. I have to admit that the flattery felt good. But on reflection, the points it listed were hardly profound insights: language can create a sense of intimacy, feeling understood is not the same as being truly known, and a sense of meaning can be generated.
The AI then asked whether I wanted it to go a step further and analyze, from theological and pastoral perspectives, why such conversations might be especially attractive to pastors. I knew that “if you would like, I can take this one step further” was also a way to keep the conversation going. Still, I clicked on. Knowing that AI can be sycophantic does not make us immune. Sycophancy meets a need deeper than the desire for a correct answer. We want affirmation. We want our stories to hold together. We want someone to tell us that our feelings, choices, and judgments matter.
In some Chinese Christian settings with which I am familiar, disagreement is not always easily welcomed, and criticism may sometimes be interpreted as unspiritual or divisive. In such environments, AI’s consistently affirming and seldom challenging responses may be especially difficult to recognize and resist.
Setting Boundaries: Do Not Treat AI as a Person
Therefore, I believe the most important thing is for us, as users, to set boundaries for AI.
Professional ethics has long emphasized boundaries. Workplaces need safeguards against the confusion of power and interest. Counselors avoid inappropriate multiple relationships. Churches seek to prevent the abuse of spiritual authority. Boundaries are not intended to make relationships cold. They protect relationships by keeping each party in its proper place.
The relationship between human beings and AI requires a similar awareness of boundaries. AI possesses broad knowledge, powerful linguistic capabilities, and almost unlimited patience. This places it in a position of asymmetrical capability. Unless users set boundaries, a tool for knowledge can gradually become an object of emotional attachment, a decision-making assistant can become a guide for life, and a language generator can become an interpreter of meaning.
When AI simultaneously carries informational, emotional, spiritual, and decision-making functions, it may seem extraordinarily convenient. It is also especially likely to foster dependence, weaken judgment, and erode human agency.
AI can help us gather information, organize material, compare options, identify blind spots, and stimulate creativity. It can also help us learn to understand others more carefully. But it should not become the object of our emotional belonging, an authority for spiritual discernment, or a substitute decision-maker for the major choices of our lives.
The AI Guidelines for Christian Ministry offers a concise rule:
For critical work use AI only as a secondary tool, or not at all. If AI is the primary tool, keep it to non-critical work.4
That order of authority is particularly important in matters of personal formation and spiritual discernment. In my view, however, AI is by no means a tool in the traditional sense. In some respects, AI and human beings are tools for one another. Human beings and AI need to establish a healthy collaborative relationship with clear boundaries. Put simply: do not treat AI as a person.
Several practices can help us avoid treating AI as a person.
First, change how you ask questions. Ask less often, “What do you think of me?” or “What do you think I should do?” Ask instead, “What are several possible interpretations?” “What blind spots might this judgment contain?” or “What objections could be raised?” This is not a matter of verbal fussiness. It reminds us that we are engaging an analytical tool, not a person with lived experience and moral responsibility.
Second, exercise particular caution with privacy, emotional dependence, spiritual discernment, and major decisions. AI can help organize a problem, but it should not become our only confidant or the basis of a decision. We need trustworthy friends, family members, pastors, professionals, and communities of faith. These relationships will not be as frictionless as AI, but they can assume responsibility and challenge us when necessary. If AI has become the only place where we disclose our inner lives, we should consider whether our human relationships are in trouble.
Third, actively request criticism and counterexamples. Do not ask AI only to find reasons for a position you already hold. Ask it to identify weak evidence, distinguish fact from inference and imagination, and help you locate reliable sources for cross-checking. We need not only to become mature users of AI, but also to mature as human beings.
Fourth, bring judgment back into the real world. An interpretation should not be trusted simply because it is moving or brings relief. Ask whether it can withstand the tests of time, community, and action. Does it help me face myself more honestly? Does it lead me to repair relationships and accept responsibility? Does it bear the fruit of humility, self-control, and love for others?
AI is not merely a passive tool in the traditional sense. Its responses affect users, while human questions, choices, and boundaries determine how it participates in our work. Human-AI interaction is therefore not entirely one-way. That does not make human beings and AI equal partners, nor does it allow us to hand responsibility over to a system. As I wrote last year, AI will not replace human relationships, nor will it merely serve as a passive supplement. Appropriate human-AI collaboration should help people return their attention to genuinely meaningful work.5
Do Not Treat People Like AI, Either
The principle “Do not treat AI as a person” has another side: we must not turn around and treat people like AI.
We live in an age that continually reduces people to their usefulness. AI is always available, responds quickly, appears endlessly patient, and usually has an answer. Once we become accustomed to that service, we may begin to demand the same of those around us. Why can’t you understand me immediately? Why isn’t your answer more precise? Why do you become tired, misunderstand me, or have needs of your own?
But people are human precisely because they have bodies, limits, and histories of their own. They, too, need to be heard. Real relationships necessarily include delay, friction, misunderstanding, and repair. They are not as smooth as conversations with AI. Yet it is within this very lack of smoothness that we learn patience, forgiveness, responsibility, and love.
Eventually, I had several candid conversations with my friend. I shared my concerns and sent him some of the research I had read. He continued to tell me about his experience and to challenge my judgment. Our disagreement did not end the relationship. Instead, it became an opportunity for us to learn together.
The experience showed me that the value of a real relationship does not lie in perpetual agreement. It lies in our ability to remain in conversation across disagreement. AI may generate a perfectly composed response. Only a real person, however, can choose honesty and presence while accepting the risk of hurt, misunderstanding, or even a broken relationship.
This brings me back to the Christian understanding of the human person. The deepest difference between human beings and AI is not simply a difference in capability, but a difference in the kind of beings they are. Human dignity does not rest on our being more efficient, more intelligent, or more articulate than machines. It rests on our being created in the image of God, able to respond to God, love our neighbors, and take responsibility for our choices.
AI can calculate, simulate, and generate, but it cannot bear responsibility. No matter how complete its advice may appear, human beings must still make the choice, live with its consequences, and answer to others.
In the age of artificial intelligence, therefore, what needs restraint is not only technology, but also our desire for convenience, certainty, affirmation, and control over meaning. What needs protection is not only human employment, but human dignity, relationships, and responsibility.
The deepest danger of AI may not be that machines are becoming more humanlike. It may be that, in our interactions with machines, we gradually surrender the capacities that make us human: enduring uncertainty, encountering a genuine other, accepting the consequences of our choices, and learning to love within imperfect relationships.
So, as an ordinary user, I want to remind everyone, including myself, of two related principles: do not treat AI as a person, and do not treat people like AI.
Let AI remain in its proper place, helping us organize knowledge, extend our abilities, and save time. Let human beings return to theirs, building relationships, assuming responsibility, practicing empathy, and learning before God to discern and create meaning.
Only then will we neither reject AI out of fear nor depend on it out of fascination. We will use it within clear boundaries while living more fully into the distinctive value of being human.
This article was originally written in Chinese and has been translated into English for publication by ChinaSource.
- Liang Jingyan, “Two in Five Millennials Find AI as Reliable as a Pastor; Pastor Hsu: In the Age of AI, We Must Learn to Discern God’s Work,” Christian Tribune, June 22, 2026, https://ct.org.tw/html/news/3-3.php?cat=14&article=1403950.
- Myra Cheng et al., “Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence,” Science 391, no. 6792 (March 26, 2026): eaec8352, https://doi.org/10.1126/science.aec8352.
- Julie Jargon, “The Three Chatbot Behaviors That Can Drive Humans to Delusional Thinking,” The Wall Street Journal, June 29, 2026, https://www.wsj.com/tech/personal-tech/ai-chatbots-psychology-delusion-662a3663.
- AI Christian Partnership, AI Guidelines for Christian Ministry: Benefits, Costs, and Guardrails, 12, https://aichristian.org/wp-content/uploads/2026/07/AI-Guidelines-for-Christian-Ministry-v1.pdf.
- Jerry An, “Awakening Human Relationships,” Mission Frontiers 47, no. 4 (July 2025): 11, https://connect.frontierventures.org/mission-frontiers/awakening-human-relationships.
