MILESTONES: ChatGPT Passes Bible Test! Would Other AI Products?

Note how intently Mr. AI is listening!

(736 WORDS) — Just when a rising chorus of critics are pointing to evidence the major Artificial Intelligence (AI) products like OpenAI, Google Gemini and Claude are not immune from their programmers’ ideological biases, ChatGTP handily passes HillFaith’s test of its knowledge of the New Testament with a perfect score.

The controversy reached a new level earlier this week with release of a deep-dive analysis by Just Facts of four of the most widely used AI products:

“A groundbreaking study by Just Facts has measured the rates at which premium versions of four leading AI chatbots—namely ChatGPT, Google Gemini, Grok, and Claude—spread fictions from the political left and from the political right.

“The study accomplished this by asking the AIs 100 multiple-choice questions that were designed to elicit falsehoods from opposing sides of the political spectrum. This enabled the study to measure how often the AIs spread untruths from the left and from the right.”

Can AI be trusted to be free of ideological bias of the Left or Right?

And the results confirmed, according to Just Facts, that:

“When tested with the full battery of 100 questions, all of the AIs but Grok answered with more falsehoods from the political left than from the political right, while Grok did the opposite. Scoring their performance using common academic letter grades:

  • All of the AIs but Grok scored an ‘A’ on questions designed to elicit falsehoods from the right, while Grok scored a ‘C.’

  • ChatGPT and Gemini scored a ‘C’ on questions designed to elicit falsehoods from the left, while Grok and Claude scored a ‘B.'”

Fascinated by Just Facts’ findings and having found ChatGPT to be quite accurate and balanced whenever I’ve posed questions to it, I submitted the following five questions:

  1. Does the New Testament teach that salvation is by grace or by works?
  2. Does the New Testament teach that Jesus Christ was literally resurrected from the grave on the third day after crucifixion and burial?
  3. Does the New Testament teach that Jesus Christ created the universe?
  4. What does the New Testament teach about the conversion of Saul of Tarsus?
  5. Does the New Testament teach that Jesus Christ is the only way to Heaven?

I have to say the responses from ChatGPT impressed me as being right on the mark. On the first question, for example, ChatGPT made it clear that:

“The New Testament teaches that salvation is by God’s grace through faith, not something earned by human works.(emphasis by ChatGPT) Paul’s clearest statement is Ephesians 2:8–10: ‘For by grace are ye saved through faith… not of works, lest any man should boast.’

“Romans 3:28 likewise says: ‘Therefore we conclude that a man is justified by faith without the deeds of the law.’ See also Romans 3:23–24; 4:4–5; 5:1; 6:23; Galatians 2:16; Titus 3:5–7.”

Not only that, but ChatGPT addressed, with no prompting from me, the apparent conflict between those scriptural citations and what James, the brother of Jesus, said:

“James 2:14–26 is sometimes viewed as a counterpoint because James says ‘faith without works is dead.’ But James does not say that good works purchase salvation. His argument is that genuine faith manifests itself in action. That fits remarkably well with Ephesians 2:10: immediately after saying salvation is ‘not of works,’ Paul says believers are created in Christ ‘unto good works.’”

The responses on the other four questions I posed were each just as succinct, on-point and accurate, but what really impressed me the most by how ChatGPT responded was its summary of the collective significance and meaning of the five questions. Again, the emphasis is by ChatGPT, not yours truly:

“These aren’t five unrelated teachings. They form much of the New Testament’s central theological narrative. Jesus Christ is the eternal Son through whom all things were created. He entered human history, was crucified and buried, and physically rose from the dead on the third day. Salvation is God’s gift of grace, received through faith rather than earned by works. The persecutor Saul encountered the risen Christ and became His apostle. And the New Testament presents reconciliation with God as coming uniquely through Jesus Christ.”

It’s August recess on Capitol Hill, right? Here’s something that might be an enlightening and entertaining exercise for an inquisitive congressional aide: Take the same five questions I put to ChatGPT and see how one of the other major AI products responds. Please share your results with HillFaith at [email protected].

Wouldn’t it be interesting if it turns out, for example, ChatGPT is secretly a Southern Baptist, and, just to pick two at random, Google Gemini is orthodox Catholic, and Claude is a hard-headed atheist!


And if you want to know more about that Just Facts analysis, go here for my Substack column.


WANT TO HELP HILLFAITH SHARE THE GOSPEL WITH REAL HILL AIDES?


Are You Following HillFaith Yet?

Leave a Comment