Source Checking AI Output
As stated on the previous page, it is always a good habit to include statements such as the following at the end of your AI prompts:
Please provide links to peer-reviewed academic journal articles to support and verify your response.
Please provide links to peer-reviewed academic journal articles from the last five years to support and verify your response.
Please provide links to reliable technology sources to support and verify your response.
But providing these instructions in the prompt is still not enough! You still need to consult those sources as well and verify the information you find in them before you use that information for your research project or other purposes.
Here are two examples showing why checking sources given by AI is so important.
| Real Life Example 1: I asked a reputable AI chatbot (Gemini) for information about some of Campbellsville University's policies. Gemini gave me a very confident answer about how the policy worked, but when I asked for a link to verify the information, the link provided was for a different university with a similar name, Campbell University. Thus, the entire response was completely unreliable since the information came from the wrong source. |
|
Real Life Example 2: I asked Copilot (which uses ChatGPT) whether or not the Montgomery Library provides editing services for papers. Copilot answered very confidently that the answer is yes, when in fact I know the answer is no. Then I checked the source it gave. The source was a Montgomery Library webpage explaining that writing help can be requested at the Badgett Academic Support Center (BASC). However, if you know the layout of campus, you know that the Montgomery Library and the BASC are separate buildings. Thus, Copilot mistakenly assumed that the BASC is located somewhere inside the library, and thus provided misinformation in the output. Similar to us humans, AI can make mistaken assumptions when connecting pieces of information and analyzing websites. |
These two examples show why it is important to include a request for sources in your AI prompt and to actually analyze those sources yourself to determine if the information is accurate.
When dealing with technical and academic topics, it is strongly advisable to consult peer-reviewed research to verify the information found on the websites that AI provided as sources. You can even engineer your prompt so that the AI system uses peer-reviewed academic journal articles as sources, as shown in one of the sample prompts above. However, you still have to read the articles yourself just in case the AI system misinterpreted something in the academic article. If AI can make mistakes and faulty assumptions when analyzing a simple webpage, as shown in Real Life Example 2 above, AI summaries should not be trusted to interpret the details in a complex academic article. AI can help you find these sources and get a basic idea of what they are about, but as a responsible students and researcher, you have to do the verification yourself.