Share the following information with your students, which can also be found in the Slides for this mini-lesson:
Generative AI models are able to create texts that convincingly mimic human-generated texts. The public does not have access to the full list of sources that generative AI models such as ChatGPT are trained on. These models draw on their vast amounts of training data to create texts, so their responses often cannot be traced to individual sources. When asked, ChatGPT will share citations for its responses, but these citations may not be the actual sources of the information it shared, and may even be made up.
Discuss the following questions with your students:
- Why does it matter what data is used to train generative AI models?
- Imagine if a generative AI model were only trained on data from one website, for example Reddit, and designed to mimic the type of content on that website. How do you think that would change the types of responses it gives?
Then, ask your students to read and evaluate two sample texts. One of the following passages was created by ChatGPT and mimics the style of an academic article. The other is an excerpt from a real, published article. Share the two passages–which can also be found in the Slides for the mini-lesson–with your students, and then ask them to vote on which one they think comes from a published article.
Passage 1:
Aged care facilities (ACFs) are residential communities with a concentration of vulnerable individuals with increased risk of severe influenza infection and complications such as outbreaks, hospitalisations and deaths. Aged care workers (ACW) are potential sources of influenza introduction and transmission in ACFs. Little is known about vaccine uptake among ACW. This study aimed to measure the vaccine uptake rate among Australian ACW and evaluate the demographic determinants of uptake during the influenza season of 2018.
146 ACWs were recruited from 7 facilities of a multisite aged care provider in Sydney. ACWs completed a questionnaire regarding their demographic, occupational and vaccination status. Vaccine coverage was calculated and variables were examined against their 2018 influenza vaccination status in statistical analysis.
Passage 2:
The objective of this study was to investigate the spread of influenza in elderly populations in Australia. A retrospective cohort design was employed, using data collected from electronic medical records of patients aged 65 years and over who had been diagnosed with influenza during the period of January 2016 to December 2018. The study population comprised a total of 14,527 patients from 12 hospitals across three states of Australia. Data on demographic characteristics, vaccination status, comorbidities, clinical presentation, and outcomes were collected and analyzed using descriptive and inferential statistics.
The results of the study indicated that the incidence of influenza was highest among elderly individuals who had not received the influenza vaccine. Additionally, patients with underlying chronic medical conditions were found to have a higher risk of severe illness and hospitalization. Spatial analysis showed that there were regional differences in the incidence of influenza, with higher rates observed in certain areas.
Share the answer with your students: Passage 1 is from an academic article “Influenza vaccine coverage and predictors of vaccination among aged care workers in Sydney Australia,” which was published in the journal Vaccine. Passage 2 was generated by ChatGPT and references a fictional study.
Then, share with your students the following three steps they can use to check information they come across online. These steps are useful for detecting misinformation created by generative AI but are also helpful for checking human-generated content.
- Research the organization that published the content.
- Verify key information in the text.
- Check the citations included in the text.
Either walk through the following three steps as a class over a projector or place students in small groups of 3-4 and ask them to complete the following steps on a computer. Ask students to discuss the reflection questions following each step as a class or in small groups.
- Passage 1 is excerpted from “Influenza vaccine coverage and predictors of vaccination among aged care workers in Sydney Australia” and is hosted on the website Elsevier. Do a quick search for Elsevier and read information you find about the organization.
Reflect: What aspects of the description of this organization make it seem trustworthy?
- Search for a key piece of information from passage 2, such as: “The study population comprised a total of 14,527 patients from 12 hospitals across three states of Australia.”
Reflect: What does it indicate about this information that there are no direct matches?
- ChatGPT provided the following reference for the text it generated in passage 2: Sullivan SG, et al. Influenza vaccination coverage among residents of aged care facilities in Australia, 2018: a national cohort study. BMJ Open. 2020;10(1):e033180. doi:10.1136/bmjopen-2019-033180. Search for this article online.
Reflect: What about this citation seems convincing? What did you learn when you searched for the article online?
Finally, ask your students to discuss the following questions using the Think, Pair, Share strategy:
- Does the information you learned in this activity change the way you think about content you see online or on social media? Why or why not?
- What would it look like if you used one or more of the steps in this activity to evaluate content you see online or on social media?