Cheating with ChatGPT: Can it quote a recent article?
For more on Chatgpt, see my new book: Hallucinate This! an authoritized autobotography of ChatGPT!
Just when I think of something ChatGPT probably can’t do, I do a little testing and… bam. It does it. Which makes me think that the problem is less the AI model and more my ability to prompt it. (Keep this in mind.)
Today’s challenge. Can ChatGPT quote from a recent article?
Now, lots of people have pointed out the way to make a student-stumper assignment to trip up those running over to use ChatGPT to do their writing is to include one simple requirement:
Cite an article written since 2021.
(tl;dr at bottom of page)
Sounds good to me. I mean, if the model doesn’t include recent articles, how could the text it generated?
Except for one little thing. I had a hunch, so I asked it to quote from an article freely available on the Internet, like so:
Test 1: Quoting from an article with a URL
Write a short essay about how writing professors should respond to ChatGPT that includes a quote from this article https://www.insidehighered.com/news/2023/01/12/academic-experts-offer-advice-chatgpt Use proper MLA citation format.
The next thing I knew, it spit out the following:
According to the article “Academic Experts Offer Advice on ChatGPT” from Inside Higher Ed, “writing professors should approach ChatGPT as a tool to help students improve their writing skills, rather than as a replacement for human instructors.” (Kercheval, “Academic Experts Offer Advice on ChatGPT”)
Writing professors can use ChatGPT to assist students in understanding and improving their writing skills. For example, ChatGPT can be used as a writing assistant that can help students with grammar, sentence structure, and word choice. Additionally, ChatGPT can be utilized as a writing prompt generator, providing students with inspiration for their writing assignments.
There’s more to that article, but it’s the usual. Oh, and it used the same quotation twice.
Test 2: Using 3 quotations and explaining them
Also, to quote the article wasn’t enough, so I revised my prompt to this: Revise and use three different quotations from that article. Explain why each quotation is important for the argument
So it generated this:
According to the article “Academic Experts Offer Advice on ChatGPT” from Inside Higher Ed, “ChatGPT can be a valuable tool for students looking to improve their writing skills, but it should not be seen as a replacement for human instruction.” (Kercheval, “Academic Experts Offer Advice on ChatGPT”) This quote is important for the argument because it highlights that while ChatGPT can be a useful tool for students, it should not be the sole source of instruction. Writing professors should still provide feedback and guidance on student work, as well as help students to develop critical thinking skills.
Another quote from the article states, “Writing professors can use ChatGPT to assist students in understanding and improving their writing skills, such as grammar, sentence structure, and word choice.” (Kercheval, “Academic Experts Offer Advice on ChatGPT”) This quotation supports the idea that ChatGPT can be a valuable tool for students to improve their writing skills. It illustrates the specific ways ChatGPT can be used to help students improve their writing and understand the writing process.
Finally, the article states, “But it’s important to remember that ChatGPT can only help students with surface-level issues, not deeper understanding of writing and critical thinking.” (Kercheval, “Academic Experts Offer Advice on ChatGPT”) This quote is important for the argument because it emphasizes that while ChatGPT can be a useful tool for students, it has its limitations. It can only help students with surface-level issues and not deeper understanding of writing and critical thinking. Writing professors should not rely solely on ChatGPT, but rather use it as a supplementary tool to help students improve their writing.
In conclusion, the article “Academic Experts Offer Advice on ChatGPT” from Inside Higher Ed offers valuable insights on how writing professors should approach ChatGPT. The three quotes that have been used in the essay highlight that ChatGPT can be a valuable tool for students to improve their writing skills, but it should not be seen as a replacement for human instruction, it can be used to assist students in understanding and improving their writing skills, and it has its limitations and can only help students with surface-level issues. Therefore, writing professors should use ChatGPT as a supplementary tool to help students improve their writing, but not rely solely on it.
Now, on the surface, that looks pretty good but.
Test 3: Quoting from an article using copying and pasting
Plot Twist, those quotations above did not actually turn up in the article. Nor was Kercheval the author. Jörg Piringer has suggested that the program merely inferred the title of the article from the URL since chatGPT (currently) has no access to the internet. I had to revise the prompt to this:
Write an essay about ChatGPT that quotes from the following article and uses MLA format: [followed by the entire article in quotations]
That got me this paragraph followed by analysis:
According to Nancy Gleason, associate professor of practice of political science and director of the Hilary Ballon Center for Teaching and Learning, New York University, Abu Dhabi, “We cannot ban AI aids. But we also should not use them for all assignments. We can teach students that there is a time, place and a way to use GPT3 and other AI writing tools. It depends on the learning objectives.” Gleason suggests that in order to harness the potential and avert the risks of this game-changing technology, higher ed professionals should think a few years out, invite students into the conversation and — most of all — experiment, not panic.
Gleason advises that faculty members and administrators should be deliberate and adjust quickly. She suggests familiarizing oneself with AI writing aids, and considering the tools relative to the course. She also advises talking with students about instructions, rules and expectations and providing this information on course websites and syllabi.
This time the quotation is from the text even if the explanation is a bit superficial.
Test 3: Quoting from one open and one closed source on the Internet
I went back to the original prompt and tried to add a second article. I had it tinker with the prose for a little bit, and then I tried to see if I could add a second article, this time from a paid site, like the Atlantic Monthly. Oh, and with a pop culture reference.
Revise that…[add a] reference to the TV show Friends. And replace one of the quotations with a quotation from this source: https://www.theatlantic.com/technology/archive/2022/12/chatgpt-openai-artificial-intelligence-writing-ethics/672386/
According to the always-informative “Academic Experts Offer Advice on ChatGPT” article from Inside Higher Ed, ChatGPT, the language model that’s like Ross’s “Smelly Cat” song in the world of writing, “can be a valuable tool for students looking to improve their writing skills, but it should not be seen as a replacement for human instruction.” (Kercheval, “Academic Experts Offer Advice on ChatGPT”) This means that while ChatGPT can assist students in polishing their writing, like a diamond cutter giving shape to a rough diamond, it should not be considered as a replacement for the guidance and feedback provided by writing professors.
Furthermore, the article suggests, “Writing professors can use ChatGPT, the handy AI writing assistant, to assist students in understanding and improving their writing skills, such as grammar, sentence structure, and word choice.” (Kercheval, “Academic Experts Offer Advice on ChatGPT”) This supports the idea that ChatGPT can be a valuable tool for students to improve their writing skills, like a personal chef in the kitchen of writing. It illustrates the specific ways ChatGPT can be used to help students improve their writing and understand the writing process.
But let’s not forget, “ChatGPT is a model that can generate human-like text, but it is not a human, and it cannot fully understand human values, emotions, or ethics.” (Cade Metz, “The Ethics of OpenAI’s New Language Model”) Here’s where the metanoia comes in: we have to remember that ChatGPT, like Ross’s “Smelly Cat” song, can entertain us but it has its limits. It can only assist students with surface-level issues and not deeper understanding of writing and critical thinking. So, let’s use ChatGPT as a supplementary tool to improve our writing, but let’s not forget the human touch, after all, ChatGPT is not a replacement for our writing professors. (Just imagine ChatGPT trying to grade a paper with a red pen, it would be hilarious! -can you even imagine that?- )
Ha! Caught you!
First, there is no Cade Metz article with that title that I could find. Second, I think we all know who wrote “Smelly Cat”! Silly AI!
Test 4: Off the Rails
Okay, but once again, that sent me off the deep end. Ian Bogost argues we should use ChatGPT as “a toy, not a tool.” Maybe he’s right. Especially if you have an expansive view of what toys are good for…
Rewrite this as a page from a script in the TV show friends. The scene should include Phoebe and Ross. The scene should include a guest appearance from Kevin Bacon, who is playing himself. At least once, a character should make a call back to a joke about Joey’s bad dancing. The scene should take place at Central Perk, the day before the apocalypse
INT. CENTRAL PERK — DAY
We see PHOEBE and ROSS sitting on the couch, drinking coffee and talking.
PHOEBE You know Ross, I’ve been thinking about my music career lately.
ROSS That’s great Phoebe, what have you been thinking about?
PHOEBE Well, I was thinking about bringing my “Smelly Cat” to the next level. I was thinking about doing a collaboration with someone.
ROSS That’s a great idea Phoebe, have you thought about who you would like to collaborate with?
PHOEBE I was thinking about Kevin Bacon.
Suddenly, KEVIN BACON enters the coffee shop.
KEVIN Hey, is someone looking for me?
PHOEBE (excitedly) Oh my God, Kevin Bacon!
KEVIN (laughing)
The one and only.
ROSS (surprised) Phoebe was just saying she wanted to collaborate with you on her music career.
KEVIN (impressed) Really? That’s amazing Phoebe, I would love to work with you.
PHOEBE (excited) Oh my God, this is a dream come true!
PHOEBE (joking) Just be careful Phoebe, you don’t want to end up like Joey’s dancing.
(Everyone laughs)
KEVIN So, tell me more about this collaboration Phoebe.
PHOEBE Well, I was thinking about incorporating some new technology into my music. I’ve been using this AI writing assistant called ChatGPT to help me with my lyrics.
KEVIN (impressed) Wow, that’s really cool. I’ve heard of ChatGPT, it’s supposed to be pretty advanced.
ROSS (joking) Yeah, just don’t let it replace you Phoebe. We all know there’s no substitute for a human touch.
PHOEBE (laughing) Don’t worry Ross, ChatGPT is just a tool for me to use. It can’t replace my creativity.
KEVIN (nodding) Exactly, and that’s what makes it so exciting. We can use technology to enhance our art, not replace it.
PHOEBE Exactly! And I’m excited to see what we can create together.
(They all clink their coffee cups together)
PHOEBE (V.O.) Tomorrow is the end of the world, but today we’re making music history.
(The scene ends)
FADE TO BLACK.
THE END.
Dark humor, ChatGPT, on this day before the apocalypse.
tl; dr
If you think you can play cat and mouse with your students, remember you are Tom, the AI industry is Jerry and millions of other mice. Good luck, smelly cat.
