Deepfake Deception
Story by Jayci Eyre
You may have seen a deepfake without ever realizing it. Or you might have gotten a message from your dad saying, “have you seen this?!?” with a clearly questionable video of “Mark Zuckerberg” pretending to be an alien. Maybe you watched the Barack Obama educational deepfake video a few years ago. Or you could be asking what a “deepfake” even is.
Jeremy Mumford, a Computer Science (data emphasis) senior from Bountiful, Utah, has been asking himself questions too. When Jeremy was looking for a thesis topic, deepfake recognition jumped out as an “intersection of several of my interests,” he said, including both data science and cinematic visual effects.
Deepfakes are videos in which the face or body of the person being filmed is digitally altered. Luke Skywalker in the Mandalorian and Harrison Ford in the new Indiana Jones movie (he’s “yoked,” Jeremy said) are examples of recent fun deepfakes. Consider the image at the top of this page, and how it was easily altered:
However, there’s a dark side to deepfakes: they can be used to deceive and manipulate. For example, Jeremy said that “when the war in Ukraine started, there was a fake video of President Zelensky saying that Ukraine had surrendered.”
When Jeremy ran a literature review on deepfake recognition, he found research gaps in deepfake educational interventions. He also noticed that there was not much demographic information available. According to Jeremy, a person’s age plays a big role: “I strongly believe that the influential power of deepfakes varies greatly by generation.”
With these missing puzzle pieces, Jeremy put together a plan. His thesis proposal was to survey three groups with varying participant ages: a control group, a text intervention group, and a video intervention group. Both the text and video would educate participants on deepfake identification. Then everyone would analyze ten videos to see if they could spot the four deepfakes.
The nice thing, Jeremy said, is that “there are websites called Mechanical Turk websites, where you can pay participants for statistical studies.” The term “Mechanical Turk” has historically referred to something that looks like a machine, but actually has human laborers on the inside.
After getting his survey responses, Jeremy worked with the BYU Statistics department on running the numbers. He said that they offer two free consultation sessions to students who are working on thesis-type projects. Typically, these sessions are expensive, so this was “such a valuable tool.” Even as a Stats minor, Jeremy said, he got overwhelmed with such a huge project.
“I would highly recommend going once before conducting your study to help construct it, because that's 90% of the work,” Jeremy said. “Then go once afterwards and they can help you analyze it.”
Jeremy was in for a surprise with his results. “Older people definitely did struggle more,” Jeremy said—as expected. But the interventions differed in effectiveness based on the generation. Young generations responded well to both the educational video and text.
On the other hand, “The video intervention worked really well for older groups. [But] we found that giving a text intervention for the older group… seemed to confuse them even more.” Jeremy said that it was a surprise that “some of my interventions had a detrimental impact on people's abilities to recognize deepfakes.”
The thesis process itself also had a couple of twists and turns for Jeremy. “The timespan of doing your research project can take so long that you'll have to adapt to unexpected changes,” he said. “For example, the professor I worked with got called as a General Authority!” Afterward, Jeremy successfully completed his thesis under a new professor, Dr. Quinn Snell.
Jeremy credits the HONRS 320 Great Question Essay class for prepping him with the skills of inquiry he used for his thesis. His essay was, “Can You be Fooled by Statistics?" He also said that Honors has made “some great strides” when it comes to preparing students on how early they need to think about their thesis questions, but that it can continue to improve.
His advice for students feeling overwhelmed about a thesis topic is to choose a question they connect with. “Consider where your discipline crosses with your interests, and find something at that intersection that is interesting, relevant and important,” Jeremy said.
Jeremy is excited to be graduating in April and plans to continue to develop groundbreaking techniques in applying AI tools in software applications. He already has a job working as an AI software engineer at a tech company in Lehi called Pattern.
Although Jeremy’s time in the Honors Program is drawing to a close, he said that it has been “life-changing in terms of both the work I’ve done and the connections I’ve made.”