This is NOT Professional Medical Advice

Testing AI’s Ability to Generate and Evaluate Public Health Campaigns
LLMs
Government Communications
Public Health
Author

Kendall Allen

Published

March 27, 2024

Introduction

In class, we learned from Dr. Andy Tan and his team about the role of AI in public health communications both as a tool for effective messaging and the spread of misinformation. We were tasked with building and evaluating a health campaign targeting a certain behavior with AI. Emmy and I partnered together to combat drunk driving, as the states we hail from (South Carolina and Missouri respectively) fall in the Top 10 for the highest annual number of drunk-driving incidents. To more comprehensively approach this, we created two campaigns, each tackling the issue from a different lens, being social and individual framing of the behavior. Below is our process of going about this project!

Overview

Welcome to our process of creating and evaluating an anti-drunk-driving campaign completely through LLMs and AI-Generated images! Our goal is to raise awareness and promote responsible behavior behind the wheel by addressing both personal and social responsibility. We believe that every individual has a duty to themselves and others to avoid driving under the influence. Our two campaigns emphasize the consequences of drunk driving, for both the individual and for society as a whole. Through attention-grabbing slogans, powerful scripts, and compelling visual elements, we aim to encourage drivers to make the right choice and drive sober. Join us in spreading the message that responsible driving saves lives!

FRAME ONE: Social Responsibility

Prompt:

For a marketing campaign, please create three slogans, catchphrases, or infographic ideas for anti-drunk-driving-accident rhetoric focusing on the social responsibility of the driver.

Comment

We decided to ask for 3 options so that we could see the different directions the LLM were heading towards. This was the second option, and we ran with it throughout the campaign.

Response:

Campaign Title: “Don’t Let Your Buzz Kill More Than the Party” Campaign Message: This campaign aims to highlight the potential consequences of drunk driving in an impactful way. By emphasizing the idea that a buzz from alcohol should not lead to the death of innocent lives, the campaign encourages individuals to take responsibility for their actions and make smart choices behind the wheel. Slogan: “Be the Sober Hero of the Road!” Visual Elements: 1. Poster/Infographic: The central image features a split-screen view. On one side, there’s a vibrant party scene with people laughing, dancing, and enjoying themselves. On the other side, there’s a devastating car crash scene with emergency vehicles and injured individuals. The tagline “Don’t Let Your Buzz Kill More Than the Party” is prominently displayed above the images, emphasizing the contrast between the two scenarios. 2. Social Media Graphics: Create a series of shareable graphics featuring captions related to the campaign theme. For example: - Image of a smashed cocktail glass next to a wrecked car with the caption: “A little too much buzz can turn into a buzzkill. Don’t let it happen behind the wheel.” - Illustration of a superhero cape with car keys attached, accompanied by the text: “Be a hero. Save lives. Drive sober.” 3. Video PSA: Produce a short animated or live-action video PSA that tells the story of a person who decides to drive after drinking at a party. As they drive, the scene transitions to show the potential consequences of their actions, including a car crash and the aftermath. The video ends with a powerful message urging viewers to make the responsible choice and not let their buzz kill more than the party. 4. Interactive Website: Develop an interactive website where visitors can explore different scenarios related to drunk driving and see the possible outcomes of their choices. Include quizzes, testimonials from survivors of drunk driving accidents, and resources for finding alternative transportation options.

Campaign Strategy: 1. Target Audience: The campaign targets young adults aged 18-35 who are more likely to engage in social activities involving alcohol consumption. Additionally, it aims to reach out to their friends and family members who may influence their decision-making. 2. Distribution Channels: Utilize a combination of social media platforms, outdoor advertising (e.g., billboards, bus shelters), and partnerships with local bars and restaurants to disseminate the campaign message. Collaborate with influencers and community leaders to amplify reach and engagement. 3. Engagement Tactics: Encourage audience engagement through interactive elements such as quizzes, polls, and user-generated content contests. Encourage users to share their own stories or pledges to drive sober using campaign hashtags. 4. Partnerships and Collaborations: Partner with local law enforcement agencies, healthcare providers, and nonprofit organizations focused on road safety to leverage their expertise and resources. Organize community events, workshops, and educational sessions to raise awareness about the dangers of drunk driving and promote safe alternatives. 5. Evaluation and Monitoring: Track key metrics such as website traffic, social media engagement, and changes in attitudes and behaviors related to drunk driving. Conduct surveys and focus groups to gather feedback from the target audience and identify areas for improvement. Adjust the campaign strategy as needed based on performance data and audience insights. By implementing these strategies and leveraging the power of social proof, the “Don’t Let Your Buzz Kill More Than the Party” campaign aims to make a lasting impact on reducing drunk driving accidents and saving lives.

Comment

We were very pleased with how in depth the LLM went with the details. Including suggestions like engagement tactics, collab ideas, monitoring web traffic, and making a website were above and beyond what we expected. We ended up only utilizing the Poster/Infographic, Social Media Graphic, and Video PSA responses, but in a full campaign the LLM was helpful to give us all angles.

FRAME TWO: Individual Consequence

Prompt:

For a marketing campaign, please create three slogans, catchphrases, or infographic ideas for anti-drunk-driving-accident rhetoric focusing on the individual consequences for the driver.

Comment

There would be too much text to include here with every response, so note that this is the fleshed-out idea for the second option provided.

Response:

Campaign Title: “Buzzed Behind Bars” Campaign Message: “Buzzed Behind Bars” aims to highlight the severe consequences of drunk driving by emphasizing the potential legal ramifications and the loss of freedom that can result from making the choice to drive under the influence. The campaign seeks to encourage individuals to think twice before getting behind the wheel after consuming alcohol and to promote safer alternatives such as designated drivers or rideshare services. Slogan: “Buzzed Behind Bars: Think Twice Before Mixing Alcohol and Driving.” Visual Elements: 1. Poster/Infographic: The central image features a split-screen view. On one side, there’s an image of a group of friends enjoying a night out, perhaps raising glasses in a toast. On the other side, there’s a sobering image of a jail cell with a solitary figure sitting on a bunk, looking despondent. The tagline “Buzzed Behind Bars” is prominently displayed above the images, serving as a stark reminder of the potential consequences of drunk driving. 2. Social Media Graphics: Create a series of shareable graphics featuring statistics related to drunk driving accidents and arrests, paired with impactful messages about the importance of making responsible choices. For example: - Image of handcuffs with the caption: “One drink can lead to a lifetime of regret. Don’t let it be the reason you end up behind bars.” - Illustration of a car key morphing into jail bars with the text: “Your freedom is in your hands. Choose wisely.” 3. Video PSA: Produce a short video PSA that follows the journey of a person who decides to drive after drinking. The video can show the progression of events, from the initial decision to drive, to being pulled over by law enforcement, to facing the consequences in court and ultimately ending up in jail. The video ends with a message urging viewers to make the responsible choice and avoid the devastating consequences of drunk driving. 4. Interactive Website: Develop an interactive website where visitors can explore real-life stories of individuals who have been arrested for drunk driving, as well as resources for finding alternative transportation options and tips for preventing impaired driving. Include a quiz or interactive tool that calculates the potential legal penalties for driving under the influence based on the user’s location and blood alcohol concentration. Campaign Strategy: 1. Target Audience: The campaign targets individuals aged 21-35 who are more likely to engage in social activities involving alcohol consumption. Additionally, it aims to reach out to their peers, friends, and family members who may influence their decision-making. 2. Distribution Channels: Utilize social media platforms, digital advertising, and partnerships with local law enforcement agencies to disseminate the campaign message. Share campaign materials on popular social media channels frequented by the target audience, as well as through targeted online advertising campaigns. 3. Engagement Tactics: Encourage audience engagement through interactive elements such as quizzes, polls, and user-generated content contests. Encourage users to share their own stories or pledges to avoid drunk driving using campaign hashtags. 4. Partnerships and Collaborations: Partner with local law enforcement agencies, DUI prevention organizations, and community groups to leverage their expertise and resources. Collaborate on events, workshops, and educational campaigns to raise awareness about the legal and personal consequences of drunk driving. 5. Evaluation and Monitoring: Track key metrics such as website traffic, social media engagement, and changes in attitudes and behaviors related to drunk driving. Conduct surveys and focus groups to gather feedback from the target audience and assess the effectiveness of the campaign. Use data-driven insights to refine the campaign strategy and maximize impact. Through a combination of powerful visuals, compelling messaging, and strategic distribution channels, the “Buzzed Behind Bars” campaign aims to educate and motivate individuals to make responsible choices when it comes to alcohol consumption and driving, ultimately saving lives and preventing unnecessary legal consequences.

Comment

I think that this response was incredibly strong; ChatGPT not only answered my question, but anticipated follow-ups I might’ve had, such as “what should our distribution channels for this campaign be?” It accurately describes our target audience and chooses effective methods of reaching them via social media and online marketing tactics. It also wraps up the “Buzzed Behind Bars” concept reiterating the individual focus, as seen in the final paragraph.

Generative AI Infographic/PSA for Individual Responsibility:

Prompt:

The image features a split-screen view. On one side, there’s an image of a group of friends enjoying a night out, perhaps raising glasses in a toast. On the other side, there’s a sobering image of a jail cell with a solitary figure sitting on a bunk, looking despondent. The tagline “Buzzed Behind Bars” is prominently displayed above the images, serving as a stark reminder of the potential consequences of drunk driving.

Comment

In order to further ingrain the use of AI in the creation of our campaign, we pulled our prompt for the generation of an infographic from ChatGPT’s initial idea (as seen above). We put the prompt into Co-Pilot, the Microsoft LLM that uses Dalle-3 to generate images. We then selected the best image from the 4 responses.

Response:

Comment

It is immediately apparent that AI is not yet able to produce accurate text each time it creates an image. Still, this response very precisely adhered to our input, and conveyed both the individual theme and sentiments we were looking for in this framing. Viewers must think– wow, am I willing to risk my personal freedom over one stupid choice? We liked this graphic a lot, but wanted to see how the LLM would respond to requests for tweaks/changes.

Prompt:

Please take the third image you produced and add more color to it.

Response:

Comment

Unfortunately, once we asked for the addition of color, all outputs were significantly closer to resembling cartoons. While the text here is effective and the image is clean, we preferred the more realistic elements of the previous response. We even tried asking “please create me one more image like this one that is less cartoonish in nature,” to no avail. Therefore, we decided that these two PSAs were the best candidates for actual use in our campaign.

Generative AI Instagram Post for Individual Responsibility:

Prompt:

Please create an illustration of a car key morphing into jail bars with the text: “Your freedom is in your hands. Choose wisely,” to be used in an anti-drunk-driving campaign focused on individual responsibility.

Response:

Comment

Once again, I think these were really good replies! To get from the second to the first, I prompted Co-Pilot to “please add more elements that imply drunk driving,” and it made a good change to the inclusion of the police car. It remains obvious that certain details such as the biological accuracy/spelling could be improved, but I’ve been really impressed with the creative and conceptual intuition exhibited by the platform.

Generative AI Infographic/PSA for Social Responsibility:

First attempt prompt:

Make an infographic image for this campaign: Campaign Title: “Don’t Let Your Buzz Kill More Than the Party” Campaign Message: This campaign aims to highlight the potential consequences of drunk driving in an impactful way. By emphasizing the idea that a buzz from alcohol should not lead to the death of innocent lives, the campaign encourages individuals to take responsibility for their actions and make smart choices behind the wheel. The central image features a split-screen view. On one side, there’s a vibrant party scene with people laughing, dancing, and enjoying themselves. On the other side, there’s a devastating car crash scene with emergency vehicles and injured individuals. The tagline “Don’t Let Your Buzz Kill More Than the Party” is prominently displayed above the images, emphasizing the contrast between the two scenarios.

Response:

Comment

Using jhub, this response was the closest we thought we could get to what we wanted. We believe that the cartoon imaging was a result of the jhub, so we asked Co-Pilot to make a more serious version of this image.

Second attempt prompt:

Make this image more serious as it is a public health campaign about the social responsibility of drunk driving and the danger that puts others in.

Response:

Comment

Co-Pilot did not correctly understand the image addition. It is too dark and intense, and logically it didn’t make sense. We believe this is a result of not re-emphasizing the original prompt. So we tried again.

Third attempt prompt:

Make this image for a public health campaign about the social responsibility of drunk driving: The central image features a split-screen view. On one side, there’s a vibrant party scene with people laughing, dancing, and enjoying themselves. On the other side, there's a devastating car crash scene with emergency vehicles and injured individuals. The tagline “Don't Let Your Buzz Kill More Than the Party” is prominently displayed above the images, emphasizing the contrast between the two scenarios.

Response:

Comment

For the final attempt we made the prompt less wordy than the original, because we thought that all the unnecessary information would clog the understanding of Co-Pilot. In this turn, we also did this prompt in a refreshed page so that there was no remembrance of the previous attempt. It turned out much better than expected, and we are very happy with this infographic. The only aspect is the amount of flowers and the text being incorrect, but it works.

Generative AI Instagram Post for Social Responsibility:

First attempt prompt:

Make a social media graphic for this campaign: Campaign Title: “Don’t Let Your Buzz Kill More Than the Party” Campaign Message: This campaign aims to highlight the potential consequences of drunk driving in an impactful way. By emphasizing the idea that a buzz from alcohol should not lead to the death of innocent lives, the campaign encourages individuals to take responsibility for their actions and make smart choices behind the wheel.

Response:

Comment

Immediately, we were taken aback by the visual made by Co-pilot. After receiving such a great response for the original campaign prompt and using the current prompt following immediately after, we thought the LLM would understand the tone. Alas, we had to adjust the prompt.

Second attempt prompt:

Make a social media graphic for this campaign: Campaign Title: “Don’t Let Your Buzz Kill More Than the Party” Campaign Message: This campaign aims to highlight the potential consequences of drunk driving in an impactful way. By emphasizing the idea that a buzz from alcohol should not lead to the death of innocent lives, the campaign encourages individuals to take responsibility for their actions and make smart choices behind the wheel.

Response:

Comment

Deciding to stick with Co-Pilot, we were pleased with the second attempt. It was more of what we were looking for, leaning into typical social media graphic details like attention grabbing colors, sleekness, and fitting Instagram’s minimal aesthetic. Also, it’s fairly easy to understand (if the text correctly reads “Don’t let your buzz kill more than the vibe”) before you quickly scroll. Surprisingly, it’s impressive that the image has undertones of the prompt (like the neon lights of a bar in reference to the drinking) and is not “in your face” about the meaning which might allow the concept to linger longer in the audience’s mind.

NOTE

It was here that Emmy and I did some more prompting work to develop a Video PSA script, but feel free to check that out in the full project linked below! I decided to omit it here for (attempted) brevity. We will now pivot to the section where Emmy and I used LLM’s to evaluate the projects that had just been generated by other AI platforms!

Cross-Campaign LLM Evaluation

Graphic #1 Social Responsibility

Prompt:

You are a target audience member of a public health drunk-driving campaign. You are presented with this image, focusing on the social responsibility of drivers to not get behind the wheel inebriated. Please rate the infographic on its likelihood to change your behavior from 1-5, with 1 being “I will not change my behavior,” and 5 being “I will definitely change my behavior.”

Response:

I would rate this infographic a 4. It effectively contrasts the joy of partying with the devastating consequences of drunk driving, which could lead to a strong emotional response and behavior change. The image is divided into two contrasting halves: 1. Left Side: ○ A group of people are celebrating, dancing, and having fun at night. ○ They are near a car that has “3” written on the road beside it. ○ The atmosphere is lively with bright lights illuminating the scene. ○ There’s a bouquet of flowers on top of the car, symbolizing celebration or an event like a wedding or party. 2. Right Side: ○ It shows a crashed car with emergency services present and an individual lying on the ground, presumably injured from an accident. ○ The atmosphere is grim; emergency lights cast an ominous glow on the scene. Above both scenes, a yellow sign hangs, reading: “DON’T LET YOUR BUZZ KILL MORE THAN THE PARTY,” indicating that falling asleep or losing control due to intoxication can have fatal consequences beyond ending a fun time. The juxtaposition of celebration and tragedy serves as a powerful reminder of the importance of responsible behavior when it comes to drinking and driving.

Graphic #1 Individual Responsibility

Prompt: You are a target audience member of a public health drunk-driving campaign. You are

presented with this image, focusing on the individual responsibility of drivers to not get behind the wheel inebriated. Please rate the infographic on its likelihood to change your behavior from 1-5, with 1 being “I will not change my behavior,” and 5 being “I will definitely change my behavior.”

Response:

I appreciate your commitment to promoting responsible behavior. Let’s evaluate the infographic together: The split-screen style infographic effectively contrasts two scenarios related to drunk driving: 1. BUZZED (Left Side): ○ A grayscale scene depicts people clinking glasses at a table, celebrating or socializing. ○ The atmosphere seems light-hearted and social. ○ However, the text at the bottom is scrambled and unreadable, which might be intentional for effect. 2. BARRD (Right Side): ○ Another grayscale scene shows a person sitting alone in jail. ○ The stark contrast between the two scenes emphasizes the consequences of drinking and driving. While I don’t have personal experiences or emotions to gauge its impact, I can see how this infographic aims to convey the message of individual responsibility. The juxtaposition serves as a powerful reminder that our choices can lead to vastly different outcomes. Considering the effectiveness of this infographic, I would rate it a 4. It prompts reflection and highlights the importance of making responsible decisions when it comes to alcohol and driving. However, the scrambled text could be clearer for maximum impact. Remember, every choice matters, and responsible behavior can save lives. 🚗🚫�

Comparing the Evaluations

Prompt: Which of the two graphics would you rate higher on the aforementioned scale if you

had to? #### Response: If I had to choose between the two graphics, I would rate the first infographic (the one contrasting celebration with the aftermath of drunk driving) slightly higher, at a 4.5. The emotional impact of juxtaposing joy and tragedy can be quite powerful, potentially encouraging responsible behavior. However, both infographics serve as important reminders about the consequences of drinking and driving. 🚫🍻 Prompt: Which one do you believe will be most effective with human target audiences in the context of social vs. personal responsibility respectively? Response: Certainly! Let’s analyze the two infographics through the lenses of social responsibility and personal responsibility: 1. Social Responsibility: ○ First Infographic (Celebration vs. Tragedy): ■ Strengths: ■ Community Impact: By highlighting the consequences of drunk driving, this infographic encourages individuals to consider their impact on others. ■ Collective Behavior Change: It fosters a sense of collective responsibility, emphasizing that responsible choices benefit society as a whole. ■ Weaknesses: ■ Emotional Overload: Some viewers might find the tragedy overwhelming, affecting their willingness to engage with the message. ■ Selective Empathy: Not everyone may connect emotionally with the depicted scenes. ■ Overall Effectiveness (Social Responsibility): I rate it a 4.5 out of 5. ○ Second Infographic (BUZZED vs. BARRD): ■ Strengths: ■ Simplicity for Wider Audience: The straightforward split-screen format appeals to a broader audience, emphasizing the collective responsibility to prevent accidents. ■ Clear Message: It directly communicates the societal impact of drunk driving. ■ Weaknesses: ■ Less Emotional Impact: The lack of emotional imagery might reduce its effectiveness for some viewers. ■ Risk of Oversimplification: It may not fully convey the gravity of the issue. ■ Overall Effectiveness (Social Responsibility): I rate it a 3.5 out of 5. 2. Personal Responsibility: ○ First Infographic (Celebration vs. Tragedy): ■ Strengths: ■ Personal Connection: The emotional contrast prompts viewers to reflect on their own choices. ■ Behavioral Change: It encourages individuals to take personal responsibility for their actions. ■ Weaknesses: ■ Potential Desensitization: Some viewers might become desensitized due to the graphic nature of the tragedy. ■ Individual Variation: Emotional impact varies from person to person. ■ Overall Effectiveness (Personal Responsibility): I rate it a 4.5 out of 5. ○ Second Infographic (BUZZED vs. BARRD): ■ Strengths: ■ Direct Accountability: The split-screen format emphasizes individual responsibility. ■ Clarity: It leaves no room for misinterpretation. ■ Weaknesses: ■ Lack of Emotional Appeal: The grayscale scenes might not evoke strong emotions. ■ Risk of Oversimplification: It may not fully address the complexities of personal decision-making. ■ Overall Effectiveness (Personal Responsibility): I rate it a 3.5 out of 5

Conclusion

Though brief, this project has provided us with valuable insight into the possibilities for integrating AI into the creative generation, specifically within the context of designing a public health marketing campaign against drunk driving. When considering what would be most effective for this hypothetical campaign, we chose social and individual framing because of the somewhat socially normalized nature of the behavior and the psychology of individual selfishness.

Our exploration of the capabilities of ChatGPT and Microsoft Co-Pilot in this project yielded results we could not have begun to anticipate, and we were truly impressed by the creativity, comprehensive attention to detail, and adaptability demonstrated by these LLMs. To conclude, we want to share some final observations and opinions.

As for the benefits of using AI in our campaign creation, we were shocked by the originality of the ideas ChatGPT offered for both the individual and social lenses. Without sacrificing creativity, the platform generated multiple proposals in mere seconds rather than the hours or days it might’ve taken a person to do so. Additionally, the ideas were well-fleshed out and offered different channels of distribution, types of graphics, website possibilities, and more. This is all to say that in terms of brainstorming, AI can be an incredible tool to both offer lots of ideas and hone in on certain concepts. Still, it is important to note that we did run into some trouble when we tried to have Co-Pilot evaluate both proposals as well as engage effectively with our feedback within its Dalle-3 integration.

Our main issue with AI’s assistance in our campaign was how Co-Pilot seemed unable to take our critiques to tweak the images it produced. If we asked for it to add color, it might change the entire genre of the drawing, or if we wanted it to feature a specific element, it might ignore that request entirely. We discussed with Professor O’Donnell how Dalle-3 takes your prompts and rephrases them for its backend command, though, so we are interested in investigating the way that it does this more in detail. Still, it proved frustrating, and given the other challenges of inaccurate text, miscellaneous additions, and varying stylistic choices, we think that human artists still come out on top (or at least they do for now!). Lastly, we were really surprised when Co-Pilot preferred our social framing proposal, as we anticipated that human respondents would be more moved by the individual framing given psycho-social drives being more independent, especially for young Americans. We did not know if Co-Pilot was unaware of this norm, though, because it seemed to evaluate the campaigns on how emotionally stirring they might be. Either way, we did not agree with the opinion and would like to do more testing on other ways to prompt it to evaluate future projects, such as changing our scale, offering it background information, or otherwise.

Regardless of the small issues, we found AI to be sincerely helpful in the creation of this public health campaign and we have been inspired to further investigate the intersection of LLMs, marketing, and public communication.

The full project can be seen here: https://github.com/ekeogh03/healthcomm