Bot Behavior: How do Citizens Engage with Congressional Representatives via Text

Do extreme personalities and opinions dominate letters sent to U.S. politicians?

In the aftermath of a shocking election, progress as we’ve known it is being radically dismantled, gutted, and revamped. The psychological disorder known as “Trump Anxiety” has become a fixed state for the next four years as Americans wake up day after day to the gray world of alternative facts. We see troves of data that highlight Trump exceptionalism as individuals and institutions scurry to address an unmanageable amount of controversy that surfaces everyday.

While we face a period of diminishing women’s rights, sporadic scientific funding, dwindling environmental regulations, and bare-bones social services, we also see the rise of the resistance movement. We see an emerging network of reactionary organizers, organizations, and campaigns rallying the progressive voices of a grumbling America. This includes grassroots activists hosting house parties and salons, online training schools, and canvassing platforms. And your newest friend, Resistbot.

On March 8th 2017, Resistbot launched to enable U.S. citizens to contact their federal Senators and Representatives via SMS texting. After completing a message via text, a formal letter is compiled into a PDF and faxed to individuals’ representatives. Within several days of launching, it quickly became clear that existing processes to contact elected officials were not adequate and people wanted their voices heard. Over ten weeks, Resistbot gained over 225,000 users. Since April 2017, Resistbot has sent over 1.5 million messages and has users in all 50 states, D.C., and Puerto Rico.

There are adversaries that harp on the fractionary nature of public-interest networks, focusing on the fact that they are specialized and often times ego-driven. Although it’s undeniable that the we live in the nonprofit complex of special interests, civic action and engagement cannot be sidelined or underestimated.

Over the following weeks, we will be posting a series of blog entries that analyze our user activity. Our hope is to provide a better understanding of Resistbot users and the resistance movement at large. All data will be anonymized to protect individuals’ privacy. This week, we are examining the users’ sentiment scores and frequency of messages sent per user, as well as the role of gender in user activity.

Resistbot Users

It is now the expectation that anyone should be able to rate or deliver feedback for goods and services — movies, products, restaurants, travel, food delivery, the list goes on… There is, however, no centralized platform for political feedback. What happens when this technological convenience is transferred into the political realm?

Do extreme personalities and opinions dominate letters sent to U.S. politicians?

*Data computation and methodologies can be found at the bottom of this article.

It turns out, Resistbot users are surprisingly rational people. 95% percent of users wrote messages with a sentiment analysis score between -.38 and 0.5. Therefore, the majority of messages are relatively neutral and do not fall into sentiment extremes. The mean number of messages sent per user is approximately 6.5 messages, an average of 6–7 messages per user, while median is 2 messages. The large difference between the median and mean indicate that there are also a significant number of outliers. The breadth of data indicates that there is not one type of Resistbot user — there are approximately 14,000 users who sent over 25 messages and 3,800 users who sent over 50 messages.

Men vs. Women

Is gender relevant in assessing message sentiment and frequency of Congressional contact?

Overall, we see a difference between male and female users. Resistbot currently has approximately 224,000 active users: 118,000 female and 105,000 male. Although there are about 12,000 more women than men using Resistbot, women account for 57% of messages, while men account for 43%.

75% of female users sent between 0–35 messages, while 75% of male users sent between 0–32 messages. In total, women sent 259,927 more messages than men; women’s mean number of messages sent was 7.67 vs the men’s average count of 6.17. There are significant outliers for total messages sent, which again highlights why there is a significant difference between the median and mean for total messages sent. The standard deviation for both women’s and men’s messages sent is large, which also signifies a large spread of data.

Women and men do not have a significant difference in message sentiment scores. The median sentiment score for men and women was completely neutral at 0.0, and the average score was also relatively neutral for both genders: .0607 for men and .0613 for women.

Is there a correlation between a negative sentiment score and the number of messages sent per user?

Based on the scatter plot below, we do not see a correlation between negative sentiment scores and the number of times a user has contacted their representatives. The visualization shows that the users with the most messages sent to their Congressional representatives fall within relatively neutral sentiment scores.

When we calculate the covariance of the respective subsets of data (female and male), we also find that the covariance is extremely weak: -0.0064 for women and 0.0038 for men. Because of this weak correlation, we are able to conclude that users who have negative sentiment scores do not contact Congress more frequently.

What We Learned

• Resistbot users send messages that have a fairly neutral tone — The average neutrality of messages suggests that users are not following a trend of negative extremism.
• There are more female users than male users.
• Men and women have almost no difference in average message sentiment scores (.01), but women tend to send more messages than men.
• On average, women send one more message than men, but the majority of women have a higher count of total messages sent than men.
• In the Resistbot user population, negative sentiment scores are not correlated with the total number of messages sent. Users with a negative sentiment score are not contacting Congressional representatives more frequently.

Data Analysis Methods

All user data for Resistbot data is collected via RapidPro, an SMS workflow platform. Message delivery is executed through the Twilio API through a Flask webhook called from RapidPro. To extract all user data, I created a Python/Flask app that saved users’ information from the RapidPro API to a Postgres database. I saved all available data from the RapidPro API in a JSON blob field and then extracted the fields relevant to this research.

Then, I created a Pandas dataframe with a csv export from the Postgres DB. (This was significantly faster than connecting to the DB itself.) Since Resistbot does not collect gender information, I used a NLTK gender predictor library to code individual names and assign each user a gender. I then used the TextBlob library to gauge the overall sentiment of a user’s most recent message. The TextBlob library assigns a value between -1–1 to each message, -1 being extremely negative and 1 being extremely positive. The final categorical values used for this analysis are: first name, gender, last message sent, message sentiment score and total messages sent. Lastly, I used Bokeh and Plot.ly to create data visualizations. Code is available here.

Considerations in this analysis

Our data would be more accurate if we collected gender information rather than relying on a gender predictor to gauge a user’s gender. It’s recognized that it is only 82% accurate and it does not account for users who fall outside of the M/F gender binary.

The dataset did not have all messages sent by each user, so our sentiment analysis relies only on the most recent message. Our dataset would be more accurate if we calculated the mean of sentiment analysis scores for each user’s messages.

Thank you!

Thanks for reading. If you want to give Resistbot a try, just text resist to 50409 on any phone, or visit the website to learn more. If you want to volunteer and help create more interesting things with our growing dataset, write us here.