Analysis on Mueller Report

Posted by Mario Wijaya on May 21, 2019

Mario - blog

Disclaimer: The analysis below is provided with an unbiased view and the opinion expressed below is mine and do not represent the opinions of any entity whatsoever with which I have been, am now, or will be affiliated.

Introduction

It has been a while since my last post but today I will deep dive into Mueller Report using the package tidytext. Also, credits to Aditya Mangal for his initial exploratory analysis article posted in Medium that encouraged me to take it further. I recommend checking his article before reading mine.

Environment Setup

Let’s load all of the libries needed for analysis.

Load the Data (Mueller Report)

Here is the PDF from the Justice Department if anyone is interested in reading it. I will use the pre-converted CSV format of the report from here.

Cleaning the data

I used the same data cleaning approach as Aditya on starting a few pages in due to failure of PDF to text parsing and drop the lines which have the majority of words misspelled using the package hunspell in the data with several modifications.

Then, I normalize the lines using tidytext. Notice that I replaced a lot of misspelling words manually due to the strikethrough on the top of documents on sentence such as Attorney Work Product // May Contain Material Protected Under Fed. R. Crim. P. 6(e)

Sentiment Analysis

One might expect that the sentiment of the document is mostly negative but let’s use the three general-purpose lexicons for comparison: AFINN, bing, and nrc.

The nrc lexicon categorizes words in a binary setting into categories of positive, negative, anger, anticipation, etc. The bing lexicon categorizes into either positive or negative categories. The AFINN lexicon assigns words with a score between -5 and 5 indicating most negative to most positive sentiments.

The code below compares all of the 3 lexicons split by the index counts up of 20 pages of text since the individual page might not have enough words in them due to the words on the document are mostly redacted. Now, we plot the sentiment scores across the index and type of lexicon methods used to observe its trajectory.

We see similar dips and peaks in the sentiment of each lexicon methods, however, we can see subtle differences where NRC sentiment is high, the Bing et al. sentiment appears to find longer stretches of similar text, and the AFINN sentiment has more variance.

Most common positive and negative words

Let’s use the Bing et al. lexicon to get the most common positive and negative words on the document. From the plot below, we see several interesting negative words such as obstruction and interference and positive words such as pardon and loyalty.

Wordclouds

What about the good ol’ word clouds? Let’s visualize word that has either positive or negative sentiments using the package wordcloud which aligned with what we have when using Bing et al. approach shown above.

Which page of the document has the highest negative words?

I don’t know about you but I am most interested in a page that contains the highest negative words ratio (negative words/total # words on the page). Again, I utilized bing lexicon to get words that contain negative sentiment and filter to the page that has more than 100 words on it.

Here is what I found by observing the top 5 page that has the highest negative words ratio on the actual PDF:

## # A tibble: 5 x 4
##    page negativewords words ratio
##   <int>         <int> <int> <dbl>
## 1   199            24   193 0.124
## 2   220            34   288 0.118
## 3    61            13   119 0.109
## 4    60            23   212 0.108
## 5   369            35   327 0.107

TFIDF (Term Frequency Inverse Document Frequency)

Without going into too much detail, the definition of tf-idf provided by Wikipedia is great. tfidf is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus. The tf-idf value increases proportionally to the number of times a word appears in the document and is offset by the number of documents in the corpus that contain the word.

The plot above shows the top 15 terms by page number (i.e. ongoing_65 indicated the term ongoing on page 65) with its tf-idf. It seems that the term harm, ongoing, matter, and protected have relatively high tf-idf.

Bigram

Notice that all of the analysis that we looked at so far contains one word such as harm, trump, etc. While one word is very simple to understand, it doesn’t give us a lot of contexts on what the word stands for. With that in mind, let’s try to look at two consecutive words which are known as a bigram.

The interesting result of the above plot is that most of the redacted pages have ongoing matter on top of the black rectangles. While prosecutorial judgment seems to appear a lot in the introduction and conclusion section of the actual PDF report.

Topic modeling

Although the Mueller report is considered as one topic, let’s try to see if we can find terms that we can separate into 2 or more topics. For the sake of simplicity, we will choose 2 topics. Topic modeling is the unsupervised classification of documents (think of it as clustering on numeric data) that finds natural groups of the terms in our case.

One of the most popular methods for fitting a topic model is Latent Dirichlet Allocation (LDA). It treats each document as a mixture of topics, and each topic as a mixture of words. In our case, a page will be considered as a document.

In order to use the LDA function provided in the package topicmodels, we have to convert the dataframe to a Document Term Matrix (DTM) format as shown below.

Next, we run LDA in our newly converted Document Term Matrix and specified k = 2 topics. Notice that we used the tidy() function to extract the per-topic-per-word probabilities, called “beta”. Then, the plot shows the term by beta value based on 2 topics. The terms appeared on the first topic seem to be the center of the Mueller Report: president, trump, campaign, manafort, cohen, 2016, justice, russian, and office. If you are up to date with news, that shouldn’t be unfamiliar to you.

The terms on the second topic seem to indicate president, trump, and series of number. 302 appeared a lot on the footnote while I couldn’t find much information on the other numbers from the good old (ctrl/cmd + f) on a PDF document. Do let me know if you happen to find any interesting things from those numbers.

Let’s try a different approach by finding the terms that has the greatest difference in beta between topic 1 and topic 2 by utilizing the log ratio of the two. Why log two you asked? The difference is symmetrical. The log ratio of 1 is equivalent to the beta of topic 2 is twice as large while the log ratio of -1 is equivalent to the beta of topic 1 is twice as large.

Now, let’s compare the terms on these 2 topics. We can see that the topic 2 has a lot of terms related to Russia and Trump’s administration official such as Kirill Dmitriev, Rick Gerson, Erik Prince, Stephen Miller, Reince Priebus, Steve Bannon, etc. While the topic 1 contains terms such as statutes, jury, intent, court, constitutional, corrupt, etc. This approach actually yields quite an interesting result that I didn’t expect to find.

Conclusion and Learning

I hope you find the analysis to be interesting and feel free to reach out with any question/suggestion. I have provided the link to the entire code on the resources section. Thanks for reading!

Resources