Coronavirus Predictions

By: Dr. Ikjyot Singh Kohli

I wrote an extensive script in R that takes the most recent data available for the number of new/confirmed COVID-19 cases per day by location and computes the probability using statistical learning that a selected location will observe a new COVID-19 case. You can access the dashboard by clicking the image below: (Beneath the screenshot are further examples of possible selections.)

Here, we see a map of all current COVID-19 locations, and an ability to select a specific location. Further, there are two calculations at the bottom of the screen: the first is the selected location(s) probability of observing a new case, the second is the current long-term trend of the daily growth rate of new cases for the selected location(s).
In this example, we have asked to return locations within the US that have more than an 87% probability of observing a new case. We can also see that for these locations, the long-term growth rate is trending towards 0.80.

Movie Sentiment Tracker

I wrote an extensive application using NLP and TensorFlow/Keras in Python that looks at all of the current and upcoming Hollywood releases for 2020 and tracks the online Twitter sentiment for each of them. The model output was then displayed in a PowerBI dashboard. In essence, we are predicting the classification probability Pr(Sentiment=Positive|Data).

You can access the dashboard by clicking on the screenshot below:

We have also included a new feature that gives a daily popularity score for movies. An algorithm was designed to rank movies according to daily positive sentiment. This can be found on Page 2 of the dashboard link.

You can select different titles by clicking the dropdown list. The left-side graph shows you the sentiment distribution of all of the tweet data corresponding to a film. The right-side graph calculates the median tweet sentiment for a given day for the selected film. (Right now, we go back 30 days from the present day). It is intended that this dashboard will be refreshed every day.

Did Clyburn Help Biden in South Carolina?

By: Dr. Ikjyot Singh Kohli

The conventional wisdom by the political pundits/analysts who are seeking to explain Joe Biden’s massive win in the 2020 South Carolina primary is that Jim Clyburn’s endorsement was the sole reason why Biden won. (Here is just one article describing this.)

I wanted to analyze the data behind this and actually measure the effect of the Clyburn effect. Clyburn formally endorsed Biden on February 26, 2020.

Using extensive polling data from RealClearPolitics, I looked at Biden’s margin of victory according to various polling samples before the Clyburn endorsement. I used Kernel Density Estimation to form the following probability density function of Biden’s predicted margin of victory (as a percentage/popular vote) in the 2020 South Carolina Primary:

Assuming this probability density function has the form p(x), we notice some interesting properties:

  • The Expected Margin of Victory for Biden is given by: \int x p(x) dx. Using numerical integration, we find that this is \int x p(x) dx = 18.513 \%. The error in this prediction is given by var(x) = \int x^2 p(x) dx - (\int x p(x) dx)^2 = 107.79. This means that the predicted Biden margin of victory is 18.51 \pm 10.382. Clearly, the higher bound of this prediction is 28.89%. That is, according to the data before Clyburn’s endorsement, it was perfectly reasonable to expect that Biden’s victory in South Carolina could have been around 29%. Indeed, Biden’s final margin of victory in South Carolina was 28.5%, which is within the prediction margin. Therefore, it seems it is unlikely Jim Clyburn’s endorsement boosted Biden’s victory in South Carolina.
  • Given the density function above, we can make some more interesting calculations:
  • P(Biden win > 5%) = 1 - \int_{-\infty}^{5} f(x) dx = 0.904 = 90.4%
  • P(Biden win > 10%) = 1 - \int_{-\infty}^{10} f(x) dx = 0.799 = 79.9%
  • P(Biden win > 15%) = 1 - \int_{-\infty}^{15} f(x) dx = 0.710 = 71.0%
  • P(Biden win > 20%) = 1 - \int_{-\infty}^{20} f(x) dx = 0.567 = 56.7%

What these calculations show is that the probability that Biden would have won by more than 5% before Clyburn’s endorsement was 90.4%. The probability that Biden would have won by more than 10% before Clyburn’s endorsement was 79.9%. The probability that Biden would have won by more than 20% before Clyburn’s endorsement was 56.7%, and so on.

Given these calculations, it actually seems unlikely that Clyburn’s endorsement made a huge impact on Biden’s win in South Carolina. This analysis shows that Biden would have likely won by more 15%-20% regardless.