A Gentle Introduction to Poisson

With some caveats, Poisson [pwah-sohn] is like an intermediary between binomial and normal distribution.

WTF is the Poisson Distribution?

Alongside the binomial distribution, the Poisson distributions is one of the many non-normal distributions that you may come across in your work as a data scientist or statistician. As crucial to knowing what the Poisson distribution is knowing when to use it.

Similar to a huge binomial distribution, for a large enough Poisson distribution it will become similar enough to normal distribution. Unlike binomial distribution one of the qualities that Poisson distribution shares with normal distribution is that the trials are not discrete and instead happens over an interval.

Imagine you work at a fast food chain — this may be easier for some of you more than others — and you’re thirty minutes away from ending your Saturday morning shift. You’re trying to predict how many customers you will have to serve before you’re allowed to clock out. On average nine­­ customers show up at this half-hour interval on Saturdays, so you start to wonder what the probability of getting just one customer will be. Enter Poisson! You plug this function into your Point of Sale console:

After you finish the math, you see that just getting one customer in a half hour is pretty low! So you begin to bargain with yourself, as long as you get less than six customers, you’ll be happy. What would that probability be? Since Poisson handles discrete quantities, this probability would be the sum of the probabilities of getting 5, 4, 3, 2, and 1 customer, which together are fortunately much better odds than your previous probability.

Little did you know the executives at corporate in a last ditch effort to save their failing brand orchestrated an unauthorized tie-in with a certain adult animation show that has a rabidly impassioned fanbase show up to your location in the hundreds screaming about a condiment that isn’t one of the seven sauces you readily have available. Not only can you not clock out, but they’ve barricade the doors. What were the chances? Well not totally impossible apparently! Even though the distribution has a strong positive skew, Poisson’s probability ranges from 0 to infinite.

Among the other assumptions of the Poisson distribution is that rate at which events occur is constant and the events are independent of each other. However perhaps the most important of these assumptions is that the mean is equal to the variance, an assumption most real world data violates. This would assume that with a λ = 9, your standard deviation would equal three, meaning your probability of seeing six customers is just as likely of seeing twelve customers. Though this might work for the franchise location you’re currently at, at the Times Square location for this time interval, they see one hundred customers on average and their probability of seeing ninety customers could be at least two standard deviations away.

Knowing these assumptions, try to figure out which of these situations could not have the Poisson distribution.

1. The average number of fatal airplane crashes in the past 40 years is three. Could one use poisson to find the probability that there will be less than one airplane crash this year?

2. The average number of fatalities of plane crashes a year is 350. Could one use Poisson to find the probability that there will be less than 200 fatalities this year?

3. The number of bicyclists that cross the Brooklyn Bridge each day. Could one use the Poisson distribution to model predictions?

Poisson Regression in Python

This last example under girds Sachin Date’s wonderful illustrated guide to creating Poisson Regression in Python.

Let’s go back to our fast food example. It is the same half hour interval the follow Saturday, only now it is raining. The average number of customers is lower on days when it rains. Or let’s say on days where you have a hard time parking your car, you notice that the average number of customers is higher.

A regression model based on a matrix of this predictor data would have the following form:

Unlike Simple Linear Regression, finding the coefficient β uses the Poisson ­­­probability function from earlier.

Sachin Date continues his tutorial with a great explanation of how to employ Poisson regression with your data using , so you can give it a try with your own count-based data set.