I have been watching the polls with a keen interest of late. As I look into the Cross Tabs and Demographics of the numerous polls that seem to favor Obama, what I find is that the polls sample Democrats at a much higher rate than is reasonably believable.
Real Clear Politics is supposed to be an average of these polls and the theory is that the polls that are out of line with the majority of the others will be evened out. And there is some validity to that theory, except when all of the polls are heavily skewed in favor of the Democrats.
This bias has been fascinating to watch and even more fascinating to document, which is what I began doing with Political Poll Check, a new site dedicated to analyzing and “normalizing” the polls used primarily by Real Clear Politics. And it also allows me to keep watch on RCP and document what they are doing.
Since the site is only a few days old, the data is somewhat limited. But the process of documenting has included taking screen shots of the RCP poll average box each day.
I have always wondered how they decide which polls to keep in their average and which ones they toss out after a certain point. And what is that point?
One would assume that they throw out stale polls after several days and keep the newest 8 or 10, right?
One issue is that most of these polls span several days, so start and stop dates overlap.
But this morning around 6:00 AM, RCP included 10 polls that show Obama ahead by +4 Points. And this evening there are only 8 polls and Obama has the same +4 point average lead.
So what happened?
Well, here is this morning’s snapshot:
Note that there are 10 polls with the dates ranging from 9/11 – 15 days ago – to 9/24 – two days old. And if you average the Spread (6+1+3+3+7+1+3+3+5+8) you get 40 , divide by 10 and you get the +4 Spread for Obama.
Now take a look at a snapshot taken at 8:30 PM on 9/26.
As you can see, there have been some changes. Most notably, there are only 7 polls in the current average. And the date range is from 9/12 to 9/25 which represents a full one day bump forward on both ends.
I’ll buy that. Fair enough. Move the sliding average one day and there is no movement in the average. Obama holds to a steady 4 point lead. No real change, right?
Wrong! Let’s take a closer look at that they did.
Rasmussen released a new rolling average poll and they swapped out the older (by one day Rasmussen poll) for the freshest one. No problem with that! Obama lost a point as the latest poll has them “even” instead of a 1 point Obama lead.
Bloomberg stayed the same.
Gallup was also swapped for the newest poll from them. This poll saw the Obama advantage of 3 points become a 6 point jump. As far as the RCP average goes, that is fine. (I am not going to comment on the individual polls here.)
Politico stayed the same as did National Journal and Associated Press / GfK.
Gone from this morning’s average is the Hartford Courant poll, the Monmouth Poll and the Pew Poll leaving the NBC News poll.
Here is where the manipulation comes in. I understand removing older polls and leaving the “freshest” data to average, but Real Clear Politics actually left the NBC Poll that had Obama +5 and deleted Monmouth, Hartford and Pew.
The Hartford poll had the oldest data in the poll – from 9/11 and I see no problem with deleting that one.
The next two older polls were the Pew and the NBC ones. Both had a date range of 9/12 – 9/16. The Monmouth poll dates are newer – 9/13 – 9/16. Yet RCP eliminated the Monmouth Poll that had Obama up by 3 points in favor of the older NBC poll that had Obama up by 5 points.
And what does that do to the average? Well, leaving the NBC poll the average remained at 4 points for Obama. Had they left the Monmouth poll with the fresher data instead, the average would have changed a bit.
Doing the average – Tie + 6+6+3+7+1+3 = 3.7 Obama lead average, a drop of .3 (RCP uses 1 decimal point in their averages.)
But by leaving in the older polling data from NBC, the average remained unchanged at Obama +4.
As far as I am concerned, the jury is still out on the objectivity of Real Clear Politics. But from now until the election, we will be documenting everything they do. I believe that objectively speaking, Real Clear Politics should have shown a 3 tenth of a point decline in Obama’s lead after their updates. There is no fair or logical reason to leave the NBC poll in their average. They had a fresher poll in the Monmouth poll that had almost double the sample size (1344 LV) and a 2.7% Margin of Error compared to only 736 LV and a 3.6% Margin of Error.
This is the first documented case of possible bias we have found with RCP and we hope that this is some oversight on their part.
The polls they average may or may not be biased or accurate, but they can be certain of one thing. Every anomaly such as this one will be documented and pointed out from now on. This is only a .3% manipulation, intentional or not, but since the Mainstream Media follows not only the averages of RCP but the trends as well, it is a significant incident.
Note: I have contacted Real Clear Politics for a response but have not received one so far.