Tuesday, March 27, 2012

Stochastic Ramblings About St Pete...

The first race is in the books and despite the predicted doom and gloom being nowhere to be found (exploding engines, indycar drivers fubar-ing it on carbon brakes, carbon fibre ass implants for cars littering the track, lotus delivering no engines etc) another flavor of doom and gloom has taken its place.  Rumblings of an impending doom were heard as people tweeted or trackforum-ed comments about a boring race, the kind that chases fans away.  The shit storm took off full bore when Jenna Fryer at the AP released the preliminary overnight Neilson rating of 1.1 on Monday. 

That lit up a conversation about “how is a sport expecting to grow” when it has racing that puts people to sleep, which was countered with the argument that there was plenty of good racing on the track, problem was that ABC didn’t bother to show any of it.  Which led to other silliness about road course racing being an acquired taste that requires the use of timing and scoring data streams being fed to super computers utilizing complex multivariate analytical modeling for maximum enjoyment.

I don’t want to rehash most of the argument because what it all lacks is the data to reconcile facts to wandering opinion.  Without data and facts, these arguments are sheer noise that I guess some people find enjoyable.  Some of this data exists, but since that data is purchased by networks and publishers under exclusivity arrangements with the data providers, we won’t ever be privy to it and quite often the data that is needed to reconcile the issues simply does not exist.  I do want to interject a few points to the conversation, some of these points will illustrate how data might swing the conversation and some just are my 2 cents worth of noise.

On Occasion, 1.1 = 1.3

This “illogical” statement might very well be true depending on the variance associated with the numbers being reported.  Understanding that variance is the point of statistical inference.  Rarely in life can we ever measure a number we are interested in exactly and that is particularly true when trying to measure the behaviors of large numbers of people.  It is simply too expensive and impractical to ask every single person in the country if they were watching the IndyCar race on Sunday.  So what gets done instead, Neilson essentially asks a sample of people if they were watching the race (via a monitoring box attached to their TV’s later reconciled and validated with set top box data purchased from cable and satellite providers – hence you have an overnight # and a final #). 

Sample stats are estimates of the numbers we want to know but they are never the exact number we are looking for.  Samples are accompanied by a measure of spread called a standard error that can be used to make an inference about how close the true number is to the sample estimate.  Think about the +/- range that is associated with political polling – marketing research data is similar.  If you want to reduce the error in a sample estimate you have to remove sources for bias (making the constitution of the sample to be representative of those in the universe) and increase the size of the sample you are measuring.

Where this is going…In sampling it is possible to observe two different sample statistics that are not “Statistically different” from one another.  To ramble on in doom a gloom terms about point estimates of TV ratings data that may not be statistically different from one another is something that I as a professional marketing researcher find annoying as hell.

So is a 1.1 for Neilson statistically equivalent to a 1.3?  My guess…working through sample size and tolerance equations backwards…The number is probably just shy of being statistically different. (Assuming that Neilson is using a sample of 15000 to estimate the ratings yields a point in time difference required for significance greater than the .2 that has been reported).  The learning here – never sweat a .1 drop, but don’t celebrate it either simply attribute the difference to sampling error and move on.  A shift of .3 is a noteworthy shift and of course IndyCar this past weekend is stuck in between the two.

Not all 1.1 ratings are the same…

A 1.1 rating can be garnered in different ways.  A Neilson rating is a composite rating averaged over the duration of a broadcast.  SO it is possible that the audience started off at 1.9 and then after 30 laps of follow the leader, dropped to .7 over the final two thirds of the race and you get a 1.1.  Or it could be that the 1.1 represents that a consistent number of viewers watched the broadcast all the way through the broadcast window.

As it relates to the controversy over the ABC/ESPN broadcast, knowing how the 1.1 averaged out is an important piece of data to know before assessing blame.  Neilson has that stream of data as do the league and the network.  Unlike the overall rating which is openly shared, this data is purchased from Neilson at a fairly high price and there may no reason for the purchasers to share and depending on contractual arrangements with Neilson they may not be able to share it if they wanted.  This stream of data was referenced last year by Randy Bernard in the context of the “turn the tire” halftime game show at Texas, the ratings dip during that part of the broadcast was enough to make sure the two step did not happen again this year.

If the rating started high then finished low, then it becomes clear that pre race promotion by the league and the broadcast partner were effective in attracting new viewers to the show.  The drop off thereafter then represents an unfortunate event that viewers were not amused and went elsewhere. 

Is that damning for the league or for the network? 

I would like to see the league go back through timing and scoring and account for how many on track position exchanges occurred Sunday, they did it for Brazil a couple years ago, so why not here as well.  We can then cross reference that number to the number of live passes ABC managed to get on air and how many they managed to replay.  The comparison should settle the dispute in a hurry.

My guess based on my anecdotal evidence is that ABC missed a bunch.  I know that after his last pit stop (one of the last overall) I was watching JR and seeing his position consistently improve from 13th, I thought wow he must be on a charge, other guys who pitted much earlier are saving fuel and he is running full rich on reds, I wonder how far he could go.  I told Jenny I thought a top 5 was possible.  Of course we never found out because all the sudden the crawler said he was DNF without a word from the crack broadcast team as to why.  Come to find out later, he was alongside (and about to pass) Willy P for sixth and the fuel pressure went bye bye.

What irks me, is that if I can stare at the crawler and know where the race is, why can’t ABC?  I have no idea what they have their people doing in the booth, and perhaps what I am about to suggest is already done.  BUT, someone should be watching T&S at all times,  I would even suggest programming a special data screen that delta’s the gaps from lap to lap so that it is easy to identify position battles that are closing and ones that are widening.  The assistant jr intern monitoring this can relay “Watches or Alerts” to the producer and camera men so that if one of these gaps closes to less than a third a second, people can be ready to cut to that battle on the track.  It’s about data, not spotters.

But what if the 1.1 was consistent?

That is the worst news of all for the league.  I feel like much of the work IndyCar has done during the off season has been solid, old school promotion and information sharing.  IndyCar has had consistent coverage via AP and USAT that it has not always had.  A great press announcement about Turbo happened shortly before the season started.  Lots of buzz about new cars and engines…and nothing much to show for it. 

To answer the reason why, we might have to address…

The Big Pink Elephant in the Room

Or at least the purple sparkle pony that left the room.  All through the debate no one has mentioned that which most of us don’t want to acknowledge as a possibility…Perhaps the Danica fans are no longer tuning in…and perhaps there were more of them than we might have wanted to believe existed…In theory it might be possible to find people who watched any IndyCar race last year and ask them if they watched St Pete and if not why.  If Danica rose to the top, we’d have our answer.  But unfortunately the only company to a list of such people is Neilson and access to that panel is very expensive for a scenario such as this.
In any stream of clientele, you are always gaining and you are always losing.  We may now know what the “Danica Factor” was and can move forward from a new benchmark.  Between the new cars and engines, a road course heavy schedule and the inclusion of Simon Pagenot and Rubens Barichello it seemed intuitive to me that anyone who considered themselves a road racing fan in this country now had few excuses not to watch IndyCar to see what it was all about.  But in reality ALMS and Grand AM have smaller ratings than IndyCar and despite some digging I can’t find evidence that F1 has larger TV ratings than IndyCar so there is a clear ceiling on growing IndyCar from the organic road racing fan base.

Growth then is going to be about either converting nascar types to watching or by converting racing virgins to the sport.  I have thoughts on both of these…but it’s getting late…so you are going to have to wait until the fancy strikes me and I feel inspired.  Right now I just feel tired and I have one more day before my Barber mini vaca begins…

2 comments:

  1. My wife and I are both fans, but she was asleep by about Lap 30 and only perked up occasionally throughout the end. By the beginning of the 2011 season, I don't believe there was any significant group of people tuning in just to watch Danica. I think that it's a big stretch to have something like IndyCar on a national network like ABC, and I'm fine with that as a fan as long as the series can stay profitable enough to get on cable and hold races that I would want to attend. Outside of Indiana, IndyCar is largely a niche sport. The NHL (barely) has a network television contract (one game a week with NBC, starting only after January 1st). The IndyCar series from a national perspective is propped up by one event (the Indy 500), much like people who never bet on horses will tune in for the pageantry of the Kentucky Derby. And, unfortunately, when you show the fans 80% road/street/airport courses that look nothing like the excitement of a high-speed oval, you're not going to retain those fans. Americans don't understand road racing for the most part, the strategy and pacing feels as alien to them as the Tour de France.

    ReplyDelete
  2. Just catching up on my Google Reader (yep, I'm almost 3 weeks behind), but I had to say that I agree with every last word here, especially the stuff in the "Damning for the League or Network" section. The lack of somebody in the ABC booth directing the action is absolutely egregious...I spent most of the last 30 laps of the St. Pete race watching my computer monitor and just listening to Marty & Scott instead of paying any attention as to what was on the screen. Watching drivers bounce up and down and gaps fluctuate was far more compelling than watching single drivers drive around on their own.

    Anyway, great post, and it was great to meet you and Mrs. JP at Barber. Here's to seeing you again in, oh, about a month and a half or so.

    ReplyDelete

My Blog List