Podcast Transcript: Temperature Check Survey

LIVE INTRO: Welcome In! Coming to you on the Concurrent Media Network in the heart of Citrus County. Thank you for joining ad-free on the Concurrent app or website.

It’s great to be back. There’s going to be plenty of time to discuss where I’ve been and what happened in the meantime, but let’s start with why now. Why now? It’s a holiday, my favorite holiday in fact, and it’s a Monday - not the traditional podcast Tuesday.

To figure out how we got here, let’s go back to the very beginning. What led me to starting the Concurrent was that I felt the Chronicle repeatedly misrepresented situations, particularly in its editorials, for its own agenda, which is not always in line with what is best for the public. Before you dispute this, I’ll provide you with a clear example of what I mean from yesterday’s paper, I’m publishing this on Monday so the Sunday July 2nd edition, and then you decide for yourself whether you agree with me or not. This sounds like it’s going to be Chronicle-bashing, and in about two minutes it might come across like Chamber-bashing, but I assure you that it is neither. Today’s episode is taking a stance on group-think. I’m against it.

Here’s what happened. The Citrus County Chamber of Commerce performed an annual temperature check survey which means they ask a bunch of questions regarding to approval rating of everything from local government officials to the level of optimism about the state’s future. The survey itself is not a problem, and I’m happy the Chamber does this. They should continue to do it every year and I appreciate the work that goes into it.

Things go from good to bad to…well, weird, in a hurry. Let’s start with the sample size. Mike Wright was the first to report on the results and he said that about 10% of the chamber’s 900 members participated. Oh, here we go, you detractors are thinking, “Bobby’s going to take issue the sample size.” No, not really. 90 of 900 would be adequate sampling for purely the organization’s membership albeit too low to project onto the entire 124,000 voting population of the county.

My first problem with the analysis is with consistency of sample size. The Chronicle editorial said that approximately 15% of the chamber’s membership participated in the survey and that this was 150 people. This would put the chamber membership at 1000 rather than Mike Wright’s guess of 900 and the difference between 90 people and 150 people participating in the same survey would yield wildly different results…except the rest of their numbers are nearly identical. In fact, the only numbers that don’t line up are a percentage point off for some of the elected officials where it appears that Mike Wright rounded up and the Chronicle rounded down.

For example, Commissioner Jeff Kinnard was at a 53% approval rating on Just Wright Citrus and a 52% in the Chronicle. There were three other instances where the ratings were off by one, all with Just Wright Citrus one higher than the Chronicle. This begs the question of what the cut off point to round up was, but the better question might be then why does the Chronicle appear to overstate the survey’s population while Just Wright Citrus understates it if the opposite is true across four other examples when it’s Mike Wright rounding up and the Chronicle rounding down.

This is more peculiar than it is problematic. So let’s get to that second part now. It’s unclear from both the Just Wright Citrus and the Chronicle Editorial Board’s analysis whether the respondents represent individual businesses or could be multiple people within the same business. For instance, did the entire Chamber staff take their own survey? Would independent contractors, who technically work for themselves but who are united by a single entity such as in the case of REALTOR offices, each get a say?

This sets the stage for what in statistics is called a selection bias. Selection bias is a bit of an umbrella term encompassing many possible errors in conducting quantitative study, so to be a little more specific, this is sample selection bias but also called just sampling bias. We see it in academia all the time. Scholars, including myself as a grad student, conduct experiments on the most convenient available populations. On a college campus, this typically means 18-22 year olds with socioeconomic status and education backgrounds above the national norm.

We even have a somewhat derogatory acronym for studies that fall into this sampling bias calling their data collection WEIRD, or western, educated, industrialized, rich, and democratic - all of this of course is relation to the global population. And you hear it all the time, wow, did you see the new study NYU Stern B-school just published? Yes, but the data collection was WEIRD. Ohp, can’t take much from it.

And that’s what we have here, especially if the results represent multiple staff members within the same business which they almost certainly do. You have an increasingly small, selectively sampled bias group of people who, for the most part, all think alike. So is this the group think I have a problem with? No, not there yet, but I promise you it’s coming soon.

Mike Wright and the members of the Chronicle ed board might not know the terminology behind this, but come on, anyone can look at a survey with somewhere between 90 and 150 respondents and say, something isn’t exactly right here. Which is exactly what both should have said. It would have sounded something like this, “the Chamber released its annual temperature check survey, and while the numbers are skewed beyond informative data due to imperfect data collection, the Chamber should continue to perform additional local quantitative studies until it has meaningful results.” I told you from the start, this doesn’t have to be Chamber-bashing. People worked hard on this and it has potential.

Instead we got this, from Just Wright Citrus, *because I’m a trend guy, I find this info useful* okay, not sure what that means and yet it’s still better than this from the Chronicle “it is a fair guess that(the Chambers) membership shares many opinions and values of the general population.” No it’s not. And a selection bias sample is not representative of the business community nor would a 150 person sample be in any way a better than chance guess of what the Citrus County voting population is thinking.

The only fair guess from this analysis is that both Just Wright Citrus and the Chronicle are completely incapable of calling out crap when they see it IF it goes against positions they hold themselves.

They go along with it because it reaffirms previously held beliefs which at worst is confirmation bias and at best is evidence of blind group-think, which is the only thing I’m against in this topic.

Look, I brought the Concurrent back for the same reason it started. Editorials like this are what communication academics call malinformation. You’ve probably heard of misinformation, which is wrong but doesn’t hurt anyone like saying 2 + 2 = 5 or the sky is plaid, and you’ve heard of disinformation which is wrong but with a malicious intent such as saying commissioners are lining their own pockets, but malinformation is reporting on true information incorrectly to serve your own purposes. These survey results did happen, that’s true, but the Chronicle editorial board is doing the public a disservice through their inability to break from the group-think with the Chamber’s cliquish inner circle that responded to the survey that produced the results the editorial board wanted. That’s it.

Here’s the point I’ll leave you with. I get why they do it. Political parties are going through the same thing right now calling for unity as establishment Democrats try to corral young progressives behind their elderly president and Republicans head toward a heated primary season, mainly between Trump loyalists and the rest of the field. But what the Republicans are missing, the Democrats too, and the Chronicle here with the Chamber is that unity is not as important as understanding - I wrote about this in the second ever column of the Concurrent two and a half years ago. Understanding is something entirely different from unity. It’s saying, I get why this happens over here, and I know why this occurs over there and these two things are separate so let’s analyze them objectively, and if we still are apart on what objective means, let’s debate them amicably. That’s completely absent from our discourse now. It’s also why we’re back, and I’m so excited you’re here too.

SIGN OFF - That’s all from the Bobby Winsler Show. Download the Concurrent app to get the latest news from around the county, join our conversation on discussion boards or social media, and be part of the best thought-provoking editorial analysis.