User ID:
Password:

 
Remember me
Lost password?

FRIDAY GETAWAY DAY
11/18/2011
Friday Fishwrap
Grade Inflation Is Destroying Wine Ratings

By Charles Olken

When I was growing up on the mean streets of Boston, if one got an 80 in a class, that was an “honors grade”. Get all 80s and above and you were on the Honor Roll. Average 85 and you made the National Honor Society. Not only that, you were a candidate for Harvard.

Today in California, if a student does not get all As, that student is virtually frozen out of the University of California system and most other leading institutions of higher learning. Apparently, the concept of grade inflation did not start with wine writers.

And I don’t solely blame my colleagues for their excesses. I also blame the retailers, the wineries and, ultimately, the consumers. You see, here is how it works. The more high grades one gives, the more wines one’s readers get excited about. But the readers do come last in this equation. It is the industry that pants after, that lusts for, that has been known to beg for scores in the 90s. And there are some in the industry who will cut off the supply to winewriters if their scores are not high enough. Never mind that reputable critics taste blind and call them the way they see them.

As one winery said to me when I rated its Chardonnay at 88 points, which is a recommendation on the CGCW scale, “You don’t like our style”. The wine had just outpointed over half the California Chards being reviewed, and it was recommended, but the winery argued vociferously that we did “not like the style” and stopped sending us wine. Last year, a winery sent us nine Zinfandels. We recommended five of the nine, a recommendation rate for that winery of 55%. A typical Zin issue sees us recommended 35-40% of the wines tasted. But we panned one wine, and that apparently so enraged the owner of the winery that he sent a slew of angry emails, the final one of which vowed never to speak to CGCW again.

The consumers come into play here as well. It is an insidious game because everyone wants higher scores. Readers want the critics to agree with them. Try giving an 85 to someone’s fancy White Burg purchase or their favorite expensive Barossa Shiraz or Russian River Pinot and see how fast the letters come in. Try getting a winery’s attention if you constantly rate their $50 Cabernet in the 80s. See how fast the retailers rush to ignore you if you do not recommend wines in ways that help them sell the wines on their shelves. It is more than economic self-interest. It is delusional to think that raising all scores will mean anything of significance.

Here’s an honest to goodness real letter, in part, “I love Connoisseurs’ Guide but couldn’t you find a way to say nicer things about the wines you review?”. In this case, it was not just the points that were a problem but the fact that we were not cheerleading for every wine that comes across the transom. Never mind that we might have liked the wines, we were mentioning the downsides in some wines we were recommending. That is what good criticism is supposed to be all about. It is supposed to tell the whole truth as the critic sees it, not just the good part of the truth.

That retailer was not alone in his reaction to our words. The idea that CGCW was not sufficiently leading the sales charge has been voiced over and over again by wineries that like us. The ones that do not like us no longer talk to us. That list is long and getting longer. I tell the wineries that I don’t care what they think. We buy a lot of the wines we taste so when a winery like Iron Horse will not even return phone calls, we are happy to buy its wines and review them anyhow.

Still, before I get too far afield, I need to come back to writers. Recently, the exceptional wine blog, Fermentation, authored by the inestimable Tom Wark, toted up the percentages of California Pinots getting 90 points or better from Robert Parker. For 2009, admittedly a fine vintage, 78% of all wines rated in the 90s. That is an astonishing percentage. Good for California.

But, folks, that kind of analysis turns out to be no analysis at all. When everything is equal, there is no differentiation upon which consumers can rely. We have heard the same thing said before. How was Parker going to be able to rate the 2009 Bordeaux vintage when he had been averaging in the mid-90s for earlier vintages this decade. There were a few tongue-in-cheek comments that he would finally break the 100-point barrier. He didn’t. But, how does one choose between 94 and 96 when so many wines earn those ratings.

I don’t mean to pick on Mr. Parker or his crony in grade inflation, Jay Miller, or Harvey Steiman or Jim Laube or anyone else. Wine is better than ever before, and there is truly less flawed wine out there on the hustings. But when winewriting begins to call everything “great”, it is also saying that everything is average. And I don’t believe it for a minute. There are always wines that are head and shoulders better than their peers. With grade inflation, however, those beauties cannot stand out like they should.

I disagree with the folks who would do away with the 100-point system. It would only get replaced by some other system of symbolic notation. But, it does seem that there is something wrong when all the wines are rated “great”. They simply cannot be. There are always standouts. And it is time for the industry and the consumers to insist that those superstars be allowed to have some breathing room between their ratings and those afforded to wine in the big middle ground.


The CGCW Experience - Take the Tour

Meet the New CGCW

For thirty-five years, Connoisseurs’ Guide has been the authoritative voice of the California wine consumer. With readers in all fifty states and twenty foreign countries, the Guide is valued by wine lovers everywhere for its honesty and for it strong adherence to the principles of transparency, unbiased, hard-hitting opinions. Now, it is becoming the California winelover’s most powerful online voice as well. And, our new features provide an unmatched array of advice and information for aficionados of every stripe.

Comments

you're putting the horse before the cart
by SUAMW
Posted on:11/18/2011 2:26:20 PM

Not only is grade infaltion ruinin rtings, it is runing wine.

Now, if only we had critics that really understood wine, had good sensory acutity and combined that with the understanding of what organoleptic components come from grape, wood, yeast as well bad decisions in farming and elevage AND if those critics then rated not on impact or which wine stands out the most out of a line up but on pre-determined criteria rooted in the character of the grape, followed by region, then winemaking and finally vintage, you would not see any creep in scores.

But that would require that critics 1) had good sensory skills, 2) bothered to really understand wine rather than inpose their worldviews and misunderstandings of nature and their own physiology on the process of wine evaluation and 3) grew a spine and (after taking the time to really learn and understand wine) set down pre-determined criteria.

 

But, it's easier to sip and spit out (both the wine and meaning less scores).

No Subject
by Andrew
Posted on:11/20/2011 4:26:07 AM

Or.......people can just enjoy the wine that suits their own palette, quit being sheep, and let discovery be the wind that sends their discovery to shore

Inflation
by Terry Rooney
Posted on:11/20/2011 4:41:11 PM

I have always thought that Harvey Steiman was a "grade inflator." I like Oregon Pinot but his scores are on average about 2 or 3 points higher than Jim Laube's.

It'a like in Zagat where Sacramento now has 4 restaurants with a 29 score. I live here and that is nonsense. Nothing close to a real 29, which I would give to Per Se in NYC and very few others. Certainly not Manresa (right, Charlie).

But, in theory, why shouldn't we have lots of great wines, with scores in the 90's.  Grade inflation is everywhere (SAT, high scool grades, etc.).  Why don't we just use a percentile score, and that would fix things.  This wine is in the 95th percentile of Pinot Noirs.  Oh well, I'll be dead when that happens.

Terry Rooney (scientist who hates grade inflation)

 

 

Rating Inflation
by Don Potter
Posted on:11/22/2011 3:48:32 PM

Please don't change your wine rating methodology. My limited wine-buying budget is mostly spent based on your strict and (yes) opinionated rankings of the great, very good, average, and below average wines you taste.  My buying decisions are based primarily on the number of puffs and your written comments.  The 100-point scale score is a tie-breaker.  A "four-puff" system (3, 2, 1, and 0) helps me spend my wine dollars on those wines with the highest quality-to-price ratios.  

Puff Inflation
by Charlie Olken
Posted on:11/22/2011 4:20:48 PM

Don--

Maybe we should add a fourth and a fifth puff (star in our own lexicon).

I love your comment about points. At the level at which points are used, the gradatiions are such that a small difference doe snto mean much. It does mean we prefer one wine over another so treating points as tie-breakers makes all the sense in the world.

I

Leave a comment below, but please limit your comments to 1,200 characters or less. We find it helpful to make a copy of our comments to be sure that they fit. In that way, you can edit them if they run long.

(Please note: your e-mail address will not be visible after posting)

Name
Email
Subject

 

Note: Refresh your browser to see your latest comments.

Having technical problems with the comment system? Click here.