Sean P. Sullivan - Washington Wine Report

All wines are sampled blind and in a standardized setting

All wines that I review for Wine Enthusiast are sampled blind in a standardized setting (location, time, serving temperature, and stemware). I am in all cases unaware of the producer, appellation, vintage, and price when sampling and reviewing wines. Doing so is an attempt to remove biases that such awareness can cause, both positive and negative.

While I regularly taste wines with winemakers at home and at wineries, these notes are strictly informational. All scores come from bottles sampled blind, in a standardized setting.

There is no screening of samples prior to evaluation

All wines that are part of my beat (currently Washington and Idaho) that are submitted for review are subsequently tasted and reviewed by me personally. No one else screens or reviews my wines under any circumstances.

Unlike some publications or reviewers, there is no screening process for wines prior to being submitted or reviewed at Wine Enthusiast. I am looking at everything that comes through, rather than a subset.

The benefit of this is that it allows me to taste broadly across wines that are part of my beat, providing both myself and consumers with a clearer picture of what is going on in the area. The benefit for wineries is that it eliminates gatekeepers when submitting wines.

Scores for all wines submitted for review are subsequently published*

The benefit for consumers of not screening wines prior to review is that it allows them to see almost all of the scores given to the wines that I taste. If a wine is submitted to me for review, a score is subsequently published for that wine.

When wines are screened, the scores by definition represent a subset of the wines submitted. This means, for the consumer, when you don't see a review for a particular wine, you never know, was it not submitted? Was it submitted and screened out by someone before getting to the reviewer? Could the winery just not get their foot in the door to get someone to even screen the wine?

If a wine was submitted to me for review at Wine Enthusiast, I will taste the wine, and you will subsequently see a score published. There are only two exceptions. The first is if the wine was rated less than 80 points. We consider these wines to not be commercially acceptable. The second is if only one bottle of a wine was submitted and it was determined to be contaminated by TCA (cork taint). These two reasons are why there is an asterisk (*) above.

The process

Receiving submissions 

Submitted wines are received by a professional wine storage facility where they are immediately placed under temperature control. I pick boxes up from this facility typically weekly and subsequently transport them home. Boxes are subsequently left in place for a minimum of one week to recover from shipment, whereupon they are opened and organized by variety/style.

Sets arranged by variety/style

The capsules are stripped from the wines, and the wines are placed in opaque bags prior to evaluation to obscure identifying information. Wines for review are sampled in small sets of four to twelve. Sets are arranged by variety or wine type, such as a Bordeaux-style blend or Cabernet Sauvignon. I do not sort wines by and an unaware of when tasting producer, vintage, appellation, or price, again to remove the possibility of bias that this awareness might bring.

Serving temperature

Serving temperature makes a critical difference in the presentation of a wine. For this reason, all wines for review are tasted at a consistent temperature: 62-66 degrees Fahrenheit for red wines, 50-55 degrees Fahrenheit for white wines. Wine temperature and ambient temperature are recorded.

Small sets and no mass tastings

I keep the number of wines I taste on any given day small, typically from a minimum of eight - the most common number of wines I sample per day - to a maximum of 20, with the latter being rare and typically only done for white wines. I avoid tasting larger numbers per day as I believe doing so is not fair to the wines or their producers.

Tasting large numbers of wines per day can lead to a variety of issues, for example tannin accumulation, that can unduly influence a score. Additionally, when tasting large numbers of wines in a single setting, even the most skilled taster who is faithfully spitting all of the wine they are sampling will be influenced by the effects of alcohol.

I have used a breathalyzer to confirm this during previous (non-Wine Enthusiast) mass tastings I participated in, where I have registered above the legal limit after a morning of large-scale tasting (60+ wines in three hours), even though I was spitting all of the wine. This effect is compounded by a morning and an afternoon of mass tasting (often well over 100 wines). These mass tastings are not uncommon at wine competitions and also professional review tastings at some publications.

During mass tastings, a reviewer's perception for the first wine they taste is obviously vastly different from the 40th or 100th due to the effects of alcohol. I believe this is unfair to all of the wines in question. Tasting in small sets removes the potential effect of substantial alcohol absorption on the subsequent scores that can occur in large scale tastings.

Wines are popped and poured prior to tasting

I believe it is critical that all wines are sampled in a standardized way, as differences in process can impact resulting scores. For this reason, all bottles that I review are popped, poured, and subsequently sampled. As noted below, I make multiple passes tasting a flight of wines, with each pass providing the wine with some air, though obviously not as much as a decant.

I have experimented with decanting wines for review in the past. It was somewhat laborious, double decanting eight to 20 wines. More importantly, not all wines were well served by decanting. Then there is the question of how long: 30 minutes? An hour? Two? Do you use the same decanter for each wine? If so, do you rinse it in between bottles, adding some water to the equation, which can subsequently impact the wine? Or do you use eight to twenty separate decanters?

One might decide to only decant wines that seem like they would benefit from it. However, in my view, this provides the reviewer with too much discretion. Perhaps it is obvious that one wine would benefit from decanting while another might equally benefit but is not as obvious, and so the reviewer does not. This gives the former wine an advantage that the latter does not have.

For these reasons, I have stuck with pop and pour, though I am not entirely satisfied with it. The benefit of this approach is that all wines are treated equally. The disadvantage is that wines that might benefit from decanting are not given that opportunity. This is a sacrifice I am willing to make to have all wines treated equally.

Multiple passes per wine

When tasting wines for review, I typically make more than one pass through the flight, taking independent notes each time. I usually spend 5-10 minutes per wine, tasting the wine and writing a tasting note. These tastings take place over several hours.

While I may taste a wine over several days to get a sense of its overall ageworthiness - and I resample all wines on the second day to look for latent cork taint - any notes beyond the first day when the wine was tasted blind and in a standardized setting do not affect the subsequent score. Rather, they might affect the wine's drinking window if it is a Cellar Selection and perhaps the other special designations listed here.

Evaluating typicity and overall quality

When tasting wines for review, I am evaluating them for typicity and overall quality. Typicity means, for example, does a Washington Merlot taste like a Washington Merlot? If it does - or does not - this may subsequently impact the score or text of the review. For example, if a Washington Merlot smells like a very green Cabernet Franc, this would be considered atypical for the variety and would most likely impact the score.

Quality looks at a number of factors, such as the pleasantness of the aromas and flavors as well as the overall balance, complexity, depth, intensity, finish, and length. I also think about what a wine is trying to accomplish and how well it accomplishes it.

I try as best I can to review wines irrespective of my own personal palate preferences. That is to say, I am trying to assess what the quality of the wine is, not whether I might personally like to drink it at home. This differs considerably from how consumers approach wine.

For example, let's say a Chardonnay is made in a ripe style and fermented in 100% new oak that has an overt, but balanced, influence on the wine. Perhaps I do or do not personally care for that style, but, regardless, if the style is well executed, I would rate the wine highly. For a consumer, if the wine is not in a style that they would personally prefer, they would most likely pass on it and not think highly of it. This is fundamentally different from a critic's approach.

If you have any questions about how I taste wines for review at Wine Enthusiast, please email me at wawinereport@gmail.com.