In October of last year we published an article called “When a 90 is not a 90” which explored how various review entities that utilize the 100 point rating system stacked up over the last 3 years to see what each looked like for average scoring, most frequent scores and etc. to determine how much score inflation was occurring. This article just reported data collected.

A month later, we published a follow up article which included opinions about each of the sites and the results from the data collection.

This article is revisiting the sites a year later and tracking their scores over that year to see what changes, if any, have occurred. We’ve also included some new sites that weren’t in the previous articles. These sites are ones that didn’t have a 3 year history of scores, ones that have just started scoring reviews this last year or have just come to our attention. This article will be a mix of collected data and opinion.

Revisiting

Cigar Aficionado

Data: The average score over the last 12 months has been 88.81 compared to 89.12 from the three years prior, so a little more than a quarter point drop. In regards to the most frequent score given over the previous 12 months, the numbers are the same as 89 is still the most frequent with 90 and 88 with just one review deciding second and third place. Cigars still need to score a 91 to start differentiating themselves from the pack in their scoring range.

Opinion: Cigar Aficionado is very consistent in their scoring over time. They are also one of the few review sources that uses a decently spread out scale. Regardless of what you think of their review process or what they review, you can at least take solace in the fact that they keep the scoring consistent.

Blind Man’s Puff

Data: The average score over the last 12 months has been 90.74 compared to 90.22 from the three years prior, so a little more than a half point increase. In regards to the most frequent score given over the previous 12 months, the number is now 91 compared to 90 from the previous three years. The score of 92 was just one review behind the most frequent score and 90 was a couple more behind that. Cigars now need to score a 93 to start differentiating themselves from the pack in their scoring range as compared to a 92 from the previous three year sample.

Opinion: The increase in scoring isn’t much of a surprise. In reading through the reviews, there are frequent times where the descriptions don’t match up with the scores as the numbers are higher than what you anticipate they would be based on the reviewers descriptions of their experiences. The site could use a re-centering of the scoring by revisiting how reviewers submit their scores or re-educating the reviewers on the existing system.

Cigar Coop

Data: The average score over the last 12 months has been 90.78 compared to 90.84 from the three years prior, so an ever so slight decrease. In regards to the most frequent score given over the previous 12 months, the number remains at 91 from the previous three years. The score of 90 remains as the second highest given as well as 92 being the third. Cigars continue to need to score a 93 to start differentiating themselves from the pack.

Opinion: The site made some modifications to its scoring system for this year, but it doesn’t appear to have changed much. It still uses a vary narrow 5 point scoring range. It would be good to see the system revamped to broaden the range of scores given which would hopefully also lower the average score.

Cigar Dojo

Data: The average score over the last 12 months has been 88.95 compared to 90.22 from the three years prior, so a very large decrease of just over a point and a quarter. In regards to the most frequent score given over the previous 12 months, the number 89 as compared to 90 from the previous three years. All of the scores seem to have shifted back one as 91 is the second most given score compared to 92 previously and for third most, 90 and 86 are tied compared to 91 previously. Cigars now need to score a 93 to start differentiating themselves from the pack compared to 94 from previous years.

Opinion: There has been some changes in regards to the reviewers that are contributing to the site, and it looks like they are a bit more discerning in their ratings. It’s good to see a reduction in the inflation and would need to be something to keep an eye on to see if the site maintains this trend.

Halfwheel

Data: The average score over the last 12 months has been 86.62 compared to 86.69 from the three years prior, so a very slight decrease of just seven hundredths of a point. In regards to the most frequent score given over the previous 12 months, the number 88 is the same as the previous three years. Something that is interesting is the second most frequent score in the past 12 months was 90 and it was just one review shy of the most frequent score, which is a bit of a deviation from the previous years where 87 was the second most frequent score given. Cigars now need to score a 93 to start differentiating themselves from the pack compared to 92 from previous years.

Opinion: In regards to the average score, the site has been very consistent compared to the previous three. There does seem to be an uptick in cigars scoring higher as well as an uptick in some scoring very low which is how the average score has maintained its consistency. It would be interesting to keep an eye on scoring into the future to see what directions things go such as getting more in line with historical scoring or trending to a slightly higher frequency of scores above 88.

The Cigar Authority

Data: The average score over the last 12 months has been 92.22 compared to 91.57 from the three years prior, so an increase of nearly three quarters of a point. In regards to the most frequent score given over the previous 12 months, the number 92 is the same as the previous three years. What’s interesting is that the second and third most frequent score are now 93 and 94 which are on the higher end of the most frequent score as compared to 91 and 90 which were second and third from the previous three years. Cigars now need to score a 95 to start differentiating themselves from the pack compared to 94 from previous years.

Opinion: As a site that uses the reviews to promote selling the cigars from their stores, you can see where a continuing inflation of scores would be beneficial. At this pace, there won’t be much room for scoring high if the inflation continues. There will then be two options. Re-calibrate the scoring to bring it back down or extend the scoring scale past 100. As silly as pushing past 100 seems, in a culture of those always wanting more, someone is going to do it, and why not Barry?

Tiny Tim’s Cigar World

Data: The average score over the last 12 months has been 90.28 compared to 91.69 from the three years prior, so a large decrease of almost a point and a half. In regards to the most frequent score given over the previous 12 months, the number is 93 as compared to 92 from the previous three years. The second and third most frequent scores are now 91 and 92 as compared to 91 and 93 previously. Cigars now need to score a 95 to start differentiating themselves from the pack compared to 94 from previous years.

Opinion: Shortly after my original scoring article was published, Tim chose to revamp his scoring system. The results are very interesting as the average score has dropped but the most frequent score given has increased. This can be explained by the range of score being given widening greatly from about a 6 point range to a 10 point range. Other than the most frequent given scores, all of the other scores are pretty evenly distributed. This doesn’t make much logical sense for how scoring systems would work, but I’m interested to see how the numbers look over time. I applaud Tim for looking into his system and making the changes he did.

Overall

Comparing average scoring of all of the sites between their most recent scoring and previous three years, five sites reduced their average and two sites increased their average. The decreases for Cigar Aficionado and halfwheel I think can just be chalked up to normal variations in scoring over time. Cigar Dojo’s decrease is directly related to changes in reviewers. Cigar Coop and Tiny Tim both made changes to their scoring systems but had very different results in regards to how much decrease it effected. As for Blind Man’s Puff and The Cigar Authority, they had fairly significant increases and I feel they both need to take the first step and admit they have a problem.

Comparing most frequent scores given from all of the sites between their most recent scoring and previous three years, four sites remained the same, two sites increased and one site decreased. Cigar Aficionado, halfwheel and Cigar Coop don’t really surprise me in regards to maintaining their frequent scores based on how close they were to their previous averages, but The Cigar Authority remaining the same was a bit surprising due to the average increasing. It just turns out that the scores higher than the most frequent saw increases. Cigar Dojo lowering their most frequent made sense based on their drastic drop in average. Blind Man’s Puff increasing makes sense since their average increased. Tiny Tim going up when his average dropped was a bit surprising, but I think is a product of how he setup his new scoring system and may be something he wants to revisit to see if that’s how he wants the numbers to shake out.

Newly Analyzed Sites

In the original article, I only included media sources that had a three year history of scoring to be able to work with a large amount of data. Now that I have been revisiting the sites, I am also working with a smaller set of data, so I wanted to now include those sites that didn’t qualify for the original article but have enough data that I can present some findings and opinion.

Cigar Smoke

Data: Cigar Smoke is a site that appears to consist of two reviewers and started in the middle of 2016. All of the reviews are from a single person and they alternate on who reviews which cigar. The average score across 171 reviews is 91.47. There were 35 reviews that received a 93, 27 were given a 92 and 23 were given a 94. With the bulk of reviews scoring between a 90 and 94, cigars need to score 95 or higher to start differentiating themselves from the pack. On both sides of the most frequent ranking of 93, they produce scores on the lower side 73% more than they do on the higher side.

The average score over the last 12 months has been 92.79 compared to 91.47 overall, so the recent scoring has seen quite an increase compared to the overall data set. In regards to the most frequent score given over the last 12 months, the number is 93 which falls in line with the overall numbers. The second and third most frequent scores are now 94 and 92 which are the same as the full data set, just in reverse order. Cigars still need to score a 95 to start differentiating themselves from the pack.

Opinion: This is definitely a site working on the higher side of scoring.The scoring explanation on the site just breaks down the scoring range and says that scores that are 90-95 are great which is pretty much 77% of all the reviews they have done. There isn’t any explanation in to how they derive their scores, so I can’t say what changes may help bring scoring down.

Have A Premium Cigar

Data: Have A Premium Cigar is a site that appears to have been around for a few years and has had a few reviewers but can’t say what the current staff looks like. There has also been some inconsistency in the review schedule. The average score across 105 reviews is 91.31. There were 21 reviews that received a 90 and 20 were given a 95. With the distribution of scores, I’m unable to determine what score a cigar would need to differentiate itself from the pack. On both sides of the most frequent ranking of 90, they produce scores on the higher side 66% more than they do on the lower side.

The average score over the last 12 months has been 91.89 compared to 91.31 overall, so the recent scoring has been a slight increase compared to the overall data set. In regards to the most frequent score given over the last 12 months, the number is 95 with the second most being 90 which is a reversal of those two numbers from the overall data set. As before, there is no way to determine what score is needed to differentiate itself from the pack.

Opinion: This site came to my attention after a few manufacturers/brand owners were sharing scores that had been given from the site. Just look at the graph of the scores and you can see what kind of disaster exists here. This site would be a brand owners dream as you’re likely to get a very good score here. I’m not able to find any info on a scoring range or system, so I wouldn’t be able to provide any insight into how something like this might even be analyzed.

Stogie Press

Data: This is a site that has been around for a while but just started adding scores to their reviews in July of 2018. I believe that there are a couple of reviewers as part of the site, but each review is just from one reviewer at a time. The average score across 31 reviews is 92.48. There were 7 reviews that received a 91 and 4 were given a 96. With the distribution of scores, it’s hard to determine where a cigar needs to score to start differentiating themselves from the pack. On both sides of the most frequent ranking of 91, they produce scores on the higher side 71% more than they do on the lower side.

Opinion: It’s still very early on in the data for this site, but it’s pretty clear so far that scoring is quite high. A nice thing about this site is that it is very transparent about its scoring system and how it works. In my opinion, the design is flawed. Cigars begin with a score of 96 and there is a very long list of possible deductions as well as a few things that could earn bonus points. With all of that said, and with all of the possible deductions available, cigars are still scoring consistently high. This scoring system weighs construction infinitely higher than flavor. I would hope that at some point this scoring system is revisited and modified to be more flavor based and reduce score bloat.

Overall Comparison

Now that some sites have been added to the analysis, we have enough data to compare them all side by side for their results over the past 12 months. This an interesting look at the majority of review sources that utilize the 100 point system (aside from a few magazines that don’t have scoring available online) and how they stack up against each other in terms of average scores and most frequent scores given.

The sites are ranked from left to right in terms of having lower average scores and lower number for the most frequent score given. The sources that have either given in to inflation or have poor scoring systems are on the right while the sites that have a more consistent or grounded scoring system are on the left.

Cigar Editorial: When is a 90 not a 90 – 1 Year Later