Read time: 13 minutes
I am really enjoying the wealth of information you have written over the years. One thing I can’t find an answer about is how I should go about sorting my beans. I have tried a bunch of different ways and nothing really seems to improve my chocolate every time. I see so many chocolate makers sorting but just can’t figure it out. Please help!!
W: We have a case!
S: No we don’t.
W Yes we do.
S: No, we have clues but the answer is obvious. We don’t have case or even a problem.
W: It can’t be that obvious or the question wouldn’t have been asked.
S: It is elementary. There are over 3 million words devoted to making bean to bar chocolate on Chocolate Alchemy. The Alchemist delves deep into flavors, roasting, refining, tempering and a host of philosophical perambulations on making things and refining the process but nary a word on sorting beans. Why would that be?
W: He forgot?
S: <withering look>
W: <looks around innocently> He…..doesn’t think it is important?
W: But what about the Dandelion book? They say they sort nearly 20% of their beans. And I’ve seen hundreds of photos of people sorting beans. They can’t all be wasting their time and money.
S: <raises eyebrow> And why are there over 4200 religions in the world?
W: <looks around for the subject>
S: Oh come on. Think man!
W: Because no one knows THE answer but are all convinced they are right?
S: And what is one of the best quotes from Abe Lincoln you have found on the internet?
W: <grins> “Don’t believe everything you read on the Internet”
S: Tell me about cannonball balls and a musket balls
W: What drugs are you on man? Speak sense.
S: Do I have to lay everything out for you?
W: Only if you want me to have a clue as to what your drug addled brain is going on about.
S: <sighs> Which one will hit the ground first if dropped together from the same height?
S: And why?
W: The law of gravity.
S And what makes a law a law and not a belief?
W There are no exceptions and the event happens whether the observer believes in it or not.
S And if there IS an exception?
W <slowly smiles> Then it can’t be a law or universal truth. It is just a theory or maybe just a poorly thought out hypothesis or worse yet, an incorrect observation of correlation.
S: Like I said. There is no case.
I’ve tried to make it pretty clear I don’t agree with sorting beans to improve the flavor of your chocolate. This isn’t from a belief. This is from making chocolate for over 15 years and making damn fine chocolate without ever sorting. But it is more than that. It is backed up by the scientific method and a hypothesis. In short it is this.
You can’t tell the flavor of a bean by what it looks like.
And the deeper result of this is this. Even if you could tell what a bean tastes like by its appearance (which you can’t – proof to come), you can’t know the contribution it will make to the overall flavor profile of the chocolate (example to come).
Before we dive into this I need to clarify something. I am NOT talking about not removing rocks, debris, screws, pieces of glass or the occasion doll head. Of course those should come out. I’ve been presented with too many straw man arguments that removing a rock is sorting and therefore sorting is ok. If that is your arguement, stop being willfully ignorant and go away. Moving on.
It is just pure logic. Here is the hypothesis.
People claim systematic bean sorting based on visual clues results in better chocolate.
My counter to that is that I have done this experiment multiple times, with multiple beans, with multiple criteria and in the majority of cases the unsorted chocolate tasted better or in nice looking beans there was no discernible affect at all.
And from a scientific point of view, it only takes ONE piece of verified contradictory piece of data to disprove a hypothesis. I have way more than one piece of data so it doesn’t matter in the least that sorting might work sometimes. It is a bad theory and practice.
Let’s take a step back though. I roast a lot of beans. I taste a lot of beans coming out of the roaster and I am here to tell you there is NO correlation between what a bean looks like and how it tastes. Cracked beans can taste like uncracked beans. Whole and pristine beans can taste utterly nasty. Chipped and partial beans can be bursting with flavor.
Again, all it takes is one piece of data to disprove a theory and I have it in troves.
But let’s keep on. We are going to pretend that you CAN sort by some magical means to get rid of bad tasting beans. Maybe by UV light, or something methodical that consistently removes inferior flavored beans.
I am STILL going to contend that that doesn’t mean you will improve the quality of the chocolate with this example.
There are many spices and flavorings that on their own are not pleasant to eat. Pure cinnamon isn’t great. Cloves are very bitter. Have you ever tasted pure baking soda? It is a great way to show what astringency is like. I can’t stand turmeric on its own. But leave any number of these out of a curry, or cake or the like (because they taste inferior) and suddenly what you are making no longer tastes like you expect it and it certainly isn’t better for the removal that nasty ingredient tasted in isolation.
We are complex creatures. Our senses take in and respond to a whole host of chemicals to create a flavor profile that can’t be readily broken apart into its constituent parts.
My take is you WANT some odd, funky, less than appealing flavors in there because they are going to make a more flavorful and dynamic tasting chocolate.
And I’ve tested it a number of times. I’ve taken beans that could be the poster child for sorting. Broken, doubles, black, flat. I sorted them all out and made three chocolates. One unsorted and each of the good and bad piles. So you don’t have to use your imagination, this is what that looked like. First, the beans as I received them and then the two piles of sorted beans.
Before I continue I need say I don’t advocate beans that look this way. It makes year to year and sometimes batch to batch consistency difficult. This is about whether you should sort or not if you end up with beans like these.
So I made 3 batches of chocolate. This is the spider graph of the unsorted one as delivered.
Pretty amazing huh? Some of you might recognize this. This is the Nigerian I had in 2017 that I rated as one of the top 5 chocolates I’ve ever had. These are the tasting notes:
The chocolate starts off smelling of cinnamon and damson plums. I'm not sure I have tasted a chocolate with more clear chocolate flavor. This is the flavor you grew up with (but better!). The overall impression is super sweet toffee, cinnamon and baked fruits. Fruitcake even. There is a clean bitterness of browned (not brown) sugar that offsets the sweetness with near perfection. The acidity is a middle level malic, soft yet present. Astringency is muted, giving just enough body to round out the flavor. And speaking of flavors, the first time I sampled this there was the unmistakable flavor roasted peanuts (no, no real peanuts). The aftertaste was like a Resse's cup. This time is it a little lighter in the nut but it is still there.
Here are the spider charts of the two sorted piles.
And no, I didn’t mix them up. The pretty clean beans made a basically horrible chocolate whereas the horrible looking beans carried forward the majority of the initial flavors.
To me, the really interesting part is that even with how bad the pretty beans were, when combined with the good tasting but bad looking beans, the overall result was better.
The points that I really want to hammer home are this:
The good looking beans tasted bad
The bad looking beans tasted good
The addition of bad tasting beans made the overall chocolate BETTER.
Now I am not saying this is always the case, but instead I am proving a point that always sorting your beans will not guarantee better chocolate because I have data that proves it isn’t always true. Again, all it takes is one verified data point to disprove a theory and I just gave it.
The thing is I have seen it over and over and it isn’t an isolated case. I’m also in a rare position that I have seen the preparation of cocoa change over the last 15 years as more and more makers are demanding eye candy beans from origin. I am seeing things like double and triple sorted beans so that the product is visually pleasing but I starting to also notice another pattern. There is a general trend that on average chocolate is getting a bit less vibrant and a little more boring. Now this isn’t proof there is correlation but coupled with what I just laid out it seems rather obvious to me that origin sorting is removing many of those odd tasting beans that otherwise would give an extra layer of flavor and complexity.
Just for a quick comparison this is from my latest crop from Tanzania. It is pretty beautiful, yet people still sort things out. “If in doubt throw it out”.
I’m incredibly skeptical that small amount can negatively affect the flavor of your chocolate.
I know I am going against the common tide here but I am really hoping to effect some change BACK to when many more of my bean choices really blew me away. I can’t help being a believer in natural process and selection. This is not saying I want hard to work with beans and that sure, doubles and triples should be removed, but it is from the standpoint of having a fully controlled fermentation process in place that produces an easy to work with product in the first place and not fixing it afterwards with sorting. The former leads to much higher batch to batch consistency whereas the later attempts to hide the fact of poor system consistency.
As you can see, I feel very strongly about this. For me decisions need to be made on a case by case basis and not because sorting worked once or twice so it was decided it must work all the time and that is now the new standard operating procedure. So my suggestion is to follow the scientific method if you are hell bent on sorting.
1. Get some beans.
2. Sort them.
3. Make chocolate from the sorted and unsorted beans.
4. Blind taste the chocolate, scoring them objectively with a something like my spider charts.
5. The next day taste and score them again.
6. And on day 3, do it yet again.
7. Now compare the sorted and unsorted data sets to themselves and calculate the 3 sigma standard deviation on the numbers to evaluate your own abilities.
8. If your grouping is tight (off hand I would hope on a 1-10 scale your standard deviation is less 1) then compare all 6 sets of data and see if the s.d. is larger or not. If it is larger there is some chance the chocolates are actually statistically different and you can make an educated and informed decision about whether the sorting had any effect.
9. If on the other hand your personal s.d. is high it is basically saying you aren’t good enough to be actually making a decision on sorting and therefore you should not sort.
10. Finally if you do find your results are solid, you are good enough and you find the sorted chocolate is better, and it is hard to stress this enough, you need to repeat the whole procedure at least once more and preferable twice more to prove your method of sorting is valid and reproducible for THIS BEAN and it wasn’t just a one time fluke.
I’m not joking here or exaggerating. This is basically the procedure we used in the laboratory for sensory analysis evaluation. It is a brutal procedure but anything less is usually biased or just not representative of the facts. After evaluating yourself 10 times successfully you can leave out the self evaluation but you should still be doing 3 sorting replicates for each bean.
Giving sorted and unsorted chocolate to a few unvetted (see above) people and asking them to score the chocolate on 3-5 point scale and adding up the points to pick the winner is not statically valid even if it sounds like you are being through. I bring this up as it or some similar variation is the most common evaluation I hear about to defend sorting.
You could just not sort. Make the chocolate, decide if you like it on its own merits, as it stands and don’t buy it if either the preparation is just too poor for you or you don’t like the chocolate.
There are so many good beans out there, why spend your valuable time sorting through every single bean when most likely all it is doing is wasting your time, and driving up chocolate prices (in the case of professional makers) due to increased labor costs and waste cost?
Ok, that is my impassioned rant. I think I’ve backed my position with logic and data and hopefully given you some things to think about if you happen to sort. And if you don’t sort I hope you never start. I never have and have built a full and thriving business off that practice. I’ve taught thousands of people to make chocolate this way with so many writing in to say how amazing their chocolate is. If there was a fatal flaw in my method one would think it would have reared its head by now. Right?
Be well all and let’s go make some chocolate.