1 Comment

Two recent experiences have reminded me of an old and very important lesson about audio I and many thousands of other audio enthusiasts learned decades ago. But the lesson seems mostly forgotten. The first experience was a couple of “run-ins” with other audio writers -- one directly, on Facebook, and the other indirectly, through an editor friend of mine. The second was recording and mixing I’ve done for what will soon be my first jazz album: a collaboration with saxophonist Ron Cyger under the name Take 2.

Both of the writers in question have at least 20 years of reviewing experience. Both would, I’m sure, tell you that their ears are their ultimate reference. Both of them rely on the most basic, “single presentation” method of product evaluation: they replace a component in their reference system with the product under test, and they describe the sonic differences they hear. They claim to “know what to listen for.” They believe their conclusions from these evaluations to be correct, comprehensive, and reliably predictive of other audio enthusiasts’ reactions to the product in question.

Bass and microphone

The problem, as I was reminded during my recording and mixing sessions, is that whether one person likes the sound of an audio product is not a reliable indicator of the performance of the product, because that person has only a foggy idea of what the music they’re listening to is supposed to sound like. Nor is their opinion a dependable predictor of others’ opinions.

As I’ve been recording and mixing, I’ve been struck by how many routine decisions I make that have huge impacts on the sound yet are made entirely according to my personal taste, with at best a vague acoustical memory of an instrument as a reference. For this project, I usually lay down the rhythm tracks on bass, guitar, and/or tenor ukulele, then send them to Ron so he can overdub the sax and flute parts. As you can see in the accompanying photo, I sometimes do a lot of EQ on his alto sax tracks. He uses good-quality, large-diaphragm condenser microphones, and presumably he thinks the recordings are a reasonable representation of what he sounds like. To me, though, some of them sound way too bright, so in those cases I boost the low frequencies, notch out the upper midrange around 1.6kHz, and pull the treble down about 2dB.


Of course, Ron’s reference for what he should sound like is his own live playing, which his ears always encounter from their position an inch or two above the saxophone mouthpiece. My reference is also Ron’s live playing -- which I usually hear from six feet behind him, with his saxophone pointed toward the audience, reverberating through the space, and clouded by the competing sounds of bass and keyboard amps booming at my feet and a drummer spang-a-langing away on a ride cymbal an arm’s length away from me.

Obviously, neither of us can be sure what’s the most accurate portrayal of Ron’s alto sound; all we know is what we like. So how could someone listening to our future album be sure what it’s supposed to sound like?

You might think that I, as a double bass player, would know how a double bass sounds. But not only do the basses themselves sound different; the strings (which could be steel, synthetic fiber, natural gut, or a combination of any of these) can sound even more different, and the bassist’s technique can play a still-larger role in the character of the sound.

And then, of course, there’s the choice of microphone to record with. I, like many double bass players, prefer the Electro-Voice RE20 -- a large-diaphragm dynamic microphone designed for radio announcers, not instruments -- but I know bass players who’ve done excellent recordings with large-diaphragm condenser microphones such as the AKG C414, small-diaphragm dynamics such as the Shure Beta 56A, and many other microphones. All of these microphones sound different, and recordings from them are always EQ’d to the taste of the recording engineer, the producer, or the bass player -- and of course, they make those EQ decisions using monitor speakers or headphones that have a different frequency response from the ones you use.

MicrophonesShure Beta 56A, Electro-Voice RE20, and MXL 990

To illustrate this, I simultaneously recorded a pizzicato passage on my bass using three different microphones: the RE20 and Beta 56A, plus an MXL 990 large-diaphragm condenser microphone. I placed them close together, matched their levels at 400Hz using a sine wave played from a Bluetooth speaker, and recorded all three at the same time. I then imported the files into Room EQ Wizard and ran a spectrum analysis on them.

You can see in the accompanying chart below that the Electro-Voice RE20 is similar to the MXL 990, but with a few dB less response below 100Hz, which to my ears gave it a somewhat less boomy sound. But the Shure Beta 56A -- which I found out about from Lynn Seaton, one of the world’s best-known jazz bass teachers, who uses it for his recordings -- has a much more trebly tonal balance, with more apparent treble detail but weak low-bass response. You could EQ these microphones to sound more similar, but it’s unlikely you could get them to sound the same.


The bottom line is that there are so many variables in any recording that it’s difficult for even the most experienced listener to be sure what’s right. Only if there’s a glaring flaw -- if, for example, an audiophile-oriented headphone is bass-deficient (or treble-abundant) enough to mask the fundamental tones of the double bass, or if a bass-heavy mass-market headphone takes all the breathiness and upper harmonics out of a flute recording -- can we be confident in our assessment that the headphones are doing something wrong. After all, those sonic characteristics are part of the reason that musicians and producers choose those instruments in the first place.

This lesson first hit me about 30 years ago when I bought the original Stereophile Test CD, which featured the magazine’s founder, J. Gordon Holt, reading his essay “Why Hi-Fi Experts Disagree.” Slyly, Holt, and the magazine’s then-editor, John Atkinson, recorded the essay employing many different commonly used, well-regarded studio microphones for the recording, changing the microphone every few sentences. The differences in the tone of Holt’s voice and in the level and character of the noise floor every time the microphone is changed are in most cases easy to hear, especially through headphones.

As the recording demonstrates, if every acoustical instrument or voice you hear through your system was captured by a microphone, and you don’t know what that microphone sounds like -- or what that instrument or voice sounds like, or how the venue affects the sound -- how can you be certain what the recording is supposed to sound like?

My intent here is not to suggest that we should ignore what our ears tell us when we listen to music and rely strictly on measurements to evaluate audio products. It’s merely to show the limitations of what music recordings can tell us about the performance of an audio product -- and to illustrate the folly of believing oneself so skilled at listening that one’s opinion of that product is definitive and universal. Anyone who’s conducted a few blind tests can tell you of the chagrin they’ve felt when interviewing test participants and finding that two experienced listeners with healthy hearing can have conflicting opinions of the same product.

But while some of my colleagues in the audio-measurement community may not agree, I feel that we need to hear about that subjective experience to understand audio, even if the subject’s experience isn’t predictive of ours. I’d guess that most audiophiles got into the hobby after witnessing the passion of other audiophiles -- something that’s unlikely to spring from a session at the test bench. The facts of audio come mostly from measurements, but the story of audio can be inspired only by listening.

. . . Brent Butterworth
This email address is being protected from spambots. You need JavaScript enabled to view it.

Say something here...
Log in with ( Sign Up ? )
or post as a guest
People in conversation:
Loading comment... The comment will be refreshed after 00:00.
  • This commment is unpublished.
    Joe Pop · 3 years ago
    • This commment is unpublished.
      Joe Pop · 3 years ago
      Well I can't seem to delete my abbreviated comment, so I'll just continue here. I was going to say that the reason I read or watch subjective reviews of experienced writers/YouTubers is because after awhile I can get some sense of their personal likes and dislikes and weigh their review against that. And if I think they are also aware of, and take into account the limitations of their review, I am more likely to assign some value to the review. That said, I also am firmly in the camp that believes that properly interpreted measurements can tell a quite a bit about the sonic characteristics of a loudspeaker, and that this is very useful information to be considered when considering a purchase.
      A problem I have is when contradictions seemingly arise, and I wonder what is missing from the puzzle. I'm thinking of the PSB Alpha P5 reviews (including by Hans Wetzel for SoundStage!Acces) that have been universally positive. Then another site (known for its dogmatic approach to measurements) uses a Klippel NFS, the measurements look terrible, and the speaker is declared to be seriously flawed. So what gives? Wrong interpretation of graphs? Too much faith put in the measurement system? Or are we all missing something?

      • This commment is unpublished.
        Doug Schneider · 3 years ago
        Brent will have his take, but since I know loudspeaker measurements, was the one who got the PSB P5 measured for us, and also know the site you're talking about where they declared it flawed, I'd like to chime in. I find that on the site you're likely talking about -- AudioScienceReview.com -- it's not that they put too much emphasis on measurements (measurements are important!), it's that they often make poor interpretations of measurements overall, in turn damning products that shouldn't necessarily be damned, but also praising other products too much. It not only takes a lot of measurements to glean how a speaker might sound in a room, it also takes a lot of experience interpreting, which, I'm afraid, many people lack. So, in short, they go overboard on certain things, and miss others.

        • This commment is unpublished.
          Brent Butterworth · 3 years ago
          As strongly as I believe in the value of measurements, I don't think measurements are the end-all, be-all. For example, there are a lot of headphones I and others like that don't match Harman curve, and if you simply declare everything that doesn't match Harman curve to be a "failed product," you're neither exercising nor developing your personal judgment. Science doesn't work that way. I have personally been involved in multiple situations where a manufacturer or publication relied entirely on measurements to judge a product, and they ended up making poor products or poor judgments. And many, perhaps most, audio measurements have little or no proven correlation to user preference. I think for someone who does measurements to have wisdom in interpreting them, they need to have considerable experience in multiple-listener blind testing, so they gain a good understanding of how accurate the science really is. Getting a bunch of listeners to agree is a rare treat.
          • This commment is unpublished.
            Mauro · 3 years ago
            My current belief is that “Comparison” is the keyword for a good and useful review. If you manage and make an effort to have measurements to compare multiple headphones/speakers, a panel of people listening/wearing headphones/using the product in general, songs of different genres.. then you will probably give to your reader and buyer the best advice.
            That’s why I visit this website so much! All checkboxes ticked!
          • This commment is unpublished.
            Joe Pop · 3 years ago
            " I think for someone who does measurements to have wisdom in interpreting them, they need to have considerable experience in multiple-listener blind testing, so they gain a good understanding of how accurate the science really is."

            Exactly why I value your opinion on the subject. I'm not into headphones, but I check in at the beginning and middle of every month to "Solo" to read your reviews and thoughts.
        • This commment is unpublished.
          Joe Pop · 3 years ago
          Thanks for your thoughts Doug. They line up with what I considered to be the most likely explanation for the disparity between the the subjective and objective in this case. As an EE working for FCA (Chrysler) I fully understand the importance of measurements and testing to validate designs. Also, as an old guy with over 30 years experience in the field, I know I'm much better at properly interpreting the data than I was as a fresh faced youngster right out of college. I consider SoundStage! to be the gold standard of audio review sites because of the measurements and experience of the reviewers. Keep up the good work.
          • This commment is unpublished.
            Doug Schneider · 3 years ago
            Thanks for the feedback.

            I've been measuring speakers at NRC for 20 years. During the first 3 years, I didn't pretend to know much. Year's 4 through 10 I thought I knew everything. After year 10, I gained even more knowledge and realized that I didn't know nearly as much as I thought I did. Today, after 20 years, I consider the measurements to be as valuable as I've ever considered them to be -- but I don't try to read too much into them. I suspect that the measurement interpreter(s) at ASR are where I was in year's 4 through 10.


Latest Comments

Nice review. Thanks!
Johny Banks 4 days ago DALI IO-4 Bluetooth Headphones
Brent, great review! Your detailed insights on the DALI IO-4 headphones' sound quality and usability ...
Traffic Riderr 6 days ago Technics EAH-AZ70W True Wireless Earphones
@MauroTraffic Rider is an exhilarating first-person motorcycle racing game that offers an immersive and ...
Jollibee burger price 12 days ago Technics EAH-AZ70W True Wireless Earphones
<a href="https://www.jollibeemenuph.com/">Jollibee Burger Price</a>  the price of a Jollibee burger varies depending on the specific ...
steve page 26 days ago PSB M4U 9 Headphones
Hey thanks my new ones were doing the same I started to think my bones ...
Viwen 27 days ago In-Ear vs. Over-Ear for Travel
I have both. I don't go for the premium brands but the better value ones ...
Phil M. 1 months ago How Terrible Is Temu?
Thank you for your sacrifice.
Ulfilas 1 months ago In-Ear vs. Over-Ear for Travel
I've always had on-ears (rather than over-ears) that fold up. Currently using the AKG N60 ...
William of Occam 2 months ago Bryston BHA-1 Headphone Amplifier
@William of OccamBrent Butterworth would not have condoned this.
William of Occam 2 months ago Bryston BHA-1 Headphone Amplifier
@Doug SchneiderDude posts the same thing on all amp reviews, with good reason: science has no ...