Do You Hear What I Hear?


By Steve Oppenheimer

(December 2002)

It is a tenet of free societies that everyone is entitled to their opinion. At least, that’s the theory. And, understandably, most people have especially strong opinions about product reviews. In general, manufacturers love positive reviews and resent critical ones. After all, nobody likes to be told that their baby is ugly. In contrast, some readers are suspicious of positive reviews, especially if they have heard stories about alleged problems that were not reported in a review. For EM‘s editors, producing reviews is just part of a day’s work. We look for a balance of thoroughness, technical accuracy, clear writing, and well-considered opinions about things that have practical meaning to the end user. We have our own views, of course, but we try not to let our emotions sway us.

Our reviews are field tests by an array of veteran producers, all of whom work in personal studios. We turn to bench tests only to resolve factual disputes. Thorough field tests take time, and we review a lot of products; so although we aren’t always the first to print a review, we try to publish quality articles that are worth waiting for.

Of course, a magazine isn’t the only place you can read a product review; you can get opinions in an online tech forum, for instance. But you don’t necessarily know whether those opinions are valid, because nobody is qualifying the reviewers.

We at EM have assembled an editorial team that understands the products and knows how to evaluate the evaluators. We check out reviewers carefully, making sure they understand the relevant technologies, test products on the appropriate sorts of projects, make practical and fair-minded decisions about how products should be tested and judged, and are unafraid to reevaluate initial impressions. We challenge our reviewers’ assumptions and ask them to retest when we think their results are unclear or unreliable. In short, we are involved at every stage of the review.

We select products that we know will be useful to our readers. Once we start to review a product, however, we finish the job and publish it, positive or not, unless the product is discontinued or we discover that we picked an unqualified reviewer, in which case, we start over with another author.

We insist that our reviewers test thoroughly and that problems be reported honestly. We tell you how we test, under what circumstances problems appear, whether the problems are significant, and, if so, why and for what type of user. If we like the product, we tell you why we like it and for what applications it especially shines.

When all’s said and done, though, the reviews still come down to the opinions of the authors, which reflect how they work in the studio, their experience, their particular production philosophies, their individual research methodologies, and their various writing styles. Two reviewers, given the same product, won’t necessarily agree about its merits and faults. We use a consistent stable of authors, so after a while you get to know their predilections.

Hopefully you find EM‘s reviews helpful when deciding which products to check out for yourself. We work hard to get you off to a good start, but it’s your money that’s at stake, and ultimately it’s up to you to decide whether you hear what we hear.