Reviewing the Reviews

steveo_2005

By Steve Oppenheimer

(February 2006)

In our February 2005 issue, we unveiled EM‘s first complete redesign in more than a decade. We strove for a clean, elegant, modern look, and we changed some basic content of the magazine by creating new columns and sacrificing a few older ones. Our huge effort was worth every bit of the time and money we invested, and the redesign process has continued beyond the February 2005 issue; we keep adjusting our fonts, rethinking our covers, and resolving a variety of design and content issues.

One question that we editors have repeatedly and often heatedly debated is what to do about the EM Meters. When we redesigned the Reviews section, we cut back the number of EM Meters for full-length reviews from four to one. (Quick Picks have always had a single meter.) Our goal was to streamline the reviews and to provide a quick and easy visual summary for our readers. But now, a year later, we think that when we spend several pages reviewing a product in detail, we need more than one meter in which to summarize our views.

Beginning with this issue, therefore, we’re returning to four meters for all full-length product reviews. (Quick Picks will still have one Overall meter.) As in previous years, full-length reviews will have meters for Features, Ease of Use, and Value (read: bang for the buck). The fourth meter will usually be Audio Quality, but for synthesizers it will be Quality of Sounds, which accounts for programming quality and sonic accuracy. We’ll use a Documentation meter for products that don’t make sound or for which sonic quality is of minor importance or is impossible to ascertain.

After considerable debate, we decided to keep the five-point meters, but without half points. Starting with this issue, we’ll just have whole-number ratings. Furthermore, at the beginning of the reviews section, we now provide a brief explanation of what the numbers mean. We define them as follows:

5 = Amazing; as good as it gets with current technology

4 = Clearly above average; very desirable

3 = Good, meets expectations

2 = Somewhat disappointing but usable

1 = Unacceptably flawed

At the same time, in our effort to control potential “grade inflation,” we have raised our expectations for what constitutes a good product. If we give a product a “3” for Features, it has the features required to do the job it’s supposed to do, so you should view that as a positive rating. A “3” for Ease of Use indicates that the user experience is satisfactory, albeit there is certainly room for improvement.

We hope that these changes will make the meters more meaningful and useful, and make our reviews even more valuable to you.