Objective Truth
How do we know we’re designing right? How do we know our design is good? Is “good enough” for a client the same as “good enough”? Who decides if a design is good? Or bad? Is good-or-bad really the right scale for describing design? How does a company manage design “quality” across a small, large, or distributed staff of designers? What is design “quality”?
These are the questions that have been bugging me for ages. I spent much of this year thinking about training: how do we indoctrinate young designers at my company into our way of thinking about design. But I got stuck when I realized that it’s not just young designers–it’s everyone. What’s the consistent face of design if your design department isn’t structured around a single vision, with no creative director or VP of design? How am I assured that all of the design work that leaves my company is consistently great?
When I first moved into the product design group at my company, there were no tools in place to measure whether the design work we were doing was “right” by any definition. Industrial design has a history of being governed by peer-review-of-coolness: if other designers think it’s cool, it’s good. Many design firms operate that way, and many designers think that way. And sometimes it’s fine. If you’re selling some piece of consumer electronics that will be on the shelf for a couple of months before being replaced by the next model, maybe a design brief that includes words like “cool,” “different,” “iconic” is the right thing. Even if the product isn’t a hit with consumers, it’ll get some press and you’ll have a new product out soon enough.
But for many kinds of products, that’s not a good way to operate. And even though designers these days pride themselves on their research savvy and ability to understand what the consumer likes, wants, and needs, tools that close the loop are scarce. Just shipping the product and seeing what happens is a poor way to test whether your design is any good. So we adopted some tools that were being used at a more strategic level in our consultancy and applied them to products. And although there was some pain in getting the tools adopted by designers (one colleague told me research would “crimp his creativity”), we were ultimately able to show that by evaluating design earlier in the process, we could deliver better work to the client – better defined as “more likely to result in business success.” And, better defined as “supporting our design portfolio” and maybe even “pushing the boundaries of what good product design can be.”
As the design staff gets bigger and more distributed, though, the tools are getting diluted. People are using them in ways never intended, sometimes for the better (that’s how we got them in the first place) but sometimes for worse. Just saying “sure, we applied some design evaluation tools” isn’t good enough. Did you apply the right tools in the right way? I don’t think a “design review” is the right answer, and certainly the “crit” isn’t going to work. The projects we work on are large and complex, and it takes so long to get up to speed that simply commenting on a design is usually useless–the comments have no basis.
I don’t have an answer, but I’m thinking about it. What I envision now is simply a meta-review. Regardless of what you’re designing, is the process you’re using believable? If so, I’m pretty sure the design will be “good” in terms of business, portfolio, and design-as-a-craft. My company already does this on the business side, looking at project budgets and giving project managers a safety net. Developing a similar safety net for the design side is on my list for 2009.
How does your company insure design “quality”? How does your company define design “quality”?
Follow Me
In the medical device arena, design is tied integrally into the functionality of the device: Does the device *work* as designed? This is proven through documented and intense testing and documented proof of the design process.
Good designs have a certain “zen” quality to them in that people just know when it’s right-it’s intuited. But that’s not qualitative.
Again, in the medical realm, user feedback should be (but granted, isn’t always) the barometer by which design quality is judged-and this is obtained frequently and often prior to launch. There isn’t a choice. The regulatory framework demands that design aspects be in place prior to launch. It NEEDS to be the right design, a quality design, as close to launch as possible-not just for competitive reasons but for safety and healthcare reasons.
Hi Michael,
You’re right about functionality – as long as someone has set reasonable, well-defined usability goals for a product, testing whether the design meets those goals is pretty straightforward. Not everyone understands the process, but that’s what design consultants are for, among other things.
My consternation is over aesthetics, and it applies as much to medical devices as consumer products. And aesthetics aren’t particularly regulated, and certainly aren’t seen as part of the functionality – they’re never part of a product requirements document.
Certainly, if you’re a startup with a novel medical device and your product is the only one that does what it does and people need it, it doesn’t much matter what it looks like. But that’s a very small percentage of products. Most medical products must communicate some sort of message and make some sort of emotional statement. Things like diabetes-care devices are basically commodity consumer products at this point. Laboratory equipment must connect with the buyer at the hospital, communicate their value to the technician, look cool on a tour of hospital trustees. So, for the most part, medical products don’t get a free pass these days. They must work as intended, but that’s not sufficient.
Finding a way to characterize aesthetic goals and include them in with product requirements is part of what I’m after.
A
Best
Post
Ever!
Hi Aaron, sorry for being a bit late to the comments party on this one.
A tricky question. First, bottling ‘aesthetics’ is always going to be tough, and there are significant differences between different product categories in their need for different types of aesthetic qualities (and of course our different tastes) – so I don’t think you’ll ever get some quant-y style measures that work along the lines of usability evaluations or ergonomics. Some companies like Google do perform hard core quantitative ‘granular bucket testing’ of changes to the user experience – but these are small, live and highly focused on increasing easily measured metrics.
Back in the physical world, easier perhaps than developing tools for evaluating the look of individual products (which can be done through qual focus groups and other person to person systems) is developing the governance systems for evaluating design language across whole portfolio’s of products.
Setting up standard parameters for aesthetic variation depending on the position of a product on a matrix of similar products makes its easier to move the conversation away from the merits of the individual design decisions, and into the realms of the relative quality of decisions against other known decisions. This of course only works where an organisation makes a range of similar products.
Finally, although I agree with the need to involve users in the design process, its got to be said that you can’t beat a talented, passionate designer working as part of a team with a clear vision of a better world made possible by their designs : )
I’ve heard that Apple (kings of portfolio management and cross channel design language consistency) don’t do user testing, they just build products that they really, really want to use…
Hi Nick,
All agreed. I think part of the issue I have stems from working in a large design consultancy – we usually don’t have control over a client’s entire portfolio of products, which is both hampering and freeing. Hampering in that we often must design within a design language that we didn’t develop–and very often merely evolved over time in uncontrolled ways. Freeing in that we can focus our energy on making *this* product as good as it can be using what we know about doing good design, without the tangle of a large company’s politics (that, we leave to our client).
As far as Apple’s process goes, I think their success comes from self-restraint as much as anything else. Less a matter of including what they should, and more of leaving out what they could.
A