Measuring Information Architecture Panel at CHI 2001

Measuring Information Architecture Quality

Panelists: Lou Rosenfeld (co-moderator), Keith Instone (co-moderator), Gary Marchionini, Marti Hearst, Jesse James Garrett, Shiraz Cupala, Nick Ragouzis

ACM SIGCHI 2001, April 2001, Seattle, Washington

Panelist Position: Nick Ragouzis (August 30, 2000)

See an extended version of the presentation: extended.ppt

Panel Reference Site:


Quantification seems an important goal.

After all, the progress of science depends on quantification. Or does it?

An ethnographer observing the development of these two cultures, the HCI research community and the application-focused design community, might conclude that the overall system is driven merely by evolution dynamics: loosely-executed mimicry of popular practices and chance mutations arbitrated by the caprice and indifference (and the plasticity and tolerance) of their customers.

One familiar with HCI research could readily conclude that those in applied practices rarely know of even established research results (let alone more recent results). They rarely have a method for evaluating that research (especially in disambiguating popularized applied-domain "findings" from well-executed research) and integrating it into practice. They are equally slack in pursuing research in other domains and reconciling it with HCI research. They almost never credibly analyze their implementations.

Even more disconcerting is this. While HCI reinforces its hallowed stanchion, the human-as-computer operating in the task domain, design practices build ever-more-heavily over the weakest pier of HCI's foundation. That weakness was encapsulated by Card, Moran, and Newell, in TPoHCI, in their then-unanswered questions regarding extensions into the realm of non-instruction-following tasks (Q16) and the role of an applied psychology (Q18). This capsule remains mostly buried, with those in the application domain unaware of their immanent peril.

However, with the rise of interaction design and information architecture, and the overt intention of delighting end users even while making their lives easier, the design community has continued their push into the experience domain. Over decades, without a credible basis for defining or measuring the whole of human experience, they have garnered an astounding quantity of successes.

One could conclude that success in this domain requires only the ability to innovate or to follow strategically, and the ability to deliver user-perceptible value. Which is another view of science: that quantification merely follows, but that science (especially the social sciences) proceeds through innovation and serendipity in theory and application, and by the delivery of ultimate value. Decided:

Abandon quantification; and may the fittest win.

Nick Ragouzis
August 30, 2000 position.htm