Not Signed-In
Which clippings match 'User-based Evaluation' keyword pg.1 of 1
23 OCTOBER 2014

Describing social and material interactions through formal methods

"To some extent, Formal Methods sit uneasily within interaction design. Human beings are rich, complex, nuanced, engaged in subtle and skilful social and material interactions; reducing this to any sort of formal description seems at best simplistic. And yet that is precisely what we have to do once we create any sort of digital system: whether an iPhone or an elevator, Angry Birds or Facebook, software is embedded in our lives. However much we design devices and products to meet users' needs or enrich their experiences of life, still the software inside is driven by the soulless, precise, and largely deterministic logic of code. If you work with computers, you necessarily work with formalism.

Formal Methods sit in this difficult nexus between logic and life, precision and passion, both highlighting the contradictions inherent in interaction design and offering tools and techniques to help understand and resolve them.

In fact, anyone engaged in interaction design is likely to have used some kind of formal representation, most commonly some sort of arrow and sketch diagram showing screens/pages in an application and the movements between them. While there are many more complex formal notations and methods, these simple networks of screens and links demonstrate the essence of a formal representation. Always, some things are reduced or ignored (the precise contents of screens), whilst others are captured more faithfully (the pattern of links between them). This enables us to focus on certain aspects and understand or analyse those aspects using the representation itself (for example notice that there are some very long interaction paths to quite critical screens)."

(Alan J. Dix, 2013)

Dix, Alan J. (2013): Formal Methods. In: Soegaard, Mads and Dam, Rikke Friis (eds.). "The Encyclopedia of Human–Computer Interaction, 2nd Ed.". Aarhus, Denmark: The Interaction Design Foundation. Available online at https://www.interaction–


abstract system models • Alan Dix • arrow and sketch diagram • context awareness • context-aware interfaces • design methods • design products • deterministic logic • dialogue models • digital devices • digital interactions • digital system • executable models • formal abstraction • formal analysis • formal description • formal design methods • formal methods • formal notation • formal representations • formalised principleshuman-computer interactioninteraction designInteraction Design Foundation • material interactions • notation • physical context • physical interactionphysigrams • product design process • product development methodologyrepresentationrich descriptionsrich user experienceshaping our relationship to the material worldsocial interactionssoftware modellingspace syntax • specification language • state machines • state transition network • structured approach • system behaviour • tangible interfacestechnology affordancesusability testinguser experienceuser-based evaluationworld around us • world representations


Simon Perkins
20 AUGUST 2013

An improved method of studying the user/search process in user-system interactions

"A major 'user/search process' limitation identified by Kinsella and Bryant (1987) is the inability to isolate and characterise individual users of on–line systems in order to describe the pattern of their use. Users' perceptions of their searches are not recorded, transaction logs cannot measure the information needs that users' are unable to express in their search statements (input), and they cannot reflect users' satisfaction with search results (output). As Kurth states, '[the fact] that transaction logs are unable to address such cognitive aspects of on–line searching behaviour is a true limit of the methodology' (Kurth, 1993: 100). Supplementary research, such as questionnaires, protocol analysis and interviews, must be undertaken in order to build a fuller picture of searching behaviour, success and satisfaction."

(Griffiths, J. R., R. J. Hartley, et al., 2002)

Jillian R. Griffiths, R.J. Hartley and Jonathan P. Willson. (2002). "An improved method of studying user–system interaction by combining transaction log analysis and protocol analysis." Information Research 7(4).


2002 • characterising users • cognitive actionsdata collectiondata gathering instruments • electronic information resources • end user studiesend-users • information needs • Information Research (journal) • information system evaluation • information-seeking • information-seeking behaviourinterview (research method) • Janet Kinsella • Jillian Griffiths • Jonathan Willson • limitations of quantitative methodologies • Martin Kurth • online systems • open access journalpatterns of usepeer-reviewed journal • Philip Bryant • protocol analysisqualitative dataquestionnaire • Richard Hartley • search and retrieval • search behaviour • search logging • search process • search results • search results satisfaction • search statements • searchersearching and browsing • searching behaviour • searching for information • success and satisfaction • supplementary research • system requirements • talk-aloud comments • think aloud (research method)transaction log analysis • transaction logging • transaction logging datatransaction logsusability testing • user search process • user-based evaluation • user-system interaction


Simon Perkins
18 AUGUST 2013

Thinking aloud: a method for systematically collecting and analysing data about the design process

"Suppose that you want to understand the design process of architects, the knowledge that they use, the cognitive actions that they take and the strategies they employ. How would you go about this? One obvious possibility is to ask some architects how they design a building. Interestingly enough, they will not find this an easy question to answer. They are used to do their job, not to explain it. If they do try to tell you how they go about their design work, it is quite possible that their account of it will be incomplete or even incorrect, because they construct this account from memory. They may be inclined to describe the design process neatly in terms of the formal design methods that they acquired during their professional training, whereas the real design process deviates from these methods. Psychologists have demonstrated that such accounts are not very reliable. Another possibility is to look at the architects' designs and at their intermediate sketches. However, now you are looking at the products of the thought processes of these architects, and not at the thought processes themselves. What is needed are more direct data on the ongoing thinking processes during working on a design. If you want to know how they arrive at their designs, what they think, what is difficult for them and what is easy, how they reconcile conflicting demands, a different research method is needed.

A good method in this situation is to ask architects to work on a design and to instruct them to think aloud. What they say is recorded and used as data for analysis of the design process. This is a very direct method to gain insight in the knowledge and methods of human problem–solving. The speech and writings are called spoken and written protocols. In this book we will describe a method for systematically collecting and analysing such think aloud protocols. This method can be used by psychologists and other social scientists who want to know more about cognitive processes. It is also an important method for knowledge engineers whose goal is to build a knowledgebased computer system on the basis of human expertise."

(Maarten W. van Someren, Yvonne F. Barnard, et al., 1994, pp.1–2)

Maarten W. van Someren, Yvonne F. Barnard and Jacobijn A.C. Sandberg. (1994). "The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes".


academic researchanalysing dataarchitectural thoughtcognitive actionscognitive processescognitive psychologycognitive sciencecognitive theoriesconceptual modeldata collection and analysisdata collection techniquesdesign knowledgedesign process • design strategies • design workdirect observationexperimental knowledgeformal design methods • human expertise • knowledge engineer • knowledge-based systems • problem-solvingpsychological analysispsychological modelsresearch methodsketching ideas • social scientists • spoken protocols • task analysis • testing theories • theoretical model • think aloud (research method) • think aloud protocols • thinking processthought process • unreliable evidence • user testinguser-based evaluation • written protocols


Simon Perkins
19 JANUARY 2011

The Fountainhead: parodying the absurdity of easy empiricism

Peter: "What do you think of this building? I'm taking a poll of the guests..." Dominique: – "A what?" Peter: "– A poll of opinion about it". Dominique: "What for? In order to find out what you think of it yourself?" Peter: "We have to consider public opinion, don't we?"

[After approaching Dominique Francon at the Enright Building opening – Peter Keating makes the assumption that Dominique Francon shares his faith in polling for deciding the worth of design.]



audience reaction • audience response • ballot • customer satisfaction • DCD • design by committeedesigner-centred design • Dominique Francon • easy empiricismend user studies • group opinion • impromptu straw poll • investment in mediocrity • limitations of quantitative methodologiesmediocrityopinionopinion pollsPatricia Neal • perpetuating mediocrity • Peter Keating • pollingpollspower without responsibilitypublic decision-making • public opinion • reinforcing prejudice • seeking approval • simple evaluations • straw poll • straw vote • testing perpetuates mediocritytesting processThe Fountainheaduninformed opinionuninformed perspectivesunqualified opinionusability testinguser testinguser-based evaluationwhat I reckon


Simon Perkins

to Folksonomy

Can't access your account?

New to Folksonomy?

Sign-Up or learn more.