On Wednesday of this week I had the pleasure of attending a best practice workshop on user-centered design. This was held at Philips Research in Eindhoven, and run by a big European project, the European Institute of Innovation and Technology (EIT) ICT Labs. There were nine of us present, which was a good number: few enough that we could realistically sit around a table and talk, yet numerous enough that we came from diverse backgrounds and perspectives. (Similar benefits as the 11 of us who attended Tiree Tech Wave 2 noticed!)

We were indeed diverse: attendees hailed from a range of business and research roles within both industry and academia, and from various EU countries. As an aside, the EIT ICT labs seem very neat in general: their projects range from digital cities and smart energy to the work Lynda is leading at CWI looking at ICT-mediated human activity.

So, the goal of the day was to collect a set of best practices to support people with limited experience in user research, and to work on demonstrating the value of user research in design. The morning involved some presentations about the Philips UX lab and UX labs in general, and the day closed with a tour of the Philips lab. The meat of the day was after lunch, where we spent a good few hours debating various issues in the ‘best practice’ arena.

Overall, I’m pleased about the number and diversity of participants; I reckon we needed more time (a second day?) to discuss things; and I sincerely hope some solid outcomes occur as a result of the workshop. It’s too easy to have interesting and useful discussions, with no tangible outcomes months down the line.

The rest of this post details our discussions. Read on only if you are interested in such things!

The Philips UX lab

The UX lab was presented as a mechanism for evaluating visions, finding out how people might experience design concepts, and gathering insights. We heard about the three types of experience research conducted at Philips (observing people in context; observing people in the lab; doing longitudinal work in the field). There was some discussion of the advantages and disadvantages of coming from industry as opposed to academia, and the tension between researchers (who may look 5 – 10 years ahead) and business types (who may look 3ish years on).

There were neat examples of some products in the Philips lab, including a smart chopping board/weighing scale, built as part of a mindful eating project: unsurprisingly, this put me in mind of the Ambient Kitchen project at Newcastle, which includes similar concepts.

UX labs in general

We discussed how to stimulate usage of experience/living labs, and making user-centered design/evaluation as easy as possible. EIT goals are to collect best practices and an inventory of labs, and to disclose this information: To this end, they’ve made a website called the knowledge centre. Fabulous idea!

I found the ‘learn‘ page, which links to approaches you might use, to be a touch confusing: the distinction between ‘methods’, ‘tools’, ‘techniques’ and ‘sensors’ seems blurred. Part of the problem is probably that there’s such a plethora of possible approaches out there.

It’s early days for the website: I hope the kinks shall be straightened out. Meanwhile, I think it’s a really, really good idea to capture this knowledge and make it accessible. As well as listing techniques, they’re trying to make clear what labs are out there, and what access policies (and pricing structures) such labs have. I was unsurprised to hear that getting such information hasn’t been straightforward: some labs simply don’t have such policies.

Best practices discussion

We debated various questions:

1. How to prove UCD is useful?

This is a dangerous question. Primarily, ‘useful’ is an unhelpful word: what does it mean? Quicker? Cheaper? Creating designs which are more accessible, relevant, fun? Meanwhile, ‘UCD’ is also an unhelpful term: as the knowledge centre website shows, there are many ways to do UCD — in this context, the term is too broad.

We pondered the relative value of qualitative and quantitative data, and their relevance at different points during design (qualitative earlier, quantitative later). We also got onto mixed methods and methodology — for my part, I re-ranted my methodology and methods rant. My main point: mixed methods yield richer and more certain results, but we need to understand methodology as well as methods.

2. How to motivate users to participate in studies?

This struck me as a fairly straightforward topic. We touched on such points as going in with a respectful attitude, asking participants about their needs, and providing them with feedback (both in terms of the architecture of the study and the impact of their data further down the line). Type of motivation (intrinsic or extrinsic) is also relevant, of course: what’s in it for people when they choose to participate in a study?

3. How to make user involvement efficient?

We touched back on use of qualitative and quantitative data to substantiate claims: for instance, a PhD student of Lynda’s ran a study with Google where she validated her quantitative data with qualitative material. We also discussed: technology for diary-keeping; avoiding delays between recruitment and experiments; and reuse of materials (questionnaires, methods etc).

4. How to efficiently recruit participants?

This was interesting to me, as I found it way easier to recruit participants back when I was working at IBM and had access to lots of willing software engineers. People talked about use of social media, incrementally building up a user base, and also reported much better responses to physically flyering people than emailing (this is not surprising).

5. How to deal with ethical and legal issues?

Talk about a broad question. This is especially important when dealing with vulnerable groups, of course, but always key. I did a quick straw poll and was intrigued to find about half of us worked in a context where an ethics committee presides, and half not: what does that say about our profession?

For the record, I am currently in the latter group — there isn’t a formal ethics procedure for me to adhere to at TU/e when carrying out experiments in a non-medical context. I did my doctorate at Southampton, where there was an ethics committee, so I am confident constructing materials such as consent forms and participant information sheets. How do researchers who haven’t worked in such environments cope?

The knowledge base website would benefit from incorporating an ‘ethics’ section: it could include templates for consent forms and participant information sheets, plus simple guidance for practitioners who are working without a formal ethics framework.

6. How can modern technology be used to engage users in the design process and enhance it?

As a technophile, I was intrigued by this question, though as we dealt with it last we spent very little time on it. We touched on the use of prototypes and real-life testing.

One question we didn’t reach concerned the validation of R&D techniques, which I’d have loved to have discussed. One starting point might be to consider the spectrum of research from the designerly / practice-based “research through design” through to academic “design research”.

I reckon we need another workshop to continue these conversations.