From BELIV 2010

Presentation

Notes

- insight method to find out what users really learned from vis, against benchmark task to prescribe what users should learn from vis

- for some tasks, benchmark tasks are done well, but not getting insights, but not when the tasks were not done well but people could still find insights

- tasks done in insight tests, didn't test in benchmark (e.g., nodes that are unlike any other)

- subjectivity associated: task methods--choice of the tasks (ecological validity threat); insight method--coding the insights (repeatability threat)

- benchmark--supports; insight--promoted

- controlled ethnography: rich insight in their process and quantify them; possible to add intervention with large-scale displays

q and a:

- juxtapose: more of a continuum than a juxtaposition, don't you need support before promote; explaining why there is blank box

- give them tasks, think aloud to solve the task, with free exploration phase to combine both methodology; sounds good, but be careful, bias insight (cannot count insight, but can tell how to improve vis) since you told them what to look for, reverse influences time;

- simple abstracts can be supported but not promoted, only safer to do in benchmark tasks? ok...

- little concerned about promoting tasks, intelligence community does not adapt vis in rapid pace, how far you do to promote a task but the world and the data change with time (and not a new tool), promote is good, but does it deter me from doing other things; granularity is low, not so much differences with time

- questions about if people are going to do supported tasks because they are not easy (e.g., finding topology)? need to define what is "easy" as the task is "fast and accurate", need different definition