From BELIV 2010

Presentation

Notes

- existing metrics are not something that dns can buy into (time and error)

- metrics of interest: novice analysts to become expert (users to gain knowledge, insight)

- learning: ask what they have learned using the vis via tests, or ask directly to see if they learned

- pre-test, training, tasks (time and error, logging), post-test (sub preferences). Focus tends to be in users solving tasks using vis

- instead, spend time testing their knowledge, questionnaire or solve a new task (quantitative score), second time around, time and error (look at the change)

- types of learning (1) gaining interface knowledge; (2) gaining knowledge about data; (3) increasing expertise in domain

- look at evaluation from the perspective of a client

q and a: - look at users themselves and what they consider as metrics, look at how vis' users are evaluation (put it in their terms): glazing over how to test learning, focus on testing at the end, may have something more tangible

- new dataset at VAST, reducing the vis process to the people's ability to use their own tools, some people are better at reasoning? current way is similar, but not measuring how fast they solve it. tool has to be mature enough to run another dataset and to solve the problem (reprogramming the tool while trying to solve it). Return of investment is not good.

- interface, system designed by engineer, there is a system model where one has to learn in order to be proficient in the tool, and is more than just interface.