Risk fixers and sweet spotters

Thomas Torsney-Weir, Shahrzad Afroozeh, Michael Sedlmair, and Torsten Möller

Outline

  • Experiment to assess usefulness of sensitivity analysis
  • Analysis types and user coding
  • Results
  • Discussion

Best investment?

Find an optimum

High level

Low level

Investment interface

Investment interface

Investment interface

Investment interface

Can we improve the decision making?

Can we show the change? Will this help?

?

Sensitivity widget

Which is better?

A: without sensitivity

B: with sensitivity

Study design

Issue is that an open-ended task has no correct answer. There is no “time and errors”

Risk

Return

Metrics

  • Quantitative: confidence
  • Qualitiative: semi-structured interviews
  • Task: Find the best investment

Experiment

ABA design

A1

A2

B

Confidence

Confidence

Confidence

Experiment

  • Interviewed participants
  • Separated based on analysis strategy

Sweet spotter

Risk fixer

Predefined risk value

Where risk changes

Quantitative results

Effect

No effect

Quantitative results

I am positive that I made the best decision given the information at
hand.

Risk fixers

Sweet spotters

Interviews and coding

Is there some quality of the people making this decision?

  • Coded interview questions
  • 2 independent coders
  • Met to resolve differences

Sweet spotter

Risk fixer

  • Code 1
  • Code 2
  • Code 4
  • Code 9
  • Code 1
  • Code 4
  • Code 5
  • Code 9

Decision tree

Code 1=

Code 1=

Code 2=

Code 2=

Code 4=

Code 5=

Code 9=

Code 9=

Code 9=

Decision tree

Decision tree

Outline

Find an optimum

High level

Low level

Best investment?

  • Experiment to assess usefulness of sensitivity analysis
  • Analysis types and user coding
  • Results
  • Discussion

Find an optimum

High level

Low level

Best investment?

Analysis type

Find an optimum

High level

Low level

Best investment?

Analysis type

Risk fixer

Mid-level

Find an optimum

High level

Low level

Best investment?

Analysis type

?

Implications for design

  • Data
  • Users
  • Tasks

Implications for design

  • Data
  • Users
  • Tasks
  • Analysis type
  • Biases
  • Skills

What we did

  • Identified two types of analysis
  • Determined factors to identify these types
  • Future: Can we predict these traits beforehand?

Sweet spotter

Risk fixer

Acknowledgements

Torsten Möller

Michael Sedlmair

Shahrzad Afroozeh

Thank you!

Questions?

thomas.torsney-weir@univie.ac.at

Extra

Good investments

Risk

Return

Good investments

45% AAPL

20% MSFT

35% IBM

10% AAPL

65% MSFT

25% IBM

Risk

Return

Good investments

Efficient frontier

Risk

Return

  • Low-level tasks to interfaces not enough
  • Lab studies with open-ended tasks (ecological validity)
  • Activity-centered design
  • Only after having an interface can people realize the benefits
  • Small features may not affect other users
  • Expanding this interview
    • how can we run this at Turk scale?

Discussion

Who are the risk fixers and sweet spotters?

  • Ran a user study using an investment scenario
  • Found 2 groups of users with different interface preferences
    • Interviews to detect strategy
    • Quantitative assessment of interface preferences