Science Thinking for Design: Validity

Last updated on 3 October 2017

Scientists and designers work to solve specific problems, so why not judge our own work along the same lines?

Validity in Science

Scientists conduct experiments to learn about the world. They design each experiment to measure something specific. There are two kinds of validity to help scientists make sure they design experiments the right way.

  • Internal validity is about how accurately the measured phenomenon is actually measured.
  • External validity is about how accurately the measurement is to answering the real question.

Pavlovian Example

Pavlov's Dogs

For example, Pavlov famously experimented with dogs to understand human psychology. He measured the amounts of saliva the dogs produced, in order to find out if they were being conditioned.

For this example,

  • Internal validity refers to how accurately the amounts of saliva were measured.
  • External validity refers to how relevant the amounts of dog saliva are to human psychology.

Validity in Design

Like scientists, we designers work to solve specific problems.

If all is well, we solve these problems in the context of a larger strategy or bigger picture.

For design,

  • Internal validity refers to how well the designed thing fulfills its purpose.
  • External validity refers to the relevance of that purpose to the greater good of the business and its consumers.

How it Often Goes Wrong

Boyd's Toast
Often, projects are started and judged on their internal validity while completely ignoring external validity. For example, let’s say a toast company wants to get into social media and wants an online marketing campaign to drive traffic to their social media channels. This company however, does not have any social media strategy and uses their social media channels to blandly report on each new product that they release.
So, let’s say the project happens, and the campaign is carried out. Analytics reveal that there was an initial surge in social media followers, but this number again decreased over a short time. In this situation, the project was internally valid in that the campaign drove lots of traffic to the company’s social media channels. It was externally invalid however, because instead of engaging people with well-crafted content, the company just spat out whatever content they have.

Designers  Scientists

Like scientists, designers should be evaluating their work on internal and external validity. We should be using strategy to inform the development of key performance indicators (KPI’s), which are in turn used to design a solution to achieve those KPI’s.

The company in the above example could have built up a loyal and engaged fan base if there would have been someone on the design team, after having received the brief, who asked “why?” It’s at the core of design thinking, and we need to make sure we don’t forget it.

Keep asking “why,” and stay valid, internally and externally!


(This post’s cover image is from JD Hancock)