Why and how to read survey reports? [Data, Intuition and Truth]

At Pluggd.in, we publish a lot of survey reports – the recent ones being

Do you believe in these numbers?

Actually, you should not.

The Truth

These are survey reports (and not data driven reports). And surveys are mostly done with an agenda in mind. That is, eBay India wants to push its ‘mobile site’ agenda and is trying to build an authority there. Similarly, Google’s smartphone survey (I took the survey when it went live) is an attempt to build authority in mobile-Internet space (which translates to advertisers/publishers etc).

That is, each and every such survey will carry the agenda of a particular company and you shouldn’t really trust the survey report as-is.

Qn: Then why does Pluggd.in shares these numbers?

Answer: Because they represent a trend. Because they act as a source of data (and not the only data source). Unless you are a B-school student who has a presentation deadline, you should not take the data as-is. But you should still read the survey reports.

Why?

Because survey reports are a good source to have hypothesis. They may not validate your hypothesis (and you shouldn’t trust survey reports to validate), but they act as a good data point (must read: Data Driven Decisions Vs. Intuition..Is It Really A “Vs”?).

Let me explain here:

Let’s take a look at the Nielsen’s latest report, i.e. Smartphone Report: Women spend 4X the amount of time men spend on Instant Messaging applications. Let’s take 2 data points:

Report #1. Women spend 4 times the amount of time men spend on Instant Messaging (Chat) applications.

Report #2. Men however, don’t mind asking for directions on their smartphones, accessing the Google Maps app more than women (45% vs 32%).

What inferences can you draw from these?

Report #1. Women are spending 4X the amount of time men spend on IM apps (i.e. WhatsApp ++).

Possible inferences?

Report #1. Smartphone has in a way, added to more chatter – i.e. women are communicating more than just using voice (the report also mentions that women spend 3 hours more on calls every month as compared to men).

Assuming T2010 = time spent in 2010, when IM apps weren’t around. That is, T2010 = Voice_2010 + SMS_2010.

And as per the report, time spend in (communication service) 2012, T2012 = Voice_2012 + IM chat+ SMS_2012

Is T2012 > T 2010? Is Voice_2012 < Voice_2010. Is SMS_2012 << SMS_2010? Is IMChat > SMS_2010 + SMS_2012?

Are women talking more (in 2012) or are spending less on voice and compensating voice with IM chats (or maybe, SMS too).

If that’s the case – is there a need for women-oriented-ergonomic keyboards? OR time to focus on women-centric apps?

That is, you (i.e. you as an entrepreneur) already had some intuition about this business- survey probably provided you more (qualified) data points to explore.

Report #2. Men however, don’t mind asking for directions on their smartphones, accessing the Google Maps app more than women (45% vs 32%).

Of course, this is true and am no different. I mean, I just don’t ask for directions and am okay going round and round. But yeah, I do use Google maps a lot!

Inference#1: I *do ask for directions* from a (smart) machine, but not comfortable asking the same to other *human beings*.

Interesting?

What behavioral inference can we draw from this?

1. Maybe, smartphones have given a comfort level for people to talk to faceless things.

2. Why not think of other things men (+ women) are uncomfortable asking to others?

Are ‘faceless entities’ a good way to provide that comfort (for example, buying lingerie from a retail shop vs. buying online?).

Think!

Long story short, surveys are a good tool to start working on a hypothesis – they provide you a lot of data points. What you do is totally your prerogative – but the worst you can do with such reports is to take them on their face-value or even worst, reject them on their face-value.

Remember, they are a harbinger to an upcoming trend, so use them appropriately (aside, look at Pluggd.in’s trends section).

What’s your opinion?