Your Data Is Your Lifeblood — Set up the Analytics It Deserves

Looker Co-founder Ben Porterfield talks about why most startups miss the mark on their analytics and provides a formula for nailing it the first time around.

Seriously, Do Not Wait

As soon as your company has users, you need to set up a solid analytics framework. It's not a waste of time or money.

Your impulse might be to save your resources and have your existing engineers cobble together an in-house solution that will do the job, tracking and storing only a sparse set of the most essential metrics. Porterfield has seen a number of companies cave to this temptation and it doesn't end well. It's well worth the time and money to find proven tools that have stability and support from the outset. Here's why:

Everyone on your team benefits from easy access to data.
Making analytics a priority means making it accessible to everyone in your company — not just technical folks. “You want everyone to be able to look at the data and make sense out of it,” says Porterfield. “It should be a value everyone has at your company, especially people interfacing directly with customers. There shouldn’t be any silos where engineers translate the data before handing it over to sales or customer service. That wastes precious time.” The right tools make it easy for anyone in your company to find the information they need and act on it — not just engineering.

When building an analytics framework, a self-service tool for accessing data is key because the people closest to the questions are often the ones running the revenue side of the business. “I’m always going to advocate for self-service tools. Game changing insights don’t always come from the analysts or data science group,” says Porterfield. “They often come from the users who are closest to the problem, understand the product process and know how to formulate the right questions.”

When you embed data directly in the everyday applications where your employees perform most of their work, you’re not just encouraging usage, you’re also creating a data-driven culture, Porterfield says.

From a business operations standpoint, a self-service platform can help IT and data scientists allocate their time in a way that's better for everyone. They can focus more of their attention on the product roadmap or more complex problems.

The right tools eliminate bottlenecks.
“Data teams too often create bottlenecks for the rest of the company. IT shouldn’t be doing the work of librarians, retrieving and interpreting data for those requesting it,” says Porterfield.

Todd Lehr, Senior VP Engineering at Dollar Shave Club shared a relevant story with Porterfield: “We have a developer named Juan and any reports we needed would flow through him. When he got backlogged, we’d call it a ‘Juan Problem’ because our teams didn’t have access to the data immediately."

With a self-service tool, a company will drastically reduce the query queue a developer like Juan needs to manage. Engineers can then prioritize building out the product and bringing things to market faster.

The 6 Mistakes Smart People Make with Their Analytics

Intelligent people mess up their analytics infrastructure all the time. It's not easy to get it right from day one. It takes time that many companies don't think they have. It gets really complicated really soon. After talking to a huge range of companies dealing with these problems, Porterfield has distilled the top six ways companies often mess up their data.

1. You move too fast.
Maybe you're not even looking at data in the first place. A surprising number of companies don't collect and consult data until they absolutely have to.

“Early stage companies tend to have the same mantra, ‘Build, build, build.’ But because startups are in such a hurry most of the time, they don't truly understand engagement — how the product is being used, what parts of it are working, and why your users are coming back,” says Porterfield. “Engagement tells a really important story about how people are responding to a product. Without looking at the data there’s no way to know.”

2. You don't track enough things.
You can't just give your team a snapshot of your top-line metrics or aggregate sales numbers. If you do, without digging into how things have changed over time (even from day to day), you won't spot the underlying forces that are actually making things happen. There won't be any way for you to see how granular changes to the product or trends in the market affected sales or engagement. There’s also no excuse for tracking too little. Data storage is so cheap, that’s not an issue anymore.

There's no risk to analyzing many things at once. “People seem afraid of tracking a lot of stuff because they think they’ll run out of space, bog the system down or take too long to query — but it’s cheap — so it’s always a win to track more,” says Porterfield. “If you don’t track enough you won’t learn enough.”

Not tracking enough stuff can end up being like a forest fire. It starts out with small tradeoffs but will grow to impact your entire business.

3. Most of your team is still flying blind.
“A lot of companies think they can throw data into Mixpanel or Kissmetrics or Google Analytics and that's all they need, but they don't really think through who on their team needs access to insights,” says Porterfield. “You really have to make a point of telling everyone at your company that they should be consulting your data and making data-driven decisions all the time. Otherwise, your product team will just build, hoping for the best, and never truly understand why things succeed or fail.”

4. You're storing things in the wrong place.
First, let's look at who's doing this the right way. Porterfield cites one Looker client who uses a tailored analytics architecture built from a unique combination of NoSQL, Redshift, Kinesis, and Looker. This framework not only captures and stores its own data at scale, but can handle clickstream data for millions of monthly visitors, and allows anyone to make queries.

In this case, engineers have to do a lot more work to get their business users the information they need. And, even if you can access the raw data, engineers will likely have to build or buy a tool to help business users understand it. This makes running experiments to optimize marketing or sales a complicated ordeal, and end users simply won’t be using data at the same level.

You need to make sure all your data is in the same place. This is mission critical.

5. You're not looking far enough ahead.
Any good analytics system is designed and built for longevity. “Yes, you can always change your analytics framework, but remember that data is heavy. The more you have, the harder it is to move around from system to system. If you decide to change things up, it's going to be a real pain in the ass that gets worse the older your company gets.”

6. You over-summarize.
While this problem is more common at companies with big data science teams, it's a cautionary tale for early stage startups too. When you try to make your data simpler or reduce size by rolling it up, you risk losing a lot of important information.

The 3 Easiest Ways to Avoid These Mistakes

First of all, dodging these errors will save you much more than you think, Porterfield says.

Not only will they cost you control over how your business and operating day to day, it's easy to sink massive engineering time and resources into patching up a flawed system. If you don't watch out, your engineers could be spending expensive hours deciphering data for your sales team. Your marketers could be missing out on all kinds of opportunities to maximize engagement. Don't risk it.

1. Appoint a business intelligence engineer.
If you choose one person who has an interest in analytics and charge them with figuring out the most effective way to log data, you'll save everyone a ton of time. Analytics support won't be spread out across your entire engineering team, and this one person will have more incentive to figure out the lowest-lift way to capture the insights you need.

2. Own your data.
It's highly recommended to use open-source analytics platforms that let you track all the events related to your product in real time, like Snowplow. It's relatively easy to use, it's well-supported, it scales, and — best of all — it's free. It's also compatible with the rest of the framework you're probably using.

3. Get your data into Redshift or another massively parallel processing database ASAP.
For early-stage companies, cloud-hosted MPPs like Redshift are often the best solution, because they're low-cost, easy to deploy and manage, and they scale well. Ideally, you want your event and operational data in Amazon Redshift from the very beginning of your company's recorded history. “Redshift gives you the flexibility to store a tremendous amount of very granular data without charging you based on arbitrary things like event volume,” he says.

How to Use Metrics to Win Your Market

Analytics are only as good as what they help your company do.
At a startup, all data should be leveraged to achieve what you've defined as success for every stage of your company. “Success metrics are anything that you want to have happening — and that's usually engagement,” Porterfield says.

Identify your key desired outcomes: What do you want your customers to experience? Some common success metrics based on desired outcome are conversion rate (how likely someone is to buy if they do X), time to transaction (how long until a user buys), and churn (users who will likely never buy again). You want conversion rate to be high while you want the other two to be low and dropping.

Focus only on engagement that matters: Different companies value different levels of engagement.

Measure retention and behavior of repeat visitors: Success metrics can also be bucketed into operational and event metrics. Make sure you're looking at both to really understand what forces are at play.

Invent new metrics: In order to track what is most valuable and impactful for your business, you have to create unique metrics specific to you.

A while ago, the payment app's support team started hearing from users that many of them were accidentally paying their friends instead of requesting payment from them. The buttons were right next to each other.

Make Your Metrics Sharable

Analytics will only be useful if you can share them throughout your company. “When you're sharing metrics, the number one troublemaker is misnaming your terms,” says Porterfield. “You have to have incredible clarity around what every single thing means.”

Data is the lifeblood of successful companies. Sharing it not only creates a healthy sense of transparency, it also creates alignment between business units that should be working together.