If you are anything like us, you are swimming in a sea of data. As marketers and designers, you are forever surrounded by data points, charts, graphs, and visualizations. A typical day could involve checking your email campaigns’ performances (ideally going up and to the right), your website visitor traffic (also ideally going up and to the right), as well as your your bit.ly clicks, Facebook insights, and more (most of them, you guessed it, ideally going up and to the right).

Even if you have a fully-staffed marketing team, reporting all of these numbers is a major undertaking. Your job—or part of your job—is reporting on progress. There are a number of people who want to see those numbers. Your boss wants to see that you are growing your audience, and the board of directors wants to see that your new website is getting a ton of “hits.”

THE CURSE OF VANITY METRICS

These various data streams create a smorgasbord of data. And, all too often, these stakeholders often want the best-tasting data, the data that shows up-and-to-the-right, even if it isn’t the right data. This leads to reporting on what has been dubbed “vanity metrics.” We define vanity metrics as a number that indicates improvement but is disconnected from the progress of your organization’s mission.

A prime example of a vanity metric is “web traffic.” More traffic can be a good or even a great thing, but the context always matters. Let’s say your stakeholders have decided that boosting traffic on the website is a critical goal, which leads to you spending thousands of dollars on Google AdWords. You will probably end up with more traffic as a result, but, if none of those users convert into email signups, then this additional traffic probably isn’t helping you achieve your mission.

JUNK TRAFFIC THAT DOESN’T CONVERT IS VERY RARELY USEFUL TO THE ORGANIZATION.

Unless you are targeting a particular audience or segment, just adding more visitors doesn’t mean achieving your mission.

This is where we (literally) take a page from Eric Ries’ book, “The Lean Startup.” While Ries’ bestselling manual for new businesses was written with high-tech entrepreneurs in mind, his lessons can easily be applied to the masses. Ries stresses the importance of smart metrics using “the three A’s”: metrics that are Actionable,Accessible, and Auditable.

ACTIONABLE METRICS

Actionable metrics tell the story of what action needs to be taken to meet your goals. The gold standard for actionable metrics is the results of A/B testing. For example, the open rate of two email subject lines in an A/B test will immediately tell you which subject line to pick (assuming your list is large enough to have the test achieve statistically significant results). Actionable metrics use the scientific method to pinpoint exactly what change will lead to the outcome you’re trying to create.

ACCESSIBLE METRICS

This means metrics that make sense. A smart metric needs to be accessible to pretty much everyone: precise enough for the technical team, visual enough for marketing, and grounded in real-world behavior for any member of your team to understand.

Some bulk email programs can be a total headache in terms of accessibility. They give you total number of opens, clicks, bounces, etc. for the email campaign. However, if you focus on the unique number of opens (a user can open one email multiple times) your results may confuse and disengage your team.

As Reis wrote for the Harvard Business Review,

KEY DATA SHOULD BE AVAILABLE TO ANY EMPLOYEE, ANYTIME, IN A MATTER OF MINUTES.

Go for clear, concise data with a short feedback loop between seeing a result and taking action.

AUDITABLE METRICS

Auditable metrics mean that ideally any member of your team could find the source data and reproduce your report. This means simple report generation. The web is full of auditable metrics, and, as long as you have your analytics and tracking programs installed and running correctly, your metrics should be auditable.

A STORY OF EMAIL LIST HEALTH

Corporate Accountability International is a Boston-based nonprofit committed to stopping life-threatening abuses by global corporations. Throughout their 35-year history, they’ve waged game-changing campaigns against giants, like Nestlé, GE, and McDonalds, and have mastered the practice of mobilizing millions.

Like most nonprofits, Corporate Accountability has significant goals on increasing the size of their email list. For years their list was well in the range of benchmarks for open rates and click-throughs, so there was never an urgent need to re-evaluate the metrics they were reporting.

However, only 60% of their email list were sendable addresses. Of the sendable portion, 45% had never opened an email. While their list looked great in terms of “size,” that metric masked an unhealthy amount of unsendable email addresses and checked-out subscribers who wouldn’t open and or click on an email. This came with big implications such as skewed testing data and higher spam scores.

After making the case to the executive team, Corporate Accountability took a radical step: they cut down on their email list by 40%. They made the case that email list size was a vanity metric and the email list health was the actionable, accessible, and auditable metric. In the months that followed, despite sending fewer emails, their open rates doubled, their click-through rates increased by 50%, and, most importantly, they raised more money from the year-end appeal. By shifting focus from numbers to impact they were able to concentrate on metrics that deliver actionable insights. With better insights they communicated better with their audience; with fewer emails they raised more money.

OVERCOMING CHALLENGES

Be warned: refocusing on actionable metrics is hard work and it takes practice! Be prepared for the possibility of pushback from your boss, staff, and, if applicable, your donors, but go forth with confidence that these changes are necessary.

We recommend you get started by picking one area you think you can improve on and focus on it for 2-4 months. Remember, start with the story and then work on finding the data.

Please feel free to use the comments section below to share your own battles with vanity metrics as well as your experiences tracking actionable metrics. What are some of the tools you’ve used? How have you been able to customize services like Google Analytics to make your metrics tell a story?

ADDITIONAL RESOURCES

• “Vanity Metrics vs. Actionable Metrics” — Eric Reis.
• “Clean Up Your Email Lists and See Metrics Surge” — A look a Corporate Accountability International’s shift to smart metrics.
• “Is Your Nonprofit Addicted to Vanity Metrics?” — Lean Impact.

ABOUT THE AUTHOR