Telling the story of metrics inside your news organization

I’m presenting this at a Poynter event called Measuring Journalism on Nov. 10, 2017. My slides are available here, but I needed some blogging exercise, too.

I love it when the numbers jump off the screen

In my dozen-plus years in the news business, whenever I’m asked — often in a job interview — what my favorite part of my work is, I tell a little story about the first time I took a serious look at an analytics report for multiple news sites, back at GateHouse Media. Sure, I had been responsible for paying attention to Omniture (or was it still HitBox?) back at the Santa Cruz Sentinel, and we even had a pretty cool heatmap plugin of some sort which let us see how stories on the homepage were doing. (This was years before Chartbeat.) But at GateHouse, I worked with around 125 community newspaper sites.

Months into the rollout of a new content management system — and really, a new way of working, the first time most of these newsrooms were publishing stories in the middle of the day, or breaking any news online at all, and definitely the first time they shot and edited their own video — I took a look at the metrics.

The differences in pageviews and unique visitors jumped off the screen at me. Newsrooms with strong leaders that had some digital experience were succeeding in the new model, as measured by these metrics, at least. It also helped me realize other newsrooms might need more training on the new system!

That experience — that aha moment — is the one I talk about when people ask me what I enjoy most about my work.

I love it when the numbers jump off the screen and answer a question.

###

Telling the story of metrics

I’m going to talk about three key elements you’ll need to tell a great story about metrics for an internal audience. I’m usually talking about newsrooms, but this should work for most organizations.

  1. How to identify the metrics that matter for your organization.
    Or at least, how to focus on the metrics that make sense for your organization’s goals.
  2. How to make metrics useful — where useful equals actionable.
    (I apologize in advance for the business-y paradigm here, and you should feel free to offer improvements on the word “actionable” in the comments, on Twitter, to my face, etc., but I think it’s a good start.)
  3. How to report smarter, not harder.
    Ban the scourge of copypastism from your analytics routines.

Which metrics matter?

Saved to Pinterest by a second grade teacher. Search “data wall” on Pinterest for many, many more like this. They have goals! And KPIs! And graphs! You can, too.

Look, here’s the thing. There are lots of metrics. Not all of them matter. I mean, they do, sure, to somebody, somewhere, yes. But probably not to you.

The metrics that matter to you, dear reader, are the ones you’ve identified as key performance indicators (KPIs) tied to a particular tactic that ladders up to one of several strategies you’ve set for, say, the fiscal year, which in turn ladder up to your vision for the next several years, which of course you’ve aligned with your mission as an organization.

[muffled laughter]

OK, OK, really, I mean, some of you have done this level of strategic planning. But in a newsroom, plans can change pretty quickly. Maybe an election doesn’t go the way most people expect it to, and the focus of your newsroom turns on a dime. Maybe you end up in a running battle with the executive branch, or Congress, or the state legislature over access to public information. Anything could happen.

But, if you’re the one responsible for answering questions like “How are our stories doing?” or “What’s driving subscriptions these days?” or even “How can we get more people to show up to our events?” you’re going to need to set some basic goals and identify the key metrics that indicate how things are going.

I’m not going to do a whole goal-setting workshop here, but suffice it to say there are plenty of methods, lots of ideas about how to do it right, and you can start small.

For example. If increasing the size of your overall audience is a goal, hey, there are some key metrics that will serve you well. Pageviews, New Users, social referrals, there are a bunch that would be good to track.

Let’s look at a big menu of metrics we could be paying attention to:

(Hey, this is obviously not a comprehensive list of every possible metric you could be tracking, right? Right.)

So that’s too many things. Don’t try to pay attention to all of this at once.

Let’s take a look at the same list, with some highlighted KPIs that might be relevant to your goals as a news organization.

Analytics bingo, anyone?

That’s better, right? If your goal is audience growth, identify the KPIs that will tell you if you’re getting your stories in front of more new users.

If you already have a sizable audience, and you’re trying to get them to come back more often — hey, maybe someday they’ll buy a subscription or make a donation if that’s how you roll — you need to look at a different set of KPIs.

Can you do both at once? Of course, sure, why not. But don’t conflate growth metrics and loyalty metrics if you can help it.

What other goals might your organization have besides audience growth and increasing loyalty? Plenty, sure, including little things like advertising revenue. Use a broader metrics menu like this one as a prop to help talk with the rest of the people in your organization (I’m trying so hard not to say stakeholders) about which metrics matter for which team, right now, based on your current goals.

Making metrics useful

Jack needed a little more context to gain actionable insights from the available data. We’re pretty big Nightmare Before Christmas fans in my house.

Cool, great, so now you’ve identified the metrics that matter, and you’re tracking them closely (oof, more on that shortly), and all this quantitative data is flowing into your dashboards and spreadsheets and you can do the math to show how the numbers change over time. You’ve got graphs! We love graphs! Data visualization is great. But… why did the numbers change over time? What happened? Where did the new users come from? Which stories brought people back for second and third visits within a 30-day period? Is the language in our email newsletter signup form working?

Questions! So many questions.

Your reports should try to answer some of them. And they should include ideas about what someone (editorial, product, business, leadership, engagement, etc.) should do next as a result. Be brave, be bold, make suggestions, especially when you know from the data these actions lead to results.

But at the start, the narrative of your quantitative report might look something like this if you wrote it out in words:

Pageviews increased by 25% in October 2017 compared to September 2017, but they were down 10% from October 2016.

OK, cool, sure, that’s nice. But we have so many questions. So, instead, let’s dig a little deeper for context and provide actionable insights (sorry, not sorry).

Most of our pageview growth this month came from new users reading our Steph Curry story that went viral on Facebook, with triple our usual shares. A KQED page posted it, too, with a ton of engagement in the comments. (BTW, last year this month, local election coverage was through the roof; we were expecting a YoY dip.)

Aha! Now we’ve identified why we had a bunch of pageviews, what kind of pageviews they were, what story drove the traffic, where the readers came from, and — this is important — how much better than normal this was. And we’ve identified a key partner who helped amplify our story. Context! OK, so let’s make this a-c-t-i-o-n-a-b-l-e with some takeaways for the team.

Takeaways:
— Contact the KQED page admins to say thanks, get an email address to send them relevant stories.
— Our reporter should reply to some comments and questions in the thread on the KQED post.
— Follow ups on Steph?

Hard metrics are really important to track in aggregate over time so you can report back to management, the board, your funders, donors, whatever, about how you’ve moved the proverbial needle over time, but they’re not terribly useful on a daily, weekly, or monthly basis.

Unless you unpack them, figure out where your traffic came from (or where it didn’t come from!), find out what worked and what didn’t, and identify follow-up tasks and takeaways to be assigned to team members.

It’s list time.

Three things you can do to make metrics (more) useful inside your organization:

  1. Include context with quantitative data.
    Why did the numbers go up or down, where did the readers come from, what did they read, how is this month/week/day/quarter/hour different from another? Is this unusual? What was different?
  2. Takeaways, takeaways, takeaways
    How can we learn from this and what can we do today to take the next steps to either try to make the magic happen again, or to figure out what went wrong and try something different tomorrow.
  3. Know your (internal) audience
    Deliver takeaways to the people who can take action today! Find the right cadence for communicating takeaways. Monthly or weekly may not be often enough, depending on your organization and publishing frequency.

There are two more examples of quantitative data / actionable insights comparisons in my slides, if you’re into this sort of exercise.

Report smarter, not harder

A live look in at the analyst the night before the metrics report is due.

We’ve all been there. It’s time to build the weekly/monthly/quarterly report, and screenshots from Google Analytics aren’t going to cut it. (Sidenote: Maybe that would work fine, though?) So it’s copy/paste time. Surely you won’t make any small errors that will undermine the entire premise of your report… Surely…

And beyond the added risk of introducing errors, building reports this way is a waste of your valuable time. Let’s automate the quantitative stuff and save your wildly flailing frog limbs for the aforementioned actionable insights.

Three principles for smarter metrics reporting

  1. Ban copypastism
    I’m tempted to make you all take a pledge to end copypastism in your news organizations within 30 days.
  2. Automate what you can; make time for developing actionable insights.
    So we come to automation. Automated, regularly scheduled metrics reports (of the metrics that matter to the internal audiences you’re serving) require a significant amount of up-front effort. Reader, they are worth it. Whether you’re populating a shared spreadsheet, firing off a scheduled email of a PDF report, or routing key metrics into Slack channels, getting the quantitative data in front of people with as little repeated effort as possible will make your life easier.
  3. The right people get the right data at the right time.
    Does your revenue team need to know about your top 10 stories of the month? Does your editorial team need to know which email newsletter signup form is working best today? Does your executive team need to know what color subscribe button is winning the A/B test you’re running? (The answer might be “no” to most or all of these questions.) Yes, be transparent enough with metrics so that everyone in the organization can find them if they need them, but don’t present a wall of data to everyone who comes asking. Be selective, and give the right people the right data, as often as it’s — useful!
    Ask yourself, is this report actionable, for this audience, at this frequency.

Automate it, for the people.

Here are three tools you can start using tomorrow. All of these are Google Analytics-related, but all have analogues in the world of Adobe Analytics (neé Omniture Site Catalyst).

  1. The Google Analytics Spreadsheet Add-on will allow you to schedule clean quantitative reports in Google Sheets, keeping executives and senior editors out of Google Analytics.

    We could have a long discussion about whether you should train everyone how to do everything in GA, or whether you should have them bring you their more complex questions, or whether you should find the right hybrid treatment that works for you — every organization is different, and frankly, every journalist is different. But I’ve seen this work for eliminating a lot of copy/paste duties for things like “what were our top 10 stories last week” and much, much more.

  2. Google Analytics Custom Reports let you ask more complicated questions and route the answers on a regular basis to stakeholder inboxes.A real world example: The homepage editor wants to know which stories perform best from the homepage itself. What are people clicking on when they get to the homepage? Create a custom report using page paths and schedule it to be emailed to her on a daily, weekly, or monthly basis.
  3. Google Analytics Dashboards can be customized as much as you like and shared with the rest of your organization. Build one for executives, one for editors, and one for the product team.

 

More automation tools
All of these tools are intended to get you, the analyst, out of copypasteland and into a routine of developing the actionable insights your organization needs to grow and succeed. Along with the above, you may already be using some combination of web, social, and other metrics tools such as Chartbeat, Parsely, Sprout, Hootsuite, Buffer, UTM codes for links, IFTTT or Zapier to connect the dots, or even Slack integrations to deliver data to the right team.

Is all of this elementary to you and you’re looking to take things To The Next Level™? Then it’s time to get serious about some sort of full-on business intelligence tool. Looker and Periscope are two of the leaders in that field right now, but we’re talking about a much different cost and level of effort.

In all cases, putting in the hours to set automated systems up to deliver actionable insights to your team is Worth It.

In conclusion

It’s a good joke.

But it’s also your goal: serious journalists publishing stories that make an impact, paying attention to what the metrics can tell them about how their story landed, who is reading it, and learning how they can improve the next story.

 

#sholinonrap on Clayton Christensen

It’s been a good week for guest speakers here at the office. Gary Vaynerchuk was here on Tuesday, and Clayton Christensen spoke yesterday. Pretty much a coincidence, I think, as their talks were part of two different programs here, but I think Christensen would happily cite @garyvee as an example of his theories in action.

So Gary was fun, but I was looking forward to Christensen. There’s a not-too-thin logical chain where I have a job in this industry because of his research getting into the hands of certain early online news adopters.

When Gary spoke, I think I promised Sean Blanda I would give Christensen the #sholinonrap treatment.

I had my dates mixed up, but you get the picture.

For the uninitiated, “#sholinonrap” is how Sean responds on Twitter or Facebook or wherever, whenever I make some sort of hip-hop reference. Last time Sean was here at the office for some sort of panel, I gave him the treatment, tweeting about the panel in the form of rap lyrics.

And not only was it fun, but it was also a disciplined approach to note-taking that forced me to do three things:

  1. Pay attention.
  2. Boil down complex topics into simple, 140-character-or-less approximations of lines of verse, complete with rhymes and flow where possible. (I am not an expert at this, kids.)
  3. Show my work by tweeting it.

In other words, an efficient and entertaining (for me, anyway) bit of exercise for my brain.

So yesterday during Christensen’s talk, I found a nondescript spot in the audience between a VP of News and a Social Media Editor and did my thing. It went okay.

Many, many thanks to Chris Amico for whipping up a Storify of #christensenonrap before I even had a chance to get back to my desk to prep for my next meeting. Here it is:

[rawr]//storify.com/eyeseast/christensenonrap.js
[View the story “#christensenonrap” on Storify][/rawr]

###

I had a couple interesting conversations later in the day about why Christensen might not have been too excited about answering direct questions about how this applies to the news business.

One reason might be that the American Press Institute (hi y’all!) spent a few years of research on how Christensen’s theories about disruptive innovation fit into the news business. Newspaper Next. You might have heard of it. Maybe not. Here are a few useful links. (Some of these are remarkably useful.)

Plenty to peruse there over your holiday weekend, eh? Anyway, hope you enjoyed this episode of #sholinonrap. Beast!

 

Link: How to Use the “Network Density” Formula to Measure the Health of a Community

I’m a sucker for engagement formulas, especially when they help provide a means to quantitatively track something that sounds as fluffy and qualitative as “the health of a community.”

Great ideas here for community managers and related parties: How to Use the “Network Density” Formula to Measure the Health of a Community

 

2012 Civic Media Conference takeaways, open questions, reactions, notes

It’s been three years since I last made the trek to Cambridge for what we once called KNCMIT, and although the cast of characters has changed (with little-to-no representation of 2007-8-9 Knight News Challenge winners, different faces at the MIT Media Lab, and a rebooted Knight Foundation posse) the outcome was similar.

All unhappy airport terminals are alike, so the sense of deja vu carries over from the hotel to the cab to the fluorescent carpeted discomfort of Logan, and the foreboding sense of dread that comes with a United flight. (Prove me wrong, airline. Prove me wrong.)

On to the obligatory, but hopefully not exclusively duplicative and obvious notes:

  • Homicide Watch is excellent, and repeatable. Whether or not you use the code powering the DC site, the model of reporting on every homicide in a city — and not just reporting it, but reporting on it, while maintaining pages for every victim and suspect — this is something that doesn’t depend exclusively on technology, although the platform is perfectly tailored to the job. But it does depend on being obsessed with telling the stories that we often hide behind numbers, or a map. (Previously.) Also, it helps to be as driven and passionate about it as Laura, and to care about people.
  • Sometimes, there’s just no story in the data. Jonathan Stray hoisted this banner of editorial force, and Daniel X. O’Neill waved it high, the sort of basic news value that journalism school drills into us if we listen: Check the facts, check the data, then double-check it and account for the fragile chain of human actions that produced the data. Because a spreadsheet packed with invalid data and intervening variables is not a story. It’s a mess, and a risk, and it might be the start of the reporting process, not the end of it. (Session video here, worth watching.)
  • The contraction of the Knight News Challenge grant cycle into themed 90-day periods is a right and good thing, as is the new prototype fund. You should apply to one or both, right now. This minute.

Not pictured in this list: An improved opinion of the food and beverage options in Cambridge, Mass.

A future history of crowdsourced reporting

During the second or third or so year of my still-brief career in what we might as well call “the news business” for lack of a more encompassing and descriptive term, I found myself jumping up and down advocating for a tool to standardize the task of gathering data from the news audience.

Crowdsourcing as a term was new, and by definition “bigger” than just “sourcing” because it could happen at scale, where scale could be thousands, hundreds of thousands, millions of people with the right call to action and programming framework.

WNYC’s “beer, lettuce, milk” price data gathering project was a favorite, although it appears to have been powered by a comment thread, mostly.

That was always one that stuck out in my mind, due to the quantitative nature of it. This wasn’t about asking the news audience for opinions; it was a method of gathering facts about the city and its bodegas, data that wasn’t compiled anywhere, and that made sense to bring together in one place, given the chaotic system (system?) of New York City bodegas.

###

Matt McAlister has gathered a Big Important List of crowdsourced reporting projects, and he’s notably compiling a list that extends beyond traditional journalism and news organizations, as we all should.

It’s a fascinating list of projects, and a reminder that it’s not always “content” news organizations are looking to “generate” from “users,” but information, or perhaps better yet, analysis of documents or images or cities or rivers or the world surrounding them.

Again, my own interest, albeit usually from afar, in tools like DocumentCloud, is the chance to bring the audience into the reporting process by giving them an assignment. “Read a piece of this giant 1100 page budget, or campaign finance bill, or FEC disclosure, or Friday night data dump (see the classic Talking Points Memo instance here), and annotate it so we can find the important stuff quickly.”

###

The fun part, naturally, isn’t in examining the past of crowdsourced reporting, but imagining the future. What does a platform to quickly spin up an instance of a crowdsourcing machine when news breaks look like? More than a map, surely, as amazing and powerful as location can be. It has to be flexible and fast and able to parse submissions into something useful, digestible, sortable so the most important information surfaces as if it were weightless.

Or are we already looking at the platform, the system, in Twitter or Facebook or Google Search or the Web itself? I don’t think I believe that. There must be more, or there must be a federated system to harvest and groom information from all these sources — not for the purpose of curation into a story or list or gallery, but for analysis, understanding, quantification at scale.

Continuing to dream of that ideal crowdsourcing platform…

Is Reddit journalism? The inevitable investigation.

If your interaction with Reddit is anything like mine, you’re a 9-percenter.

Remember the 90-9-1 rule of online community interaction? Well, on Reddit, I rarely say a word, and I’ve probably never started a thread, but I do so enjoy their magical little UI for upvoting posts and comments, especially on my phone, often in the middle of the night while trying to get a child back to sleep.

That places me somewhere between a lurker (90 percent) who never logs in, just reads and scans, and at best, might link to a thread from elsewhere, and an active participant (1 percent) who posts daily, optimizes their headlines to be more likely to garner enough upvotes to land on the homepage (please note the title of this blog post), and/or creates “novelty accounts” — usernames designed to be part of the joke themselves.

It’s a fascinating community, with Reddiquette that has evolved over the years, and a language of acronyms as described by David Weinberger in a blog post this weekend that acts as the beginning of a set of open questions along the lines of “Is Reddit Journalism?” But those quotation marks are my own. David’s questions are much better than that.

His questions revolve around the idea of “Reddit and community journalism(the actual title of his post, clearly not optimized for upvotes at the time of this writing.) Several key Reddit acronyms are covered, including TIL (Today I Learned) and AMA (Ask Me Anything).

Sound familiar?

Open up a daily newspaper, and find what in no uncertain terms we’d call “community journalism” in the form of interviews with and profiles of local personalities, unsung heroes, hidden gems, people in your neighborhood, etc.

That’s an AMA.

Admittedly, the request queue for print coverage in this vein could be considered a little less democratic than on Reddit, where a search for “IAMA request” strongly resembles the early days of the Help A Reporter Out mailing list.

And of course, we’ve all read columnists elaborate on some interesting tidbit of information or history of their community, sharing a discovery with their readers, who often write back in the form of letters (and now, comments, naturally) and share their own point of view, rebuttals, or even memories of the factoid in question.

That’s a TIL.

Now, go upvote this on Reddit.

If it makes it to the homepage, I’ll write a sequel titled “10 ways Reddit is like a newspaper in the 1980s.”

 

Hardly Strictly takeaways

By now, you’ve surely forgotten the barrage of tweets and check-ins from 30 or so of us — the “Hardly, Strictly Young” David Cohn invited to the University of Missouri’s Reynolds Journalism Institute a couple weeks back for a round table Carnival of Journalism mission to gather alternative ideas about how to implement the Knight Commission’s recommendations.

We spent a full day in four rotating groups of around eight people, taking on one of the big Knight Commission questions at a time. Here are some mixed notes, with findings from various groups that still stick out in my mind, some key ideas tweeted, and associated other free associations.

The Four Questions

1. Town and Gown connections

Along with the standard j-school/local news organization mashups, we tried to dig a littler deeper into a goal of breaking down the barriers between a university and the community that surrounds it. One key bit of epigrammar: We need both Public Professorship and to learn from the Professorship of the Public.

At Matt Thompson’s lead, we even went so far as to imagine what a layer of 106 & Park-like hashtag trappings might look like when draped over a civic issue, as a tool to teach modern media literacy. Maybe even a local debate framed as an American Idol-style tournament of viewpoints, complete with SMS voting.

And then, there was Cody Brown’s media literacy in the classroom recommendation:

[blackbirdpie url=”http://twitter.com/#!/CodyBrown/status/60382812557152256″%5D

2. Increase the number of news sources

This is one that was subject to a great deal of interpretation during the pre-conference Carnival of Journalism. At my table, we had the benefit of insight from #expertmode crowdsourcer Amanda Michel of Propublica, so we took the word “source” literally.

I walked away from the table with at least two excellent (you might even call them actionable) product development ideas:

  • Add a tip form to your 404 and/or empty search results pages. “Didn’t find what you were after? Tell us what we should be reporting on.”
  • Build yourself a source dashboard behind the scenes where you can connect a commenter or contributor to their Twitter and Facebook accounts (and blogs, etc.) even if they haven’t connected them to their public accounts, so that you get a holistic view of reliability, expertise, and behavior.

3. Expand local media initiatives to reflect the communities they represent

This was one I felt strongly about, as evidenced in part by my open Carnival letter to new Knight Foundation VP Michael Maness. I think our host might have remembered that, as I was denoted as one of two presenters on the topic for our table, along with Nieman Journalism Lab reporter Megan Garber.

I’ve posted the full notes from our group’s recommendations, but here are a few highlights of our three point plan to connect Knight journalism grant technology with community nonprofits and ethnic media organizations that are already providing community information needs, but could use a little push to expand their reach and build capacity:

  • Outreach: A recurring theme in recommendations on this topic: Find the existing community information providers that are already thriving, but need support for capacity building in order to expand their reach.
  • Pilot Grants: Having identified the key community information providers with a lot of upside, show up at their doorstep with small grants (preferably with quick turnaround times on approval, having already taken steps to establish criteria and identify potential grantees). The goal of these grants? Build their capacity, expand their reach.
  • Apply a generous layer of News Challenge technology: Given the community, the focus of each organization, and the pilot project, connect these new community information grantees with News Challenge (and other Knight journalism program) grantees. Apply KNC technology (and programmer/journalist resources) to the new grantee problems and challenges.

4. A local information hub for every community

The group I was in had a great deal of trouble finding a niche to work with here. The technology to make this available already exists, and in many communities, local news sources get this job in some form, too.

One successful model we weren’t alone in identifying? Local wikis, like Davis Wiki, and my personal favorite, RocWiki in Rochester, NY. The Davis Wiki team, matter of fact, is a Knight News Challenge winner, currently building out tools for building local wikis.

And my other favorite implementation idea, intended to bring Web access, literacy, and skills to areas where broadband coverage is still sparse, the Web-a-bago.

[blackbirdpie url=”http://twitter.com/#!/moniguzman/status/60373160259891200″%5D

Hardly Strictly More to Read

Content asteroid belts

Cameron Koczon on how publishers and readers manage satellites like Instapaper, Readability, and other unbundled flying objects: A List Apart: Orbital Content.

“Many publishers will ask—and it is a fair and familiar question—why should users have the right to carbon copy my content and share it in other contexts? It is a question that belies a concern about something slightly different: compensation. If publishers were compensated $10 every time content was shared and $1 every time it was read on their site, they would do everything in their power to get their content shared. Copying is not the problem—compensation is. Today’s web environment makes it nothing less than a struggle to support content creators. We have unlimited tools for sharing and virtually none for payment.”

It seems safe to add News.Me, Summify, and Trove to the list of satellites — or rather, to the list of, say, content asteroid belts that circle readers and publishers.