Monday, October 23, 2017

How I Became a Data Plumber

The other day I realized that big data, business intelligence, and dashboards may sound sexy in my strategy and analytics world. But all that and the insights they promise is not really possible without the less glamorous work of what I call "data plumbing." Yes, mom, all my years of schooling and work experience has led me to a career as a plumber =) Having a modern plumbing system in place is not only critical for your home, it's also essential for marketing.

Let me share a real life example of one of the many things I do as a Data Plumber. In this example, the role of the Data Plumber is to make sure all the pipes are connected end-to-end for more precise data to flow to improve tracking and optimization.

Here's a common use case: To optimize paid search campaigns, we need keyword level data for offline conversions. This happens a lot for lead gen campaigns where sales are closed offline, especially for B2B clients and B2C clients whose products or services require more hand holding than ordering a book from Amazon to close the deal.

First, I work with the media team to generate and pass a unique click ID to the landing page when a user clicks on an ad. The click ID will provide granular keyword level tracking. That's the first upstream plumbing connection from the search engine to the client's site. This is like getting water from the main line in the street to your house.

Then we need a way to pass the click ID to the client's lead management or CRM system. This is where years of experience talking "tech" with developers and "data fields" with database administrators pays off as I make the business case on why the Marketing Department needs these changes.

I visit the client's CRM team to see if they can create a field for me to house my new incoming click ID from the client's site. In plumbing speak, I need to make sure there is a faucet in the house that can receive the water from the street once I hook up the pipes under the house.

Once that is taken care of, I am off to work with the web development team to pass my click ID at the point of lead generation, usually when a user submits a lead gen or order form on the site. Normally this involves altering the API to the lead management/CRM system to capture the click ID upon form submission. Now we got pipes to allow water to go from the street to the kitchen sink!

But we're not done yet. We need a way to drain the used water back out of the house. I'm back with the DBAs and the CRM team to create a way to send us the data we need via an ETL or dropping a file regularly to a FTP server. We are creating a conversion file that passes every lead and sale with our associated click ID back to the media team's search campaign management system...ideally in real-time (or near real-time) for bid optimization, as well as reporting.

Almost I go back to where I started with the search team. We create a process where they can consume the client's conversion file and match the data back to click ID. What's powerful about all this is you don't just tell the campaign management tool that a keyword led to a sale or not (which is binary), but you can also pass the amount of the sale, product type, and even the customer segment for even more insightful reporting!

Without such plumbing in place, you can optimize at the keyword level on web leads, but not sales. Now, you may find you've been spending a lot of media budget on a keyword that generates a lot of leads, but has either a low sales conversion rate or a higher CPA (cost per acquisition) than you thought because you've been optimizing to CPL (cost per lead). Or perhaps you weren't getting the quality of sales or the right customer or product sold than you thought for some keywords. That's like having hot water enter your house, but only warm, foul-tasting water may be coming out at the tap unbeknownst to you because you just saw water coming out of the tap (assuming everything was working fine), but you never felt the temperature or tasted the water to realize something was wrong.

So, there you have it. Just a day in the life of a Data Plumber. =) Actually, in this case, the above would have taken me weeks, not a day, to setup in partnership with my client, media teams, web dev teams, and CRM/database administrators. But when it's all done, it feels great!


No comments:

Post a Comment