Category: Analytics

Data Driven Product Management by Taner Akcok

Every product manager talks about data-driven product management but what is the real explanation of it?

Product decisions are used to be based on product managers/owners and C-Level executives’ desires and instincts. What is the main driver of these instincts; customer feedbacks, competitive market intelligence or digital analytics results? The answer should be all of them. In this article, I will try to explain crucial data sources and which metrics should be considered in these data sources to make right product decisions.

Successful products belong to customers more than product managers. But it doesn’t mean product managers should only rely on customers’ needs and demands. Sometimes even customers don’t know what they want and need. Product managers must validate their customer feedbacks with different data sources.

I strongly recommend to product managers and company executives to check Marvin L. Patterson‘s innovation engine business model in his book “Build an Industry Hot Rod.

As stated in the chart, Product Management should consider the data from Market Researches, CMI (Competitor Market Intelligence), Internal Feedbacks, Digital Analytics and Customers’ Feedbacks to make data-driven decisions and create a value-added information. Sounds easy but assessing the value of the information and tying them to product roadmap needs an effort. Let’s deep dive into them one by one, and I will try to explain how I am using this model in my product management efforts.

Market Researches

What is the probability of heads or tails when we flip a coin? 50%? Unfortunately, no! If we consider all the variables in the equation such as the initial force applied when flipping, gravity, air resistance, etc. there is no probability of the coin. It is same in product management and positioning too. If product managers can consider right market metrics and trends by proper market researches, and useful customer feedbacks, there is no probability to build a successful product.

There are a bunch of useful market research companies that product managers can use. But I’ll explain only two of them in this article; Forrester and Gartner.

Forrester provides ReportsPlaybooksBlogsData and Events to spread out their research outputs. Especially market specific Forrester Wave Reports are useful for market researchers. Forrester Wave researches provide the same outline in all researches for standardization. This standardization allows researchers to combine multiple research easily and reach useful insights. Usually, their Wave Report outline is;

  • Key Takeaways
  • Overview for Market Professionals
  • Evaluation Overview of Key Players
  • Positioning Graph
    • Leaders
    • Strong Performers
    • Contenders
    • Challengers
  • Evaluation Table
    • Current Offerings
    • Strategy
    • Market Presence
  • Vendor Profiles

However, Gartner uses “Magic Quadrant” methodology in their researches to show the market positioning of vendors. They assess key vendors by the ability to execute and completeness of vision. They categorize them as Leaders, Challengers, Niche Players and Visionaries. Usually, their report outline is;

  • Market Definition / Description
  • Magic Quadrant
    • Vendor Strengths and Cautions
    • Vendors Added and Dropped
  • Inclusion and Exclusion Criteria
  • Evaluation Criteria
    • Leaders
    • Challengers
    • Visionaries
    • Niche Players
  • Context
  • Market Overview

These researches show strengths and the weak points of the vendors in the market and help product managers to validate their positioning. Sometimes niche products and ideas struggle to find a market or they may be in need to validate their vision and roadmap in accordance market trends. Market researches help product managers to validate their positioning and the product vision to give data-driven decisions.

Competitor Market Intelligence

This term is used to be named as a function in enterprise companies. CMI departments constantly check other competitors, monitor their contents, annual reports and request a demo to understand competitor value propositions and differentiation. The easiest way of making CMI data available for product roadmap is comparison methodology. The critical point of this methodology is comparing your product as a value proposition and comparing your product as a feature set.

Value proposition comparison usually comes from vendor selection criteria in the market. For instance, let’s consider “Implementation,” “Timeliness,” “Customization,” “Accuracy” and “User Interface” as a main vendor selection criteria for analytics market and define KPIs of these value propositions to be able to evaluate between 100-0 pts. Regarding this evaluation and comparison, you’ll have a pentagon graph to visualize each dimension of your company and your competitors such as an example in below.

Understanding strengths and weaknesses of the value propositions, these pentagons can be used as an area graph and see the comparison by overlapping them.

Feature set comparison is used to be based on product use cases and feature offerings. You compare features or use cases which represent sub-KPIs of value propositions that we listed in the previous paragraph. The general approach to evaluate and compare feature sets or use cases is giving a point them between 1-5 or 0-100 pts.

I’d like to take attention to product positioning at this point. A successful product doesn’t mean a product which has a full advantage on all value propositions and features set comparisons. Your strength areas must be compatible with your product positioning.

For instance, here is a sample table to compare feature sets and validate the positioning of an imaginary company in analytics market:

Comparing value propositions and feature sets help product managers to improve and prioritize their roadmap for competitive advantages. If it requires an improvement for current product, product managers usually tie these insights to their relevant product concepts. If it requires a new feature, product managers build a concept around of it.

Internal Feedbacks

Defining right unique selling points (USPs) and building experience around of those USPs’ use cases is crucial for successful product management. The best way to gather these kinds of insights is internal feedbacks especially from “Pre-Sales,” “Sales”, “Consulting” and “IT” departments. These departments are important stakeholders in product management decision making.

Every product has some “AHA!” moments to hook the potential customers. Pre-sales is an important stakeholder to understand which use cases are the most important for customers and what customers are thinking about the product when they first see it.

Covering important use cases mostly doesn’t mean that product is good enough to satisfy customers in those use cases. “Sales” and “Consulting” are important stakeholders to understand how good product’s feature sets to answer real needs and wishes in those use cases. For instance, a customer wants to go from A to B in the product, if the product takes the customer to a lot of mandatory destinations before B, we can’t consider the experience is covering that use case perfectly even the product covers that use case. “Sales” and “Consulting” are critical stakeholders to discover this kind of bottlenecks.

Ease of integration and use, onboarding experience and documentation are another topic that product managers must consider. “Consulting” is the key stakeholder in this kind of inputs. Maximizing the billable time of consulting, the product should have a good onboarding experience and documentation. This part is important to keep COGS (Cost of Goods Sold) at a reasonable level.

Another important stakeholder of “Internal Feedbacks” is IT. Understanding the challenges behind of the concepts, estimated workforce and investment for the roadmap items are important to be able to calculate more accurate business values before making a product decision. Also synchronizing with product owners and IT leads may helpful to see and evaluate product team solutions in some different perspectives.

Digital Analytics

To quote W. Edwards Deming, “We trust in God; all others bring data.”. Regarding pixels and tag integrations, online solutions gather almost all device, user, and action data. Right event schema and analysis structure is the key point to drill down and reach to the right data for understanding customer behaviors and their pain points easily.

I would like to mention two important topics about digital analytics in this article. Please find detailed information from my “7 Predictions for the Analytics Market” article and my next posts.

The first topic is the importance of combining all your data into one platform. Product managers used to reach combined data by data warehouses, data management platforms or customer intelligence solutions. Nowadays, businesses gather big amount of data from various data sources and internal/external services such as MagentoMailChimpZendeskSalesforceOlark, etc. Combining these platforms’ data into one environment is important to be able to do cross-tool-analyses. For instance, we should combine digital analytics and email service data and match them with unique identifiers to reach how many men appeal buyers opened our “New Season Men Appeal Products” newsletter. Then maybe we can target them with retargeting campaigns by their cookie data and label returners as a “potential new season product buyers” in our database automatically by connecting our advertisement and CRM services to our digital analytics platform.

The second topic is using proper analyses and monitoring right metrics for the desired outputs to give data-driven decisions. Digital analytics solutions offer a bunch of analyses options. I will deep dive to all these analyses in my next blog posts, but I will mention just a few important analyses and metrics in this article.

Funnels: Funnels are one of the best ways to monitor conversions between predefined steps. The most common visualization way of funnels is Sankey diagrams. Product managers or business analysts can create various funnels. For instance, for check-out process, we can examine some steps such as Add Product to The Basket => Go to Checkout Page => Choose Payment Option => Check Out. Then we can create some segments of people who bounce in this funnel and set a marketing trigger to send them an email like “Do you need help for check-out.”

However, in the real world, sometimes funnels may be more complicated than this. For example, customers may visit other products before the checkout. This may affect our funnel, and we may consider them as bouncers. For this kind of scenarios, we may define some of our steps as an “optional” step in our funnel. Also, a lot of analytics solutions allow defining time condition to steps for users to complete the goal within a specific timeframe from the previous or first goal.

Fallouts: Fall out analyses are the combination of the page flow analyses and the funnel analyses. Product managers and business analysts can see the detailed user behavior between funnel steps to understand why people are bouncing and where they are going. This kind of insight may helpful to create more accurate funnels or user flows.

Cohorts: Cohorts are well-designed visualization way of the measure the desired metric for conditioned target groups. Product managers should decide “Granularity” of the cohort with “Inclusion Metric” and “Returning Metric” before running the cohort. Inclusion metric helps you to define conditioned target group such as All Visitors or Buyers etc. And Returning Metric will help you to choose which metric you want to measure out of Inclusion group. The “Visit” can be an excellent example for returning metric.

The most common use of the cohorts is for monitoring the “Retention Rate” of visitors for specific pages or actions.

Flows: Flow analyses let users visualize a sequence of any dimension or dimension item. Also, flow analyses are good to visualize multi-dimensional sequences too. For instance, Entry flow of the “Campaigns” dimension. If you add “Pages” as a next dimension, you can easily visualize how many visitors are coming by campaigns and which pages they are landing. The most common visualization way of flow analyses is “Sankey Diagram” in the market.

Product managers can visualize “Entry Dimensions,” “Exit Dimensions” or just any “Dimension Item’s” flow in single or multiple dimension perspective.

Anomaly Analyses: The most common use of these analyses is visualizing the standard deviation levels for each data point and show anomalies (exceed or underperform) by supporting area chart with a line chart.

Product managers can notice anomalies in their main metrics and drill down to sub-metrics to understand the reason for the anomaly. But nowadays, some of the enterprise analytics solutions are providing also related sub-KPIs of that anomaly.

Analysis Variant Comparison: Analysis variant comparison let users compare the metric with different times ranges or segments or both at the same time.

As I mentioned, I will be deep diving into analyses and corporate metrics in my next blog posts, but I’d like to mention Dave McClure‘s “Startup Metrics for Pirates”. It may give a basic overview of the main metrics that product managers and marketeers should monitor and deep dive.

According to Dave McClure‘s “Startup Metrics for Pirates” figure above; category represents the maturity of the customer regarding customer journey. User status shows the metric(s) that product managers can monitor for that step. Conversion represents avg. conversion rate for that funnel step and estimated value doesn’t represent the cost of the user; it represents the avg. value of each user for that step.

Product managers should use right analysis to reach useful insights for data-driven decision makings. But also, the term of “digital analytics” is way more than visualizing the data. I will be deep diving into this topic in my future posts.

Customer Feedbacks

Customer feedbacks are strategic and crucial for giving customer-oriented product decisions. But gathering, organizing and evaluating customer feedbacks should be in a proper way to be able to consider useful customer feedbacks and avoid non-useful ones.

For gathering and organizing feedbacks, you should build topic-oriented breakdowns for feedback creation process and cluster those feedbacks around current product features or future concepts.

For instance, according to the figure above, we have three different scenarios for Tag Integration – Containers – Container Action – Delete function.

Scenario 1 is representing a bug, which should be clustered around of that feature and be solved with other clustered bugs. Amount and type of bugs can give a feeling about the maturity of the feature in the product.

Scenario 2 represents a problem of costumer which looks like a bug. But in this scenario, let’s say that deleted containers are disappearing in 24 hrs. because of some cache issues. In this case, the product should warn the customer about this condition. This is why we can consider this scenario as a knowledge center enhancement or UX improvement.

Scenario 3 represents a feature request of a customer for that specific function. These kinds of requests should be label as a feature request to consider in next version/iterations for this feature. If the business value prioritizes Tag Integration improvements, this feedback should be considered for go/not to go decision with other requests.

If you have any questions or comments about my blog post, please don’t hesitate to contact me and stay tuned in to my website or Medium account for my next blog posts.

Taner Akcok

7 Predictions for the Analytics Market

How analytics solutions are going to change.

Marketers constantly use insights delivered by their analytics tools to look into the future of their businesses. But how about the future of analytics tools themselves?

Today, analytics has become an umbrella term that spans a wide range of actions – basically anything that delivers meaningful insights from data. With a virtually  unlimited quantity of data and data sources, analytics providers face an important challenge: How to deliver services and products capable of handling the Herculean task of getting actionable insights from data – even at the enterprise level.

Analytics has evolved to reflect the breadth of data being collected. Indeed, analytics is only meaningful when used in tandem with a more specific term – web analytics, customer analytics, marketing analytics, predictive analytics, big data streaming analytics, performance analytics and so on.

That said, all of these analytics branches center, in one way or another, on understanding user behavior. In line with current market needs and trends, understanding the customer behavior and activating those insights requires three steps.

  • Collecting data via SDKs, third-party services and internal services such as CRM, ERP, etc., to combine online and offline data in one platform. This lets analysts and marketers drill down into data with desired metrics and dimensions to reach the right insights.
  • Answering the question of “What do we do with this data?” This involves building segments to manage data and conduct predictive analyses with that managed data.
  • Activating this data with marketing triggers. You have segments and you can trigger your third-party services and combine the results of those marketing activities with your current data to do accurate analyses.

For instance, let’s say you have a clothing shop with a new men’s collection. You can easily create a segment of people who bought men’s clothing before. Then you can trigger your email service to send relevant recommendations and offers. According to the results of the email campaign, you can create a retargeting list for the people who opened your targeting email. Whenever they buy an item from that particular collection, you send that data to your CRM and label those customers as a loyal buyers.

Online actions are getting more and more agile, and you are supposed to be able to act on your data with one click – without losing time to code and connect services.

This is the backdrop we used to determine seven predictions for analytics market. Let’s dive in.

1. User-centricity will be the “must-have” for product-focused analytics solutions

Analytics solutions have evolved to become a lot more than data visualization tools. Businesses collect and drill down into data to create meaningful insights for their corporate decisions and marketing actions. Maximizing profit depends on understanding the users, personalizing your activities and segmenting according to interests or personas.

This is why analytics solutions have become user-centric, allowing users to drill down customer data with precise dimensions and metrics. But this user-centricity trend has only just begun. The ways in which marketers and analysts can build use-centric profiles – merging behaviors, purchases and intentions across devices and over long periods of time – will continue to evolve and improve.

Channel-centric, device-centric – it will all be replaced by user-centric.

2. Digital Intelligence will transform into Customer Intelligence

Digital Intelligence is one of the biggest differentiation points for analytics providers. It is a broad term, but its scope can be boiled down to

  • “Data Management and Availability,” such as anonymous data association, data portability and predictive analytics
  • “Reporting and Analysis Functionality,” such as industry-specific reporting and benchmarks, path analysis and mobile apps
  • “Integration Support,” such as application programming interfaces (APIs), post-implementation managed services and product performance.

Conforming to the new market needs and demands, Digital Intelligence started to transform into Customer Intelligence. The most significant difference between the two is that Customer Intelligence is better suited to incorporate CRM and ERP connections, and allows you to enhance your data with other third-party service connections. This unleashes insights from data and ultimately enables data activation by triggering third-party service actions.

3. Third-party data connections and audience stream will be embedded features in customer analytics solutions

Nowadays, businesses gather data from a huge (and growing) list of different platforms. You have email data in your email tool, e-commerce data in your shop system, support data in your ticket system. The list goes on. Analytics providers need to facilitate the merging of all this data into one platform, and enable businesses to stream this data to other services directly from their analytics solutions.

According to these market requirements, audience stream is becoming more and more essential every day. Audience stream has three main functionalities. The first is pulling data from third-party service connections. The second is pushing data to third-party service connections. And the third is syncing your data with third-party solutions.

What does syncing data mean? For example, say you have a segment of customers who bounced from the process of buying men’s apparel in the last three weeks. You want to synchronize this list with your retargeting list.

The segment is dynamically growing, but also some customers are exiting from this segment after a number of weeks. A data sync enables you to feed that data to your third-party services, and remove them from the list to optimize marketing costs and efforts.

Examples like this abound. Marketers have to be able to execute these kinds of actions without any dependence on technical skills or third-party solutions. That is why it will become a market standard in analytics solutions.

4. Activating data with marketing actions will be part of analytics more than marketing automation tools

Marketers are using marketing automation tools to run their automated SaaS funnels. But all these marketing automation solutions are CRM-focused, and they offer their own marketing services independent from the marketing services already implemented. This results in extra marketing costs and disorganized data.

Analytics solutions will solve this problem in the near future by allowing their customers to combine their online and offline data with better CRM, ERP and third-party data connections. Analytics solutions will also enable custom marketing actions – or trigger users’ third-party services for email, push notifications, advertisement, etc. – with one click.

5. Cross-device tracking and custom track domains will define the new data quality standards of the market

Optimizing operational and marketing costs is extremely dependent on the quality of data that businesses possess. Why is data quality so crucial? Let’s examine this topic with two hot technologies: custom track domains and cross-device tracking.

As you are probably aware, ad blockers are becoming more and more popular every day. According to research, businesses lose 13% of their overall visitor data as a result of ad blocker usage – which also corresponds to 40% of overall Millennial data. That’s why custom track domains – which enable companies to collect better data by setting analytics cookies from within their own domains – will be so vital. They fill in that missing data.

Another hot concept is cross-device tracking. It is easy when people connect to your system from different devices with the same unique identifier. But how about unifying the device data within a big pool of device connections?

Advanced cross-device tracking will let businesses match their data by using extensive device pools consisting of all available unique identifiers and device information, as well as their mappings. As a result of cross-device tracking, businesses will be able to unify their visitor data between 13% – 40%, according to Webtrekk research.

These two technologies directly impact the marketing and operational costs of your business by merging data. Therefore, they will be an integral part of analytics solutions in the near future.

6. Data privacy and data ownership will become a must-have in vendor selection criteria

Data privacy and handling of private data have always been topics of concern. Recent news suggests, however, that it will continue to be a primary focus and will be even more strictly regulated than it already is.

Industrial espionage, governmental regulations and the enomrous penalties attached to those regulations are causing some angst. We can explore data privacy by breaking it down to the following subjects: “importance of data privacy for businesses” and “data privacy regulations for businesses.”

Cybercrime has grown rapidly in the last five years. One of the biggest weapons in a cybercriminal’s arsenal is PII, or personally identifiable information.

Businesses have started to engage in preventative measures to keep their data and, by extension, their customers safe. It is crucial yet not sufficient if only the corporations take responsibility and are held liable. Usually, a lot of third-party services and cloud technologies are involved in the mix; the data has to be shared with other parties in one form or another.

Tech giants such as Microsoft are acknowledging the gravity of this situation and taking steps toward new offerings to tackle cybercrime. For instance, Microsoft recently released Azure Stack, which offers similar levels of functionality and services to Azure, but which lives in on-premises servers to meet the demands of customers and requirements of data privacy. Analytics solutions are one of the most important and prominent services that businesses utilize. This is why data privacy will be more essential in the near future.

And then there are regulatory changes. The European Commission has already adopted data privacy regulations that will go live in May 2018. These regulations, spelled out in the General Data Privacy Regulation and the corresponding ePrivacy Regulation, will place much stricter requirements on how companies can collect and use data. The European Commission is not seeking to outlaw analytics, but they are introducing handcuffs that will limit the ways in which analytics solutions function, and the corresponding data is used.

7. Predictions, machine learning, AI and smart notifications are the next destination of the market

Predictions and machine learning enable actions that are virtually impossible to accomplish even with very large teams at your disposal.

There are lots of different ways to use machine learning algorithms in analytics tools. Those algorithms allow analytics customers to do predictive analytics, compare predictions and actuals, determine influencing metrics for specific goals, understand anomalies and give insights to customers to maximize their profits – just to name a few use cases.

Machine learning definitely has the potential to unleash all the capabilities of analytics tools in not-so-distant future.

Artificial Intelligence has recently become the focal point of a lot of debate and news coverage. For instance, Facebook AI was shut down after it started to create its own language. Additionally, a Chinese AI project started to criticize the political systembefore being shut down as well. Those were instances that clearly demonstrate the vast potential of Artificial Intelligence – and the risks involved.

Will AI have an impact on the analytics market soon? Quite frankly, there is no definitive answer to that question yet. But it sure looks like it will, especially in the Business Intelligence market. AI could be the ultimate consultant for business and marketing decision making.

In conclusion, the big data and analytics market is transforming rapidly. According to customer feedbacks, market research and market trends, we tried to present some our predictions for you. If you have any questions or comments about these predictions, please do not hesitate to contact me.