Metrics

By now you should have realized how much I like metrics. In my first book, Startup Guide: how startups and established companies can create profitable digital products, I dedicated 6 entire chapters in addition to talking about metrics in the other chapters. In my second book, Product management: how to increase the chances of success of your digital product, there are 5 chapters dedicated to the theme of metrics, in addition to this theme appearing in other parts of the book again. It is no different here, the last two chapters talked a lot about metrics and I have also spoken in some other chapters about OKRs, objectives, data, and results.

Metrics are fundamental tools for any head of a product development team. Without metrics it is impossible to know if the team is progressing, evolving and fulfilling its mission. However, it is very easy to get lost in huge amounts of metrics.

In this chapter I want to share a little bit about how I usually work with metrics.

Metric types

I classify metrics into 3 major groups:

Internal metrics

These are the metrics that show how the product development team is at the moment and has evolved. The previous two chapters showed the metrics of productivity and quality. These are metrics that help the head of product and the team to understand how the work process is doing and where they should focus the energy to improve on these metrics. How can we increase productivity? How do we decrease the number of bugs? How do we guarantee the best product performance for our users?

Still within the theme of internal metrics, there are others that are more “soft” but also very important for you to understand the health of your product development team. They are metrics that help you understand if people are happy working in this team, if they are aligned with the culture and purpose of the team and the company.

A very simple metric to follow is the entry, exit and average time of the people of the team each month. If more people leave than enter, there may be a problem with the team. If people stay a few months and then leave, it is another point of attention. You can also run an NPS survey (Net Promoter Score) with team members periodically to understand whether they would recommend working on your team for others and why they answered this way.

The following are two examples from Gympass. The first is the evolution of the total number of people on the team month by month, with total new and total people who left, also showing how many of those left voluntarily, that is, asked to leave. The second is the eNPS (employee NPS), which shows whether people were willing to refer the Gympass product development team to their friends to join us.

volution of the Gympass product development team
NPS from Gympass product development team

User metrics

These are the metrics that show that your product is being used by users, that is, that it is fulfilling its objective. These are the metrics of engagement and user satisfaction with the product. What is your user’s ideal engagement with the product?

At Conta Azul we wanted the product to be the first window he opened in his browser in the morning and the last he closed at night. We tracked the number of invoices issued per user per week, per month, and the amount of the invoices. At Gympass, we monitored how many users went to the gym per month, how often they visited gyms, and so on. At Lopes, we are monitoring the use of the new CRM by brokers. We want them to use it at least 4 times a week and that’s the number we’re tracking.

Another user metric that can be useful is satisfaction metrics. This metric tends to be a little more subjective, in addition to being laborious to measure. You should send a mini-survey to a portion of your customers every day. Therefore, before measuring and monitoring it, I recommend closely monitoring the engagement with the product. If the user is engaged, using the product at the expected frequency, there is a good chance that he will be minimally satisfied.

Business metrics

These are the metrics that show the product development team is actually delivering value to the business owner. What was the goal that the business owner had for the product? Increase sales, decrease costs? These goals vary with the type of company where the digital product is located. Is it a digital company, a traditional company, or a traditional born-digital company?

At Conta Azul, as the product sold was the digital product, user metrics and business metrics mix a lot. The number of users that use several times a week are those that will certainly continue to pay the monthly subscription and, consequently, will continue to generate revenue.

At Gympass, a traditional company born-digital, and at Lopes, a traditional company, revenue exists without the need for a digital product. So, what can the digital product do to increase revenue or decrease costs? At Gympass, at the same time that we wanted to reduce operating costs by automating most manual tasks, we also used the product as a revenue enhancer, helping the HR of our customers and their employees to understand and, consequently, become subscribers to the service. At Lopes, the main focus is on increasing sales, but we also have a focus on how to lower operating costs.

A very important point to monitor every month is the comparison between the value delivered by the digital product – revenue increase and cost reduction – and the cost of product development. What is expected is that the value delivered will be greater than the development cost. And managing this is the role of the head of product.

Tracking metrics

Metrics can be tracked daily, weekly, monthly, quarterly and yearly. Of course, the longer the update period, the more difficult it is to understand the metric’s behavior and make decisions about how to impact it. I prefer metrics that we can monitor daily and weekly. With weekly metrics, in a quarter we have 13 opportunities to evaluate a metric, understand how we can work on it and remove any impediments that are hindering us in reaching any objective linked to it. And the daily metrics give the pulse of the business. How many new users per day? How many users canceled?

At Locaweb, we always followed these numbers daily and, if something was out of the expected, positively or negatively, we tried to understand what had happened that had impacted the number. When we did something with the intention of changing these numbers, such as a new marketing or retention campaign, we could monitor these results daily. It is even possible to measure even more frequently in the case of products with large scale, such as Google Search.

On the other hand, there are metrics that are less frequent, such as the example above that I gave of new people joining or leaving a product development team. Even with monthly metrics or less frequent ones, I recommend monitoring the partial of this metric at least weekly. If you only follow up monthly, in a quarter you will have only 2 opportunities to assess progress and correct the course.

Summing up

  • The metrics of a product development team can be classified into 3 major categories: internal, user, and business metrics.
  • Internal metrics show how is the health of the team. User metrics show the relationship of the user with the product. Business metrics are those that show whether the product is, in fact, delivering value to the business.
  • Metrics should be monitored as often as possible, at least weekly. Even if it is a monthly metric, try to follow the weekly or even daily partials of this metric, so you can have more opportunities to act earlier when there is a course variation.

In the next chapter, we’ll look at how to manage relationships with different areas.

Digital Product Management Books

Do you work with digital products? Do you want to know more about how to manage a digital product to increase its chances of success, solve its user’s problems and achieve the company objectives? Check out my Digital Product Management bundle with my 3 books where I share what I learned during my almost 30 years of experience in creating and managing digital products:

  • Startup Guide: How startups and established companies can create profitable digital products
  • Product Management: How to increase the chances of success of your digital product
  • Leading Product Development: The art and science of managing product teams

Measuring and managing quality

In 2015, we decided to extinguish the Quality Assurance (QA) function of our Locaweb product development team. We had 12 QAs, some with a developer profile and others with a SysAdmin profile. In proposing the extinction of the QA role, some of the QAs became devs, while others took on the role of system administrators. The reasons that led us to extinguish Locaweb’s QA function were:

  • When there is a QA function separate from the software development function, it is common to hear phrases like “the functionality is ready, now it is in the QA phase”, which denotes a cascading product development culture. This culture can considerably increase development time and negatively affect the quality of the software.
  • When there is a QA function separate from the software development function, it is also common to hear phrases like “why didn’t QA detect this bug?”, Which denotes a culture of finding the culprits. This culture can be very detrimental to the engagement and motivation of the team and, consequently, negatively impact the quality of the software.
  • Quality should not be the concern of a person or team, it should be the concern of everyone working on creating the software.
  • Quality is a non-functional requirement, that is, it specifies a criterion to evaluate the functioning of a digital product, while a functional requirement specifies a behavior of the software. Performance, scalability, operability, monitorability are some examples of non-functional software requirements that are just as important as quality. Even so, there are no defined functions to guarantee performance, guarantee scalability, guarantee operability and guarantee monitorability. Why is quality the only non-functional requirement that has a specific dedicated function to guarantee it?
  • Quality control focuses on ensuring the quality of the software development process. If a separate function is needed to guarantee this quality, why is there no need for a separate function to guarantee the quality of the product management process, the design process, the product marketing process, the sales process, the finance processof a company?
  • There was a concern that, if the engineer himself now had to test, deliveries would take longer to be ready. This concern existed because the engineers considered that their work was finished – and the delivery was ready – when they passed the story to the QA to test. However, the engineer’s definition of ready is incorrect, as he has just passed the story on to the next phase, the test. From the user’s point of view, the story is only ready when the user can use the new feature. Therefore, the time that the delivery remains in quality control is still development time, even though it is no longer in the hands of the engineer. And that time gets even longer when the story goes through quality control, but it is rejected and needs to go back to engineering.

When I joined the Conta Azul, they had also just extinguished the role of QA, and the ex-QAs became business analysts, mainly helping product managers.

I saw other companies also discussing the need for QAs, and in some cases, a heated debate emerges around this topic. However, having or not having QAs should not be the focus of the discussion. Having or not having this function is the solution to a problem, usually referred to as “how can we improve the quality of our product?”, And that problem should be the center of the discussion.

How can we improve the quality of our product?

A simple Google search for software quality will yield tons of definitions typically related to meeting functional and non-functional requirements. When the software does not meet a functional or non-functional requirement, it has a defect, a bug. Therefore, to improve the quality of a software product, we need to work on two things:

  • reducing existing bugs;
  • not generating new bugs.

A good way to track this is to have a weekly measurement of your bug inventory and new bug creation per week and discuss this every week with the team. We did that at Gympass. We define at the beginning of each quarter what the target is for the bug inventory and the average new bugs per week.

Total amount of bugs at Gympass

The image shows the evolution of our bug inventory during the 2nd quarter of 2019. We started the quarter with 215 bugs in our stock and we aimed for a target of less than 166 at the end of the quarter, a reduction of almost 23%. We ended the quarter with an inventory of 136 bugs, a reduction of 36%. We did this by focusing not only on resolving bugs in our inventory, but also on controlling the number of new bugs per week.

Number of new bugs detected per week at Gympass

In the first quarter of 2019, we had an average of 26.2 bugs created per week. During the second quarter, we reduced this average to 17.4 new bugs per week, to a total of 226 new bugs during the quarter. This is a 33% reduction in the number of new bugs per week.

That looks like a pretty good improvement, right? But there is a lot of room for improvement there. Let me explain the math of bug management:

If we were able to reduce our bug stock from 215 to 136, it means that we have resolved at least 79 bugs. However, we created 226 new bugs (17.4 new bugs per week x 13 weeks) during the quarter. So we solved 79 + 226 = 305 bugs during the quarter, a lot of bug fixing work. If we had generated 90 new bugs during the quarter, an average of 6.9 new bugs per week, instead of the 226 new bugs, we could have zeroed out on the bug inventory.

An additional aspect of the bug resolution to be measured is the SLA (Service Level Agreement) resolution, that is, how many days the team took to resolve a bug from the day the bug was first identified. For this, we classify the bugs by their severity, which is the impact it has on users and the business. The most serious bugs are those that we need to resolve on the same day; high severity errors, in 7 days and average severity, in 14 days. The following chart shows how we were at Gympass in the fourth quarter of 2019.

Gympass bug resolution SLA

This is not the ideal way of viewing this info because it shows only an image of the moment, and not an evolution. To understand the evolution of any metric, you need to see how it did at different points in time.

As soon as I joined Lopes, I started bringing this topic up for discussion with the teams. One of the things we noticed is that 50% of deployed items were bug fixes. I was informed that “these bugs were caught before going into production, which is a good thing”. In fact, it is a good thing that these bugs did not reach the production environment and appeared to our users. However, they reached pre-production and needed to be corrected. Wouldn’t it be better if these errors didn’t even exist, not even in pre-production?

The OKRs we defined to help us with the quality theme were 3 additional KRs in order to Increase the cadence of deploys in production that I mentioned in the previous chapter:

  • KR: Reduce the number of new bugs to 5% in pre-production.
  • KR: Reduce the number of total bugs to 10% in pre-production.
  • KR: Keep the number of total bugs below 5% in production.

And we add the following OKR:

  • Objective: To improve the quality of the deliveries to the squads.
  • KR: Review 100% of new stories to find poorly defined and / or ambiguous requirements.
  • KR: Perform a 25% review of the Pull Requests of the squads.
  • KR: Measure the Pull Requests volume of the squads.

Another example of bug tracking

At Conta Azul, we doubled the product development team in a period of 8 months between November 2017 and July 2018. This growth was aimed at increasing the team’s productive capacity.

Number of deliveries and people per week at Conta Azul

In addition, we divided the quantity of deliveries by the total number of people on the team to assess whether we were managing to increase our productivity individually.

Number of deliveries per person at Conta Azul

However, with the increase of people on the team, it ended up increasing the amount of bugs. So much so that the team that had already had 40% of its deliveries as a bug fix ended up having to increase this proportion to 60%. That is, despite having increased individual and total productivity, this increase in productivity was not being felt by the user, as it ended up being used for bug correction.

Percentage of bug fixes at Conta Azul

To control this problem, we increased our focus on fixing these bugs within the SLAs, which were:

  • 85% of tickets resolved within 7 days
  • 98% of tickets resolved within 30 days
7 day bug fix SLA at Conta Azul
30 day bug fix SLA at Conta Azul

See that the quality has worsened and the customer suffered from it. But, after some time, we managed to return to the defined SLA levels. We looked at this metric weekly and, whenever we discussed this metric, we agreed that the best way to comply with the SLA was not to create bugs!

Quality is not just bug control

In addition to bug control, there are several other aspects that impact the quality of the digital product that we deliver to users. Performance, scalability, operability, monitorability are some examples of non-functional requirements.

When I joined Gympass, on my second Monday the system went down for users around 7 pm. I started asking people on the team what was going on and the answer was that Mondays are peak days in terms of gym visits and that the system could not handle the volume at all. As there was no monitoring, we were not alerted that the volume was higher than usual and we were unable to prepare properly. Two months later, when Rodrigo Rodrigues joined Gympass as CTO, he dubbed the event “Black Mondays”. To address the problem, we started to monitor and implement an infrastructure that could handle Monday peaks. And we set OKRs for uptime, successful HTTP requests and backend response time.

Uptime – Gympass
Successful HTTP requests – Gympass
Backend response times – Gympass

Why is quality so important?

Any user prefers to use a good quality product that behaves as expected. This is a sine qua non condition to provide a good user experience.

In addition to the user experience, there is another important aspect to consider when we talk about quality and bugs. Whenever someone needs to work on resolving a bug that was found in a digital product, that person needs to stop working on whatever they are currently working on in order to resolve the bug. This is an interruption in the workflow. If that person were able to deliver the software without that bug, they could continue to work on new things without interruption, which would make them more productive.

The relationship between productivity and quality

I had the opportunity to participate in an MIT course on how to create high-speed organizations. The course was taught by Professor Steven J. Spear, author of the book The High-Velocity Edge: How Market Leaders Leverage Operational Excellence to Beat the Competition. This is one of those very dense courses, full of content, but which can be summarized in one paragraph:

High-speed organizations are able to learn very quickly, especially with their failures, and to absorb that learning as an integral part of the organization’s knowledge.

A high-speed organization works by following the 4 steps ahead:

  • Be prepared to capture knowledge and encounter problems in your operation.
  • Understand and solve these problems to build new knowledge.
  • Share the new knowledge with the entire organization.
  • Lead to develop skills 1, 2 and 3.

The classic example is Toyota, with lean manufacturing and the concept of stopping production whenever there are failures, correcting them and using them as a learning opportunity so that they no longer happen. This ability to learn from failure is what gives Toyota the ability to stay ahead of its competitors for so long.

Another good example is Alcoa, which had a work incident rate of 2% per year, which was considered normal. Alcoa has more than 40,000 employees, so 2% of work incidents per year means that 800 employees per year have some type of work incident. This is a very impressive and worrying number.

To combat this problem, they implemented a zero error tolerance policy. Before implementing this policy, mistakes were seen as part of the job. Employees are now encouraged to report operational errors within 24 hours, propose solutions within 48 hours and report the solution found to their colleagues to ensure that knowledge spreads throughout the organization.

This caused the risk of incidents to drop from 2% to 0.07% per year! This reduction in the incident rate meant that fewer than 30 employees per year had any work incident problems after the zero error tolerance policy was implemented and Alcoa achieved an increase in productivity and quality similar to that of Toyota.

Fail fast vs. learn fast

An important factor in the Toyota and Alcoa examples is that recognizing and learning from failures must be part of the company’s culture. This is somewhat more common in the culture of technology companies, but not so common in traditional companies. During the MIT course I shared a table with a Brazilian executive from Grupo Globo, the major TV company in Brazil, a Spanish executive from AMC Networks International (Walking Dead, Breaking Bad and Mad Men), a German project manager living in Azerbaijan who works for Swire Pacific Offshore (oil and gas industry) and an MIT postdoctoral student in solar energy from Saudi Arabia.

All of my table mates were from more traditional industries. I was the only one at an internet company. The executives at Globo and AMC were there because they saw Netflix with its streaming video on demand and YouTube with its huge catalog of user-generated videos as major threats, stealing their audience very quickly and they wanted to understand how they could defend themselves.

Although the theme is somewhat obvious for internet companies, especially with the culture of technology startups that value fail fast. That’s what makes Netflix and YouTube a threat to traditional media companies like Grupo Globo and AMC Networks. However, even though this is part of the culture of internet companies, sitting and discussing it with people from more traditional companies was a great opportunity to reflect on the relationship between failure, failure recognition, learning and high speed:

  • Recognizing the flaws and using the flaws as a learning opportunity must be well rooted in the organization’s culture. If people are not careful, as a company grows, it may lose the ability to accept failures as learning opportunities. It is very common for companies, as they grow, to become more and more averse to failure and to create a culture that ultimately encourages people to hide mistakes and failures.
  • Another important aspect of learning from failures is to make this process a company standard. There is no point in failing, acknowledging the error, stating that you will no longer make that mistake and, some time later, making it again. This learning process with failures must be part of the company’s culture. Whenever a fault is identified, learning should happen as quickly as possible to prevent it from happening again. If the same failure happens again, something is broken in the learning process with the failure.
  • Even in Internet companies, I realize that learning from failures is more common in the product development team, as retrospectives and continuous learning are part of the culture of agile software development. In other areas of the company, learning from failures is less common. This ability to systematize learning from failure must permeate the entire company.

Even though we hear a lot about the culture of internet companies to fail fast, talking about failing fast diverts our focus from what is really important, learning fast. We must put our energy into learning, not into failure itself. It is the learning process that makes people and companies evolve. And it is the ability of an organization to learn fast, especially with its failures, that will allow it to move at really high speeds.

Summing up

  • Questioning whether or not product development should have a dedicated QA team is not the right question.
  • The problem you are trying to solve is how to improve the quality of your product.
  • A good quality proxy metric is the bugs. Bug inventory, new bugs per week and bug resolution SLA.
  • A product development team must have all its members following these metrics and taking action to improve them.
  • Managing bugs is not enough to manage the quality of the digital product. Performance, scalability, operability, monitorability are some examples of non-functional requirements that directly impact the quality of the digital product.
  • Quality is at the forefront to provide a good user experience. In addition, it is essential to increase the speed of your product development team. The less bugs a team has to fix, the more time it has to focus on new things.
  • High-speed organizations are able to learn very quickly, especially with their failures, and to absorb that learning as an integral part of the organization’s knowledge.

In the next chapter we’ll see a little more about metrics.

Digital Product Management Books

Do you work with digital products? Do you want to know more about how to manage a digital product to increase its chances of success, solve its user’s problems and achieve the company objectives? Check out my Digital Product Management bundle with my 3 books where I share what I learned during my almost 30 years of experience in creating and managing digital products:

  • Startup Guide: How startups and established companies can create profitable digital products
  • Product Management: How to increase the chances of success of your digital product
  • Leading Product Development: The art and science of managing product teams

Measuring and managing productivity

How can we deliver faster? How can we deliver more with the same team? Why do we have the impression that the team is slow? When the team was smaller, it seemed that it could deliver more. These are very common questions and statements that I hear about product development teams. Every company that has a digital product development team would like this team to be faster. In this chapter, I will show you the two essential tools to achieve faster and more productive teams.

Measurement

There is no way to improve something that is not measured. What is product development speed? It is important to have a clear definition of this metric and have it measured.

In my last year at Locaweb, we were focusing a lot on productivity, that is, on how Locaweb’s product and software development teams could produce more, without having to add more people to the teams, and without dropping quality levels.

The following graph shows our numbers. We count quantities of deliveries per week and, as you can see, in a few weeks we were able to more than quadruple the quantity of deliveries per week.

Amount of deliveries per week at Locaweb in 2016

This increase in productivity happened when the team grew only 10% in number of people, so it is not possible to credit this increase in productivity to the increase in number of people in the teams.

When there is such an increase, in addition to the natural question about whether the increase in productivity is due to the increase in the number of people in the teams, another question that exists is whether there was a drop in the quality of deliveries. One of the quality measurements we had was the number of rollbacks. As you can see below, even with the increase in productivity, the number of rollbacks was reduced by 40%!

Number of rollbacks per week at Locaweb in 2016

At Locaweb we made estimated calculations of deliveries per week from September 2015 to February 2016. The calculation was very simple, total deployments made in the period divided by the number of weeks. We then started to communicate the entire company about the week’s deliveries.

After I arrived at Conta Azul, we decided to implement the same type of control of weekly deliveries and ended up also achieving a good increase in productivity.

Amount of deliveries per week at Conta Azul in 2017

Both at Locaweb and Conta Azul, each product manager sent me the deliveries of the week on Friday, I compiled the data and wrote down the quantity for each week, generating these graphs. From the moment we started measuring, it became clearer the level we were at, and the actions we started to take in order to increase productivity began to show results on the graph. In addition, the teams started to use a single measurement tool, JIRA, which gave them a better view of each team’s progress and allowed comparisons with the exchange of experience, that is, something like “look how interesting your graph is, how did you manage to increase this indicator? ”.

At Gympass, as we scaled the team very quickly, we decided to control the number of deliveries per person per week. We considered people who joined 2 months before since people need 1 to 2 months after they joined Gympass to become productive. In one quarter, we managed to increase our productivity per person by 16%. The number of deliveries was extracted directly from JIRA.

Amount of deliveries per week at Gympass in 2019

At Gympass, we also measure the number of deploys, both in our core, aka monolith, and in microservices. We also achieved a considerable increase in one year.

Number of deploys per week at Gympass in 2019

At Lopes, as soon as a deploy was done, an email was sent with a list of deployed items. One of the first things I did when I joined the company was to compile these reports in a spreadsheet to build the chart below. Hence it was easy to notice that the deploys did not happen every day. They happened once per week on average. As soon as we noticed this, we defined OKRs to increase the frequency of deploys, which has been yielding results. The OKRs we defined were:

  • Objective: Increase the cadence of deploys in production;
  • KR: Increase the number of deploys per week to at least 3 (the more the better);
  • KR: Reduce the maximum number of new features per deploy to a maximum of 10.
Number of deploys per week at Lopes

What impacts productivity

The productivity of a product development team is impacted by several factors. I once found a very interesting article written by Michael Dubakov, founder of the company Targetprocess where he shows a mind map with all the elements that can positively or negatively impact the productivity of a product development team. This article is no longer available, but thanks to the Wayback Machine website (http://web.archive.org), you can access it at:

http://web.archive.org/web/20150827162352/http://www.targetprocess.com/articles/speed-in-software-development/

The mind map is this one:

Number of deploys per week at Lopes

This diagram shows things and activities that affect the speed of development in some way. Green means that the activity increases speed. The more you have, the better. Yellow indicates that there is some maximum. For example, you can accumulate technical debt and increase speed, but if you accumulate too much, it will significantly delay you. Red shows things that slow development, the less you have, the better. The green arrow indicates an increasing effect. For example, focused work increases the speed of development. The red arrow indicates a decreasing effect. For example, better development skills decrease the complexity of the system (good engineers create less complex systems).

What I like about this image is that it shows how complex this theme is and how many things can positively or negatively impact the team’s speed. At Conta Azul, we followed this topic every quarter at the Product Council, a meeting where we talked about the quarterly planning of the product development team with the leadership. There was a slide where we listed all the topics that could impact the speed to discuss what we were doing about each of these topics.

Themes that impact the speed of the Conta Azul product development team

Place the topic of productivity at the center of the discussion

There is no silver bullet, with each team I’ve worked on there were several actions we took and we were always sure that there are always more actions that could be taken to increase productivity even more.

The only silver bullet that exists is we made productivity into an important topic in our conversations. Everyone started talking about productivity and what we could do to improve it.

This movement made us initiate several changes and experiments that helped us to increase our productivity considerably. If you also want to increase the productivity of your product development team, place this topic at the center of your conversations and experiment a lot. You will see how there is room to greatly improve the productivity of your software development team.

Another important point: be sure to discuss the topic of productivity frequently. My recommendation is that you do it weekly. Creating a weekly cadence will give you the opportunity, each week, to experiment with something new and discuss the results with the team.

Summing up

  • There is no silver bullet to increase the productivity of a product development team. However, there are two essential tools to help achieve this goal.
  • The first tool is measurement. There is no way to improve something that is not measured. What is product development speed? It is important to have a clear definition of this metric and its measurement.
  • The second tool is to bring the topic of productivity to the center of the discussion. Everyone on the product development team is responsible for the team’s productivity. Therefore, by bringing the topic to the center of the discussion, everyone will be able to collaborate to improve productivity.

In the next chapter we will see another metric that has a direct impact on productivity, quality.

Digital Product Management Books

Do you work with digital products? Do you want to know more about how to manage a digital product to increase its chances of success, solve its user’s problems and achieve the company objectives? Check out my Digital Product Management bundle with my 3 books where I share what I learned during my almost 30 years of experience in creating and managing digital products:

  • Startup Guide: How startups and established companies can create profitable digital products
  • Product Management: How to increase the chances of success of your digital product
  • Leading Product Development: The art and science of managing product teams

Vision, strategy, objectives, and team structure

This is the beginning of Part III of the book, about Tools, I’ll talk about the tools I’ve been using in my almost 30 years of product development leadership career and passing them to other leaders so they can use them with their teams. The tools I’ll talk about include vision, strategy, objectives, metrics, relationships, hiring, feedback, and ceremonies.

You will notice that the tools that I’ll comment on here are not software such as spreadsheets, presentations, documents, Slack, JIRA, Confluence, etc. These apps are normally very useful, but they are only a vehicle for documentation, communication, and metrification of the tools that we will see in the following chapters.

Vision, strategy, objectives, and team structure

All of these items are, in addition to very important concepts, essential tools for any head of product. In all the opportunities that I started as head of product, these were the first topics that I dealt with, always starting with a product vision and then moving on to the themes of strategy, objectives, and team structure. This is also my first focus when I start any advisory work. I try to understand what’s the vision, strategy, objectives, and team structure. If any of these elements is missing, I help people in the company to create them.

I have already explained what they are and how to create each of these items in their respective part I chapters, so I will just do a quick review of these tools.

Vision

As I explained in the chapter Product vision, to make the product vision, it is necessary to be clear about the company’s objectives with the product, as well as to deeply understand the problems and needs that customers have and that will be solved by the product. The 6 steps to build a product vision are to obtain strategic objectives of the company, gain an understanding of the problems and needs of customers, design the first version of the vision, iterate and refine, communicate and review.

I usually document and communicate the product vision in a presentation. If necessary, I put some theoretical introduction explaining the concepts of platform and marketplace. In every presentation I make, I usually present the product vision as an introductory topic to set the context and to make it clear to everyone.

Strategy and objectives

The product strategy is nothing more than the path you will take to reach your product vision. To create your product strategy you need to have a good understanding of your market, that is, the competitors, the potential and accessible market, the growth of that market, if there are disruptors, and how it is regulated. You also need to understand your strengths and weaknesses, opportunities and threats. A SWOT analysis can help. With this information in hand, you define what you and your team will do to achieve the product vision, what goals you need to achieve, and what metrics will tell you that you are achieving those goals. OKRs are a great tool for working on your goals and metrics.

There are some books and courses talking about how to define and use OKRs. So I’m not going to go into too much detail here. In a very succinct way, you define together with the team some objectives for a period – usually a quarter or a year – and define which metrics you will use to show that the objective is being reached.

I usually document the OKRs in spreadsheets, which I use to follow the evolution every week with leaders of my product development team, and also present and discuss the OKRs with other leaders and areas of the company. The following is an example:

Exemple of an OKR worksheet

Note that every week the KRs are updated. Each team leader updates their KRs with their teams every Monday, and then leaders and heads of product go through the KRs to see how they are doing and if there are any impediments where energy can be put in to remove. I like to do it on Monday because it helps the team organize the work of the week. The weekly cadence is essential to help the team review performance at least weekly and make adjustments if necessary. If the cadence is higher (biweekly or monthly), valuable opportunities to correct problems and remove impediments are lost.

The column T0 indicates the initial value of the metric. The responsible column has the name of the person who is responsible for that KR and who will lead the effort to make it happen. And the support column lists the people or areas that will help the person in charge.

It is worth remembering that the closer to the company objectives and business metrics the OKRs are, the better it will be for the team, as they will be working to help the company with its objectives and results.

Team structure

In the chapter Team Structure I said that product development teams are organized into minimal teams, also called squads, composed of engineers, product designers, and product managers. It is important to keep the team as lean as possible to help your productivity. These minimum teams are grouped into product teams called product tribes.

There are 4 ways to organize product teams: by product or functionality, by type of user, by journey or by objective. You can also use two different types of organization to create a hybrid organization. There are also the structural tribes, which create the necessary structure for the product tribes to perform. Teams that make up the structural tribes are SRE / DevOps, Data, Architecture / Tools / Foundation, Design Ops, Information Security, Internal Systems, Sales Engineering and Professional Services.

To help organize the team, I usually use a simple spreadsheet template like the following:

Example of team structure worksheet

This worksheet contains the team structure and the people who are part of it. Note that we are not documenting functional leadership, but the leadership of each team. In the example, we have John as GPM leader of PM Lucy and PD/UX Patricia, Mary as GPM leader of PD Rafa and Sandra as leader of the structural teams of SRE / IT, Data and Peter, who leads the engineering of 2 squads of the tribe A and tribe B squad.

An important point is that it is not enough to just create these elements and then not use them. These tools are useful the more you use them. I use OKR worksheet at least every week. Whenever I have the opportunity I talk about Vision and strategy. Whenever I talk with my leaders about hiring or changes in the team, I use the team structure spreadsheet I showed above.

Summing up

  • Vision, strategy, objectives and team structure are, in addition to very important concepts, essential tools for any product head.
  • For vision and strategy, a presentation with a few slides is sufficient. For OKR and team structure, spreadsheets do the trick.
  • More important than the software you use to document Vision, strategy, objectives and team structure is what you do with these tools. I use OKR worksheet at least every week. Whenever I have the opportunity I talk about Vision and strategy. Whenever I talk with my leaders about hiring or changes in the team, I use the team structure spreadsheet I showed above.

In the next chapter, we’ll look at how to measure and increase the productivity of a product development team.

Digital Product Management Books

Do you work with digital products? Do you want to know more about how to manage a digital product to increase its chances of success, solve its user’s problems and achieve the company objectives? Check out my Digital Product Management bundle with my 3 books where I share what I learned during my almost 30 years of experience in creating and managing digital products:

  • Startup Guide: How startups and established companies can create profitable digital products
  • Product Management: How to increase the chances of success of your digital product
  • Leading Product Development: The art and science of managing product teams