In June 2014, Masa K. Maeda, CEO of Valueinnova, Playcamp organizer and Conteneo Certified Collaboration Instructor, began work on an Agile Transformation project at the Ecuadorean office of the fourth largest telecommunication company in the world. As you'd expect, this company has a corporate presence in each country where it offers services and among all of its offices, the Ecuadorian headquarters was considered the most innovative by the senior leadership team.
“The agile transformation began with a very positive impact,” Masa relates, “spreading from 34 people in one department to more than 200 people in eight departments in only six weeks. This happened despite the fact that original contract was for the transformation of just one department.”
“The key to such an accelerated rate of adoption,” Masa continues, “was the ubiquitous introduction and widespread use of high collaboration frameworks (a.k.a. “serious games”) in the teams and at most levels of decision making.”
This initial success gave Valueinnova the opportunity to propose to the general manager that the company use Decision Engine and the collaboration framework “Buy a Feature/Budget Games” to prioritize the company's 2015 project portfolio, and Valueinnova’s proposal was accepted.
The company’s typical project portfolio prioritization process would begin in October and be complete in December. Each of the company’s twelve departments first prioritized its own project portfolio, which was comprised of 10 to 15 project proposals. The set of twelve prioritized project lists were then handed to a board led by the general manager.
“The board would then go through the painstaking and time-consuming task of merging all those projects to generate one project portfolio of around 140 projects!” Masa relate. “They also preserved the order of projects from each department. No project proposed by any department was rejected, save rare exceptions.”
The issues with the original process were:
- Resource and time consumption: Many employees and decision makers were involved for too long—they had to give up a good portion of their daily activities during the three month period.
- Quantity and quality repercussions: Because the company didn't make hard prioritization choices, they ended up with too many projects, causing some projects to be delivered late due to insufficient resources and other projects to be delivered with poor quality due to cutting corners.
- Local optimization: Since each department did its own project filtering, the board rarely rejected any projects, resulting in green-lit projects that had little relevance to the company’s bottom line. This localized optimization problem meant that some departments which should have been given more resources to grow faster were starved of their potential.
- Silo mentality: Each department focused on its own projects without knowledge or interest in the projects from other departments. This is also why the board only merged the departments’ portfolios and did no filtering.
- Failing economy: All the issues above ultimately had a negative impact on the overall profitability and economic viability of the company.
“By using the collaboration framework “Buy a Feature/Budget Games” and the online prioritization platform Decision Engine, we sought to minimize—and possibly eliminate—those issues,” reported Masa.
In the Beginning
“There was some hesitation,” Masa said, “because the teams were afraid that this would increase the time needed to create each project portfolio.”
However, creating the Business Model Canvases ended up saving time overall; the act of creating the BMCs collaboratively meant that the teams actually better understood each project and were able to eliminate irrelevant projects early on. The total number of projects in the portfolio of each department was reduced by 30% to 45%, Masa reports, so the total number of projects to be sent to the board was considerably smaller than in past years.
To make sure the person was focused on the most important needs of the business, each project was classified as either strategic or progressive during the second and third week of November. The progressive projects remained under the decision-making control of the departments while the strategic projects were elevated to be used in Decision Engine under the belief that collaborative prioritization among the department heads would produce the best overall choices for the company.
In the second week of December, each department generated a spreadsheet that included each project name, a one-paragraph description, and one paragraph indicating its benefits and compromises.
“I used these spreadsheets to prepare the three-round Decision Engine tournament,” said Masa. “I gave a copy of the list to all the managers and the board who were to participate, three days prior to the tournament for them to read and start getting acquainted with all the projects. In hindsight, I should have given them more time, but the schedule didn’t allow it.”
The day prior to the tournament, Masa organized two activities. First, the department managers and the board gathered together for a set of presentations by each department on its proposed projects. Each project was allotted 5-minutes (3 minutes for presentation and 2 minutes for Q&A). Second, everyone participated in a practice session using the online platform, Decision Engine, using dummy data to ensure everyone was comfortable with the platform and the game mechanics. “I wanted them to be able to focus entirely the prioritization activity,” reports Masa.
Masa also added two new elements to the process to gather even more data. The data analysis done after a Decision Engine forum typically compares the exhaustive data gathered by the online system (chats, bids, purchases etc.). Sometimes, the producers will also assign observers to work with the facilitator to record notes on participant behavior, which is very valuable information that influences the study for better results. In this case, Masa decided to add:
- Video and audio recordings of the sessions, and
- Heuristics based on fundamentals of Bayesian Statistics, to weigh variables taken from game observation such that applying the corresponding algorithm together with the game results would provide a better prioritization.
Room preparation began one hour in advance. In addition to Masa, the facilitation team included two volunteers with experience in high collaboration dynamics. One volunteer handled logistics, the other audio/video recording.
“We placed three session tables so that I could monitor all of them at the same time from a central table where I had 3 computers set up to facilitate the sessions,” detailed Masa. The team also had high-quality video and audio equipment set up to record each table. And they posted Prune the Product Tree posters on one wall, with large sticky notes printed with all the projects titles.
Everyone was on time and when the tournament began, all tables prioritized the first 50% of the projects in around 50 minutes.
“This first session started a bit slow,” Masa relates, “mostly due to discussion about the projects, and fortunately not due to the platform or game mechanics, demonstrating the benefit of the practice session done the previous day.”
The participants had a 15-minute coffee break at the end of the first session, so that Masa could set up the second forum. The participants then prioritized the remaining 50% of the projects in only 40 minutes.
“At that point I had to take the results of both games from all three tables and extract the top 10 projects to run the third session,” Masa reports. “We didn’t waste the time, however. While I set up the next set of forums, the two volunteers facilitated a Prune the Product Tree forum with all the participants to prune the entire project portfolio. I was ready to run the final prioritization session by the time they were done with the trees.”
The last Decision Engine forum took less than 30 minutes to complete, and all participants were able to leave earlier than scheduled. According to Masa, their familiarity with the projects was a huge contributor to more effective and proactive discussions. The discussions were also shorter because they focused on the value of the projects, rather than on understanding them.
Masa collected the video and audio recordings, and the Prune the Product Trees (thus pruned!) and returned to his hotel room to begin analysis.
“This was a very involved activity,” said Masa. “I had to listed to every recording very carefully and map the information onto relevant variables to apply my algorithm. This was rather dynamic, since the variables emerged from the observation itself rather than being pre-determined, but this made it more effective.”
“I also added the results of Prune the Product Tree as a variable. Criteria included aspects such as the order in which projects were being discussed and purchased, the level of interest, amount of participation and other for a total of 15 variables.”
Masa reports that the analysis consumed the better part of two days. Once the data mapping was done, he ran the algorithm over the data. “I was very pleased with the results, because with the exception of one project, all were in agreement with what I had learned and observed during the past weeks. There was no bias since I didn’t participate on the games, and the data feeds were based on the observation captured by the cameras, microphones and the Prune the Product Trees.”
Masa used the one project that was in a higher priority than expected as a point of verification by reviewing all the data related to it. He found that the data effectively gave the project higher ranking. He then proceeded then to write the full report.
Masa met with the team who helped him organize the Decision Engine tournament first, and they were amazed by the results and pleased with his explanation. Masa reports, “They were also surprised by the same project that I had surprised me. But they too agreed based on the data that its higher priority was correct.”
“The low esteem, so to speak, towards that project was because it wasn’t a sexy project. So while most people didn't care for it, it absolutely needed to be done because it had to do with external governance.”
The next step was to present the results to the board. They were very impressed by the quality of the results, the process itself, the fact that the entire process took less than three weeks, the reduced number of projects and the already obvious economic benefit that was taking place.
The department heads and those who participated in the prioritization were also very pleased. The teams that generated the business model canvases and their department’s portfolio, also related to Masa that the experience was fun and helped them truly understand the projects.
“The decision makers said that it was the first time in the history of the company that they truly understood all the projects, and truly collaborated,” said Masa. They even gave higher priority to projects that weren’t their own; whereas in previous years, it was a battle to defend their own projects.
Moral of the story? Using Decision Engine and collaborative prioritization to prioritize their annual project portfolio brought the best out in all of them.