People Analytics Primer: Leadership Analytics

Introduction

Cole Napper

Welcome to “People Analytics Primers”. In this series, I plan to dive deep into a technical and tactical people analytics topic and discuss how I’ve used it in practice. These thoughts are my own and do not reflect my employer.

The second installment of this series is: leadership analytics and the root causes of poor leadership.

Leadership

As a people analytics practitioner, how would you know what the most important thing to measure at your company? Is it headcount? Is it turnover? Something else? One variable that should be measured and has an outsized impact on organizational performance is leadership. Leadership is arguably the most important human capital investment your organization is making. Books like the Leadership Capital Index by Dave Ulrich provide methodology for quantifying leadership in a way that makes sense to investors — which is not the focus on this article. Yet, as the adage goes, “what’s measured gets managed”. Which begs the question: How do you measure leadership?

Definition

To define leadership we must first ask a few questions. What is leadership? What is management? What does it mean to be an effective leader? These are surprisingly tough questions to answer. Before a variable can be measured it must first be operationally defined, and internal stakeholders must agree with the definition that is chosen. Most people conflate management and leadership. I see them as distinct concepts. Management is anything related to the selecting, developing, and retaining of team members, the prioritization of work/tasks, and ultimately team performance. It is the technocratic side of being a people manager. Leadership, however, involves more subtle behavioral characteristics, such as team motivation, influence and sway, team engagement, communication, and the interpersonal relationships involved in being a people manager. That’s why it’s possible to be a leader without being a people manager. From my estimation, that also makes it possible to be a good manager, but not a good leader as well.

Another way of thinking about leadership is describing what constitutes poor leadership. A recent Freakonomics podcast reignited the debate around the concept of the “peter principle”, based on the findings of a recent research study. For those who aren’t familiar, the peter principle states the “employees will get promoted to their highest level of incompetence and then stay there”. It’s a funny anecdote, but the study found evidence to support it being true in the real world. To summarize their findings, if you promote someone solely for being the highest performer, that person is more likely to underperform as a manager. Therefore, this phenomena spreads throughout organizations until all the high performers get promoted into the jobs they are unequipped to do. That’s a sobering thought.

Findings such as this suggest the need for a separate promotional track for strong performers rather than just leadership. Many companies already do this, especially those in highly technical fields. Those organizations have separate management and technical tracks for employees to allow for high performers to never have to manage people if they aren’t equipped to do so. What I haven’t seen is companies measuring the employee lifetime value (ELTV) of the technical track members. This seems like a slam-dunk people analytics endeavor. Upon completing the ELTV analysis, the highest level of the technical track could be capped based on the highest level of ELTV that those performers bring to the organization via their strong performance.

Measurement

How do you measure leadership? Is it behavioral? Is it an innate, ineffable quality of a select few? Is it only in the eye of the beholder? For a while, I really struggled with this question, and even quit believing leadership existed for a short period, due to the difficulty of pinning down how to measure it. That said, I’ve gotten over it, so let’s discuss practical measurements of leadership and management. I’d like to give Cultivate (recently acquired by Perceptyx) credit for introducing a novel way of measuring leadership behaviors and employee treatment via their communication behaviors (admittedly, Cultivate only measures manager behavior in online modalities, but it is an impressive tool nonetheless). You should check them out if that interests you.

Another way of measuring people manager capability and effectiveness is with a Manager Quality Index. We’ll get to the details of the Index in a second, but first it’s important to understand how Manager Quality fits into a broader corpus of organizational impact. My working model of how to assess the impact of managers on the business is called the Management Value Chain seen below.

Management Value Chain

Let’s go through each step:

• Manager Selection — This step could be an article unto itself, so I’m not going into detail here. Manager Selection involves any type of assessment (e.g., internal promotions, rotations, and/or lateral movement, or external talent acquisition) that lead to an employee moving into management.
• Manger Quality — We will dive deep into creating a Manager Quality Index shortly.
• Team Performance — Very few organizations are assessing the quality of their managers, but the ones that do, usually start by trying to quantify the outcome of a manager’s Team Performance. Team Performance is important to know, yet it’s also an outcome variable (i.e., lagging indicator). If you want to influence the outcome one must first look at leading indicators that predict the desired outcome. It’s also important to note, that Team Performance aggregated across all the teams in the organization is what causes Organizational Performance, which is the most important outcome measure of any in the organization.

Manager Quality Index

So how does an organization assess manager quality (presuming the organization already validated the effectiveness of their Manager Selection process)? Manager Quality is a proximal measure, with the ultimate measure being Team Performance which should be measured with objective measures specific to the team in the short, medium, and long term. Here are the basic elements of Manager Quality that can assist your organization in quantifying manager effectiveness.

Manager Quality — Basic

• Retention — Managers should be influencing the voluntary (regrettable) and involuntary (due to poor selection) attrition of their team in a positive way. This part of the index should calibrate managers against the baseline attrition rates of a function/department in which they reside, so that being a manager in a high-turnover function doesn’t overly penalize that manager or vice versa.
• Team Engagement — Managers should be influencing team performance via the engagement of their direct reports. This part of the index should be tenure-adjusted for the employee tenure of each direct report, due to the U-shaped relationship between tenure and employee engagement. That way, managers aren’t rewarded for having a bunch of highly engaged newbies (the honeymoon effect) or penalized for having a bunch of two-year tenured curmudgeons.
• Upward Feedback — Managers should be influencing the followership of each team member via the managerial & leadership behaviors they demonstrate. An organization can modify the managerial behaviors specific to their organization, but Google freely gives away a great starting point via their open-sourced upward feedback survey. The survey results should be corrected for any leniency or strictness bias of the followers/direct reports.

Manager Quality — Upgrades

• Hiring Skill — Managers should be influencing the quality of their team via their skill at assessing talent in the hiring process. Interviewer quality is assessed using behavioral based interviewing with a scoring guide to track how hiring managers and interviewers rate each candidate. Hiring manager’s ratings should then be linked to the future performance of the candidate on the job as an indicator of the hiring manager’s ability to judge talent.
• Manager Interaction Behaviors (Communication, Collaboration, & Influence) — Managers should be consistently interacting with their direct reports, other leaders, and stakeholders of the team. Organizational network analysis (ONA) can be used to model the positive communication behaviors of a manager with their direct reports, how the manager facilitates collaboration within the team and between teams, and the manager’s level of influence in the organization via metrics such as network centrality.
• Psychological safety — Technically, psychological safety could fall within the upward feedback section of Manager Quality. However, as Google’s Project Oxygen indicates, psychological safety is a key differentiating factor of strong leadership and team performance. In terms of measurement, I prefer Ethan Burris’ work on employees’ ability to speak freely and bring their ideas to the table without fear of reprisal as a way of indicating psychological safety.

Manager Quality — Outcome

Team Performance, and by extension organizational performance, is the outcome of managerial effectiveness, given that business results are downstream of behaviors. Here are the facets of measuring Team Performance:

• The Manager Quality Index inputs should be weighted based on the linkage/correlation of MQI inputs to Team Performance (with Team Performance measured via the appropriate performance measure of the team’s function).
• The ROI of Manager Quality should be calculated to quantify the $$$ impact of Manager Quality on Team Performance.
• There should be a time-lag from when MQI is assessed to the subsequent Team Performance — for example 3–6 months — to prove causality.
• The success of employee interventions, such as new leader trainings, should be evaluated on the intervention’s relationship to improving MQI and therefore improving Team Performance.
• Corrections should be made for aspects of the job that managers cannot control such as: inheriting a team from someone else, re-orgs, mergers and acquisitions, etc.

The goal of the Manager Quality Index is to create a variable that is versatile and multi-trait/multi-method. The MQI can be collected via data that a people analytics team may natively has access to (e.g., engagement surveys, employee turnover, upward feedback in performance management, etc.) with minimal new data needed to be collected. The Manager Quality Index can also be used as both a predictor and an outcome variable. Variables that can be both independent and dependent variables are my favorite kind of data point. For example, MQI can be used to coach managers on an individual level to improve; the MQI can be used to diagnose managerial trends in an aggregate sense; and the MQI can even be used to evaluate the effectiveness of management improvement initiatives.

New Leaders

One finding from research my teams have completed in the past is that nearly 100% of new leaders do not fall into the top 50% of scores on the MQI. Said differently, almost all new leaders are bad at first. This is a startling finding, but to some researchers it’s not surprising at all. Robert Hogan, the founder of Hogan Personality Assessments, shared on a recent podcast that 65–70% of managers are failures according to Hogan’s research partnering with leading organizations. And those aren’t just new managers. I personally don’t know of any organization confronting this problem directly. Most organizations fail to acknowledge bad leadership is a problem at all. And, it is a serious problem. Think of it this way: if you know that the “rising tide [of leadership] lifts all boats’’, then the opposite is true as well. The Management Value Chain from earlier also plays a role. If Manager Quality leads to Team Performance, and Team Performance leads to Organizational Performance, then organizations damn-well better be focusing on improving Manager Quality for sheer self-preservation purposes.

The premise is this: If all new leaders are bad at first, then the problem really is identifying quickly which new leaders will change and improve, and which new leaders will never change — and therefore should not be in leadership. For the new leaders who are determined to improve, self improvement starts with self-awareness, and a gap analysis must be conducted before a new leader can begin focusing on their strengths and improving their weaknesses necessary to improve. I don’t care if you start with strengths or weaknesses, but you do have to start somewhere…But a bigger question persists: Why are all new leaders bad at first, and what should organizations do about this finding?

Hogan Personality & Poor Leadership

The famous expression from Marshall Goldsmith, what got you here, won’t get you there, applies directly to the problem of why all new leaders are bad at first. What got a new manager there, really won’t get them to being a good leader. I was recently certified on Hogan’s personality assessment, and there I learned about what personality characteristics can lead to poor management. Low and behold, the personality characteristics that get someone promoted seem to be related to the exact behaviors will make them a poor leader. I knew from my prior research that almost all new leaders were bad at first, but I never knew why. Two research-backed ways of getting ahead in the world include ingratiation and self-promotion. For getting promoted into management, integration, self-promotion, detail orientation, and track record of strong performance are usually the top reasons why someone is promoted.

And guess what? According to Hogan, those behaviors are the top prerequisites of why any manager (new or experienced) is a bad manager. The evidence shows up on four Hogan personality scales: Dutiful, Bold, Diligence, and Prudence. Why these scales?

• Dutiful — According to Hogan, someone high on Dutiful is more likely to be ingratiating as an individual contributor, but they may lack the courage and bravery needed to stick up for their team when moved into management because of their desire to please superiors.
• Bold — According to Hogan, someone high on Bold is more likely to self-promote their work as an individual contributor, but they may overpromise and underdeliver when moved into management because of their penchant for risk-taking behaviors. (Caveat: Bold is the only one of these four scales that can be related to strong performance as a manager, but it is not assured.)
• Diligence — According to Hogan, someone high on Diligence is more likely to be highly detail-oriented and organized as an individual contributor, but they are more likely to be prone to micromanaging, being nitpicky, and refuse to delegate when moved into management because of the need to be in control. This is the most common problem amongst new managers, and why many will never improve. Micro-managers are born here.
• Prudence — According to Hogan, someone high on Prudence is more likely to be conscientious and strong performing as an individual contributor, but they may lack the “people-focus” when moved into management because of their proclivity to be “task-focused” at work (remember the peter principle from earlier?).

Finding this out was like a bombshell to me. This proved that the primary characteristics organizations look for in Manager Selection are often what causes poor leadership. Holy moly, talk about self-inflicted wounds… What got the new leader promoted, really won’t make them a good manager. As a consequence, what should organizations be looking for when they hire leaders? Sadly, the reality is a lot of high performance and potential is outside of a person’s control:

“Some people begin their careers with a clear performance advantage. They may be smarter than you, come from a better socio-economic background, be physically attractive, or have helpful personality characteristics. Each of those factors is scientifically proven to help someone perform better than you. Those combined items predict up to 50% of anyone’s individual performance, according to academic research.” “Steps to High Performance” by Marc Effron

If you’re like me, now you are deflated after hearing that 50% of performance is outside of my control and then looking at my own flaws… But how are the other 50% of performance determined for new managers? Stay tuned for future articles…

I hope you like this article. If so, I have a few more articles in the “People Analytics Primer” series coming out soon. Stay tuned.

If you are interested in learning more directly from me or how to implement a Manager Quality Index, please connect with me on LinkedIn. (I’m more likely to accept the connection if you add a note referencing this series in your request. That way, I know you’re not trying to spam me.)

Previous articles

1st Installment: “You’re leading People Analytics: Now what?” Creating useful people analytics dashboards

2nd Installment: “You’re leading People Analytics: Now what?” Strategy and implementation of a people analytics function

3rd Installment: “You’re leading People Analytics: Now what?” Executing an effective people analytics project

4th Installment: “You’re leading People Analytics: Now what?” Building a team and team capability

5th Installment: “You’re leading People Analytics: Now what?” Overrated vs underrated people analytics topics

6th Installment: “You’re leading People Analytics: Now what?” The first principles of talent and competitive advantage

1st Installment: People Analytics Primers: Data Exploration & Visualization Tips & Tricks