The New Analytics of Workplace Culture

​Editor's Note: SHRM has partnered with Harvard Business Review to bring you relevant articles on key HR topics and strategies.

A business's culture can catalyze or undermine success. Yet the tools available for measuring it—namely, employee surveys and questionnaires—have significant shortcomings. Employee self-reports are often unreliable. The values and beliefs that people say are important to them, for example, are often not reflected in how they actually behave. Moreover, surveys provide static, or at best episodic, snapshots of organizations that are constantly evolving. And they're limited by researchers' tendency to assume that distinctive and idiosyncratic cultures can be neatly categorized into a few common types.

Our research focuses on a new method for assessing and measuring organizational culture. We used big-data processing to mine the ubiquitous "digital traces" of culture in electronic communications, such as emails, Slack messages and Glassdoor reviews. By studying the language employees use in these communications, we can measure how culture actually influences their thoughts and behavior at work.

In one study, two of us partnered with a midsize technology company to assess the degree of cultural fit between employees and their colleagues on the basis of similarity of linguistic style expressed in internal email messages. In a separate study, two of us analyzed the content of Slack messages exchanged among members of nearly 120 software development teams. We examined the diversity of thoughts, ideas and meaning expressed by team members and then measured whether it was beneficial or detrimental to team performance. We also partnered with employer-review website Glassdoor to analyze how employees talk about their organizations' culture in anonymous reviews to examine the effects of cultural diversity on organizational efficiency and innovation.

The explosion of digital trace data such as emails and Slack communications—together with the availability of computational methods that are faster, cheaper and easier to use—has ushered in a new scientific approach to measuring culture. Our computational-lingustics approach is challenging prevailing assumptions in the field of people analytics and revealing novel insights about how managers can harness culture as a strategic resource. We believe that with appropriate measures to safeguard employee privacy and minimize algorithmic bias it holds great promise as a tool for managers grappling with culture issues in their firms.

The Studies

Our recent studies have focused on cultural fit versus adaptability, the pros and cons of fitting in, cognitive diversity and the effects of diversity on organizational performance. Let's look at each in detail.

Fit versus adaptability. When managers think about hiring for cultural fit, they focus almost exclusively on whether candidates reflect the values, norms and behaviors of the team or organization as it currently exists. They often fail to consider cultural adaptability—the ability to rapidly learn and conform to organizational cultural norms as they change over time. In a recent study two of us conducted with Stanford's V. Govind Manian and Christopher Potts, we analyzed how cultural fit and cultural adaptability affected individual performance at a high-tech company by comparing linguistic styles expressed in more than 10 million internal email messages exchanged over five years among 601 employees. For example, we looked at the extent to which an employee used swear words when communicating with colleagues who themselves cursed frequently or used personal pronouns ("we" or "I") that matched those used by her peer group. We also tracked how employees adapted to their peers' cultural conventions over time.

We found, as expected, that a high level of cultural fit led to more promotions, more-favorable performance evaluations, higher bonuses and fewer involuntary departures. Cultural adaptability, however, turned out to be even more important for success. Employees who could quickly adapt to cultural norms as they changed over time were more successful than employees who exhibited high cultural fit when first hired. These cultural "adapters" were better able to maintain fit when cultural norms changed or evolved, which is common in organizations operating in fast-moving, dynamic environments.

These results suggest that the process of cultural alignment does not end at the point of hire. Indeed, our study also found that employees followed distinct enculturation trajectories—at certain times in their tenure demonstrating more cultural fit with colleagues and at other times less. Most eventually adapted to the behavioral norms of their peers, and those who stayed at their company exhibited increasing cultural fit over time. Employees who were eventually terminated were those who had been unable to adapt to the culture. Employees who left voluntarily were the most fascinating: They quickly adapted culturally early in their tenures but drifted out of step later on and were likely to leave the firm once they became cultural outsiders.

To further assess how cultural fit and adaptability affect performance, Berkeley's Jennifer Chatman and Richard Lu and two of us surveyed employees at the same high-tech company to measure value congruence (the extent to which employees' core values and beliefs about a desirable workplace fit with their peers) and perceptual congruence (how well employees can read the "cultural code" by accurately reporting the values held by peers). We found that value congruence is predictive of retention—employees with it are less likely to voluntarily leave the company—but is unrelated to job performance. We found that the opposite is true of perceptual congruence: It is predictive of higher job performance but unrelated to retention. These results suggest that companies striving to foster a stable and committed workforce should focus on hiring candidates who share similar values with current employees. Employers needing people who can quickly assimilate and be productive should pay greater attention to candidates who demonstrate the ability to adapt to new cultural contexts.

The benefits of not fitting in. When might it better to hire a cultural misfit? People who see the world differently and have diverse ideas and perspectives often bring creativity and innovation to an organization. But because of their outsider status, they may struggle to have their ideas recognized by colleagues as legitimate. In a recent study two of us conducted with V. Govind Manian, Christopher Potts, and William Monroe, we compared employees' levels of cultural fit with the extent to which they served as a bridge between otherwise disconnected groups in the firm's internal communication network. For instance, an employee might have connections with colleagues that bridge both the engineering and sales departments, allowing her to access and pass on a greater variety of information and ideas.

Consistent with prior work, we found that cultural fit was, on average, positively associated with career success. The benefits of fitting in culturally were especially great for individuals who served as network bridges. When traversing the boundary between engineering and sales, for example, they could hold their own in technical banter with the former and in customer-oriented discourse with the latter. People who attempted to span boundaries but could not display cultural ambidexterity were especially penalized: They were seen as both cultural outsiders and social outsiders without clear membership in any particular social clique. However, we also identified a set of individuals who benefited from being cultural misfits: those who did not have networks spanning disparate groups but instead had strong connections within a defined social clique. By building trusting social bonds with colleagues, they were able to overcome their outsider status and leverage their distinctiveness. These results suggest that an effective hiring strategy should strive for a portfolio of both conformists—or at least those who can rapidly adapt to a company's changing culture—and cultural misfits.

Cognitive diversity. Proponents of cultural diversity in teams presume that it leads to cognitive diversity; that is, diversity in thoughts and ideas. But the findings about whether cognitive diversity helps or hinders team performance are inconclusive. Part of the problem is that these studies use imperfect proxies for cognitive diversity, such as diversity in demographics, personalities or self-reported beliefs and values. Moreover, this line of research has rarely looked at how diversity is actually expressed in communications and interactions, which is problematic given that team members are sometimes reluctant to share their real feelings and opinions. Finally, cognitive diversity is often assumed to be static, even though we know team dynamics frequently change over a project's life cycle.

In a new study, which two of us conducted with Stanford researchers Katharina Lix and Melissa Valentine, we overcame these challenges by analyzing the content of Slack messages exchanged among team members of 117 remote software-development teams. We identified instances when team members discussing similar topics used diverse meanings, perspectives and styles, and then analyzed the impact of that diversity on performance. For example, in discussions of customer requirements, different interpretations of the desired look and feel of the user interface in some cases led developers to talk past one another and fail to coordinate, but in other cases sparked creative new ideas.

Our results indicate that the performance consequences of cognitive diversity vary as a function of project milestone stages. In the early stages, when the team is defining the problem at hand, diversity lowers the chances of successfully meeting milestones. During middle stages, when the team is most likely to be engaged in ideation, diversity increases the likelihood of team success. Diversity becomes an obstacle again toward the end of a project, when the team is deep into execution.

Cultural diversity and the organization as a whole. We've seen that there are trade-offs associated with diversity in teams, but how does it affect the performance of entire organizations? Conventional wisdom holds that firms must choose between a homogeneous, efficient culture and a diverse, innovative culture. A homogeneous culture improves efficiency and coordination, the theory goes, because employees agree about the norms and beliefs guiding work, but the benefits come at the expense of fewer novel ideas about how to accomplish tasks. In contrast, a heterogeneous culture sacrifices the benefits of consensus in favor of healthy disagreement among employees that can promote adaptability and innovation. The evidence supporting this thinking, however, is scant and inconclusive.

In a recent study, we analyzed the language that employees used when describing their organization's culture (for example, "our culture is collaborative," "our culture is entrepreneurial," and so on) in anonymous reviews of nearly 500 publicly traded companies on Glassdoor. We first measured the level of interpersonal cultural diversity, or disagreement among employees about the norms and beliefs characterizing the organization. We found that interpersonal cultural diversity makes it difficult for employees to coordinate with one another and reduces the organization's efficiency as measured by return on assets.

We then measured the organizations' level of intrapersonal cultural diversity. Those with high intrapersonal cultural diversity had employees with a large number of cultural ideas and beliefs about how to accomplish tasks within the company (measured as the average number of cultural topics that employees discussed in their Glassdoor reviews). For instance, employees at Netflix conceptualized the work culture in terms of autonomy, responsibility, collaboration and intense internal competition. We found that organizations with greater intrapersonal cultural diversity had higher market valuations and produced more and higher-quality intellectual property via patenting, evidence that their employees' diverse ideas about how to do work led them to be more creative and innovative.

This suggests that organizations may be able to resolve the assumed trade-off between efficiency and innovation by encouraging diverse cultural ideas while fostering agreement among employees about the importance of a common set of organizational norms and beliefs. Again, consider Netflix: Although "multicultural" employees contributed to the company's diverse culture and drove innovation, the culture was nonetheless anchored by core shared beliefs, such as the importance of radical transparency and accountability, which help employees coordinate and work efficiently.

Implications for Practice

How can these findings inform leaders' understanding of culture as a tool for improving the performance of employees, teams and the broader organization?

First, managers can increase retention by hiring candidates whose core values and beliefs about a desirable workplace align well with those of current employees. However, too much emphasis on cultural fit can stifle diversity and cause managers to overlook promising candidates with unique perspectives. Hiring managers should look for candidates who demonstrate cultural adaptability, as these employees may be better able to adjust to the inevitable cultural changes that occur as organizations navigate increasingly dynamic markets and an evolving workforce.

Hiring managers should also not overlook cultural misfits. They can be wellsprings of creativity and innovation. But to make sure they flourish inside the organization, managers should consider assigning them to roles in which they are likely to develop strong connections within particular social groups. That's because misfits need the trust and support of colleagues to be seen as quirky innovators rather than outlandish outsiders.

Second, leaders should be mindful that the expression of diverse perspectives in teams needs to be managed. Cognitive diversity is essential for generating novel, innovative solutions to complex problems, especially during the planning and ideation phases of a project. However, the expression of diverse perspectives can quickly become a liability when the team needs to focus on execution and meet looming deadlines. It is during these times that team members have to unify around a common interpretation of the problem and come to agreement about what needs to get done to solve it. Leaders must be adept at switching back and forth, learning when and how to promote the expression of divergent opinions and meanings and when to create a context for convergence.

An important distinction is warranted here. The term "diversity" is often used to connote variation in the demographic makeup of a firm's workforce. This has been particularly the case in recent years, as companies have tackled pernicious problems such as the underrepresentation of women and minorities in decision-making positions in organizations. In our work, we use "cultural diversity" to refer to variation in people's beliefs and normative expectations, irrespective of their demographic composition. As we pointed out earlier, demographic and cultural diversity are related, but a demographically homogenous group may be culturally diverse, and vice versa. Our research on cultural diversity is relevant to but ultimately independent of efforts to increase gender, race and ethnic diversity in firms.

Third, leaders should foster a culture that is diverse yet consensual in order to promote both innovation and efficiency. Such a culture is composed of multicultural employees who each subscribe to a variety of norms and beliefs about how to do work. These diverse ideas help employees excel at complex tasks, such as dreaming up the next groundbreaking innovation. Managers should encourage employees to experiment with different ways of working—extensive collaboration for some tasks, for example, and intense competition for others. At the same time, a culture should also be consensual in that employees agree on a common set of cultural norms—shared understandings—that helps them successfully coordinate with one another. Leaders can signal the importance of these norms during onboarding and in everyday interactions, just as leaders at Netflix do by rewarding employees for sharing their mistakes with colleagues in order to promote beliefs about the value of transparency.

A New Management Tool

Many of the tools we used in these studies are off-the-shelf products, and there is great potential for managers to use them to help solve practical challenges inside organizations. For instance, Stanford Ph.D. candidate Anjali Bhatt is working with two of us to demonstrate how language-based culture measures can be used to anticipate the pain points of post-merger integration. We are studying the merger of three retail banks, and analysis of emails has revealed stark differences in the rates of cultural assimilation among individuals. Such tools can be used diagnostically to assess the cultural alignment between firms during premerger due diligence, as well as prescriptively during integration to identify where and how to focus managerial interventions.

Yet the accessibility of these tools also raises important ethical concerns. In our work, we maintain strict employee confidentiality, meaning that neither we nor the organization is able to link any employee to any specific communication used in our studies. We also strongly advise against using these tools to select, reward or punish individual employees and teams, for at least four reasons: Accurately predicting individual and team performance is considerably more challenging than estimating average effects for broad types of individuals and teams; culture is only one of many factors influencing individual and team performance in organizations; algorithmic predictions often create a false sense of certainty in managers; and finally, giving any algorithm undue weight can have unintended consequences—for instance, exacerbating human biases that negatively affect women and members of underrepresented social groups.

Algorithms make estimates, but it is ultimately humans' responsibility to make informed judgments using them. Managers must be vigilant about keeping metadata anonymous and must regularly audit algorithmic decision-making for bias to ensure that the use of language-based tools does not have unintended adverse consequences on culture itself—for instance, by breeding employee distrust.

These important ethical questions notwithstanding, we believe that these tools will continue to generate insights that allow managers to finally manage the culture as a strategic resource, and ultimately lead to more culturally diverse and inclusive teams and organizations.

Matthew Corritore is an assistant professor of strategy and organization at McGill's Desautels Faculty of Management. Amir Goldberg is an associate professor of organizational behavior at Stanford's Graduate School of Business. Sameer B. Srivastava is an associate professor and the Harold Furst Chair in Management Philosophy and Values at the University of California, Berkeley's Haas School of Business. He and Goldberg codirect the Berkeley-Stanford Computational Culture Lab.

This article is reprinted from Harvard Business Review with permission. ©2019. All rights reserved.