Why HR’s Data Problem Threatens Growing AI Investments

As HR functions continue to find new uses for artificial intelligence to save time or create efficiencies, a pernicious headwind threatens to undermine their growing investments in the promising technology: HR’s historic data quality problem.

Human resources has long struggled to produce accurate and trustworthy data from the wealth of information residing in its proliferating number of technology systems and apps. The problem results from a host of issues: lax data cleaning and auditing policies, poorly integrated and often-rigid technology platforms, a dramatic rise in data complexity, and limited resources for purchasing sophisticated data orchestration or analytics tools.

But the challenge of creating more reliable and accessible data to help inform people decisions has come into stark relief with the advent of AI tools. The quality of AI’s outputs rests solely on the quality of underlying data. Inaccurate, outdated or biased people data—connected to metrics like workforce costs, recruiting performance or employee turnover—poses a rising threat to the investments companies are making in traditional and generative AI (GenAI) tools.

“The application of AI on flawed data simply leads to more flawed data,” says Mary Faulkner, a principal with IA, an HR advisory firm in Atlanta. “Much like the application of automation to a flawed process, it only enhances the flaws that are already there. Worse, it covers the flawed analysis with a cloak of accountability that it has not earned. AI has the potential to be a black box if HR does not understand its data first.”

Getting HR’s Data House in Order

Poor data quality isn’t only an issue that affects HR functions. Recent research from Gartner found that data quality issues cost organizations an average of $12.9 million per year from flawed decision-making and increasing complexity in technology ecosystems.

Experts say avoiding the cart-before-the-horse problem of implementing AI tools before HR data quality issues are addressed requires a number of strategies. Dennis Behrman, vice president of marketing for One Model, an Austin, Texas-based people analytics software firm, says organizations first need to address basic data orchestration issues to tame the growing volume of people data generated by new cloud-based HR technology platforms and apps.

“There is more data flowing through the HR technology ecosystem than ever before,” he says. “That makes it a challenge to extract, orchestrate and structure that data in a way that facilitates strategic decision-making.”

Compounding the challenge: Much of the new data being generated in HR today lives in unstructured formats, making it harder to collect, store and analyze. Examples of unstructured data include text responses to open-ended questions on employee engagement surveys, narratives in performance reviews, or content in video or audio formats.

The problem also manifests in the age-old challenge of accurately measuring organizational headcount, Behrman says. “If you were to sit down with the CHRO and CFO in many large companies and ask them what the company’s headcount is, you’re likely to get two very different answers,” he says. “Part of the reason is they may have different definitions of headcount—whether it includes seasonal, contract or part-time workers, for example—but part of it is a data quality issue as well.” 

Rebecca Wettemann, CEO and principal analyst of Valoir, an Arlington, Va.-based research and analyst firm specializing in HR and AI technologies, says in addition to technology-based issues, human factors also have contributed to HR’s data quality problems.

“The legitimate concerns about data security and stewardship of sensitive data were often overplayed by ‘owners’ of that data in different systems,” she says. “That meant bringing that data together in a meaningful and timely way was not just technically but politically challenging.”

Wettemann says institutional knowledge of complex HR systems often has been limited to only a handful of employees in organizations.

Lydia Wu, senior director of people strategy and operations for Panasonic Energy of North America, says the amount of data now being generated—as HR has moved from on-premises to cloud technologies—requires more frequent cleaning, validation and architecting, whether that be accomplished through internal resources or third-party vendors.

“The data in the cloud has been piling up, and the bigger the pile the more difficult the solutions,” Wu says. “In HR, data cleaning can feel like a never-ending exercise, and it can be challenging to clean up and continuously improve the process at the same time.”

In some cases, the problem isn’t the lack of accurate or unified data, but rather how accessible HR data is to leaders—and how willing they are to use it to inform their decisions. A recent study from Envoy, a workplace technology platform provider in San Francisco, found that 80 percent of executives said they would have approached their company’s return-to-office (RTO) strategies differently if they’d had access to more employee data. The Envoy study found only 7 percent of executives used any employee feedback to make their RTO decisions; most simply relied on their own opinions.

The Next AI Frontier

Solving the data issue is even more imperative as HR and IT accelerate investments in GenAI tools. A January 2024 survey of HR leaders from Gartner found that 38 percent of companies are piloting, planning implementation or have already implemented GenAI, up significantly from 19 percent in June 2023.

“Generative AI is not a tool that extracts truth from HR data,” Behrman says. “Large language models essentially only ask, ‘Do these words make sense given the context of the prompt I was just given?’ Generative AI doesn’t ask if its answers are true or not.”

HR functions seeking accurate, real-time data as a foundation for AI tools should first focus on extracting that data from disparate technology platforms, structuring it, then combining it into a robust, centralized data repository, Behrman advises.

“Organizations that skip over that part … will encounter a lot of risk if they’re throwing the latest AI technology against poorly structured or unreliable data,” he says.

Wu believes fixing HR data issues and applying AI tools are initiatives that should happen in parallel.

“If you buy an AI solution without having the right data in place, you just bought the equivalent of a really expensive paperweight,” she says. “On the other hand, if you fix your data and aren’t using an AI solution to create new efficiencies in analysis or to save your HR staff time, you’ll be lagging behind the market.”

Clarity Around HR Data Strategy, Governance

Addressing HR data quality issues should start with creating sound data strategy. “That includes agreement on what data should be called, how it is collected, where it should be stored and how it is audited,” says Kimberly Carroll, managing principal with IA.

“On the data side, it’s critical to first get clear on your philosophy,” Wu says. “What things do you want to measure, how will you design your HR and recruiting systems to capture data for those metrics, how do you want to deliver analytics throughout the organization, how deeply do you as an HR function want to get into data architecting and process engineering, and how much of that do you want to leave to IT? Once those questions are answered it becomes easier to focus on cleaning and auditing your data and bringing in AI tools.”

IA’s Faulkner says it’s also important for organizations to establish clear HR business process and systems governance policies. Without proper governance—which includes clearly assigned roles and responsibilities across functions for data quality, integration, security, regulatory compliance and more—there is little accountability and ownership of data, which becomes a particular issue during auditing.

Clarifying the HR data strategy should begin now. “If you look back at the last decade, every HR technology innovation required high-quality data to succeed,” Wu says. “Without properly cleaned, structured and integrated data, the duct tape and chicken wire can only last so long for HR professionals.”   

Dave Zielinski is a freelance business journalist in Minneapolis.