The efficiency argument is compelling.
Generative AI promises to screen hundreds of applications in seconds, draft candidate summaries overnight, score assessments without fatigue, and do it all at a fraction of the cost of a human recruiter. For organisations under pressure to move quickly in a competitive talent market, it sounds like exactly the kind of strategic advantage leaders should be asking HR to adopt.
But there is a category of risk that is rarely discussed in vendor demonstrations or efficiency reports. It sits squarely in the domain of governance, data integrity, and legal liability.
This post examines four material uncertainties that any responsible executive or board member should understand before signing off on Generative AI in the recruitment pipeline.
When the Machine Invents the Truth:
AI Hallucination in Candidate Assessment
Hallucination is the term used to describe what happens when a Generative AI produces output that is factually incorrect, fabricated, or entirely invented but presented with the same confident tone as accurate information. In a consumer context, this might be mildly inconvenient. In a recruitment context, the consequences are materially different.
Consider what hallucination looks like in practice: an AI screening tool summarising a candidate's CV might disregard qualifications that they hold, require experience in a technology listed nowhere in the application, or generate a 'cultural fit' score based on pattern-matching that has no grounded basis in the candidate's actual submissions. The system will present none of this as conjecture.
The legal exposure here is significant and underappreciated.
If an AI tool generates a false or misleading summary that causes a qualified candidate to be rejected, your organisation may bear liability regardless of whether the decision was made by a human who trusted the output. Organisations cannot outsource accountability to the algorithm.
The more difficult governance question is not whether hallucination occurs (it does, in every major Generative AI) but whether your organisation has a process to detect it. Most do not. When AI-generated candidate assessments are reviewed by time-pressured HR teams, plausible-sounding errors will pass through. The accountability gap this creates is not theoretical; it is structural.
The Siloing Problem:
When Candidate Data Flows Where You Didn't Intend
Recruitment data is amongst the most sensitive personal information an organisation manages.
A job applicant submits their professional history, skills, sometimes health disclosures, and often demographic information, all with an expectation that this information will be used solely to evaluate their suitability for a specific role.
When Generative AI is introduced into the HR workflow, that data often begins to move in ways that neither the candidate nor the hiring organisation fully understands. The issue is not malicious intent, it is architecture. Many AI-integrated recruitment platforms are built on shared infrastructure, where data from one 'tenant' (your candidates) exists within systems that are not rigorously siloed from other processes or other users of the same platform.
Specifically, the risks of data non-siloing in AI HR tools include:
- Candidate data being accessible across departments within the platform beyond those with a legitimate need to know
- Cross-contamination between your recruitment data and that of other organisations using the same underlying AI model or interface
- Data aggregation, where an AI tool combines candidate information from multiple touchpoints: CV, cover letter, social media scraping, video interview transcript. In ways that were never disclosed to the candidate
- Data persistence beyond the retention period your privacy policy specifies, retained within the vendor's infrastructure
Under Aotearoa New Zealand's Privacy Act 2020, organisations are required to collect personal information only for a lawful purpose, use it only for that purpose, and ensure it is held securely. A new Information Privacy Principle (IPP 3A), effective May 2026, requires organisations that collect personal information indirectly (including through AI tools that scrape or aggregate data) to notify the individual. This is a meaningful tightening of obligations that many AI-assisted recruitment deployments may already be breaching.
Data Leakage:
The Exposure You May Not Know Has Happened
Data leakage in the context of Generative AI is distinct from a conventional data breach. It does not require a malicious actor, a system compromise, or a catastrophic failure. It can occur silently, as a byproduct of normal use and by the time it is identified, the data has left the organisation's control permanently.
The most common vector is straightforward: an HR professional, under time pressure, pastes candidate information (CVs, reference notes, salary discussions, assessment summaries) into a consumer-grade or insufficiently governed AI tool to generate a summary, draft a rejection letter, or compare candidates quickly. The information entered into that prompt may be stored, logged, reviewed by the vendor, or in some configurations, become accessible to other users of the same system.
This is not a hypothetical risk. Well-documented incidents have demonstrated that employees at major organisations have inadvertently transmitted sensitive internal data through AI tools in exactly this way. In the recruitment context, the data at stake includes names, contact information, employment history, salary expectations, and in some cases health or disability information disclosed in good faith by candidates.
For organisations operating in multiple jurisdictions (and most New Zealand organisations do, especially through cloud infrastructure hosted offshore) cross-border data transfer obligations under the Privacy Act 2020 apply. Sending candidate data to an AI vendor whose servers are hosted in the United States or the European Union creates a transfer obligation that requires prior assessment of whether the receiving environment provides equivalent privacy protections.
The governance failure here is not always at the point of technology selection. It frequently occurs in the gap between policy and practice: an organisation has an approved AI tool, but individual staff members are supplementing it with personal accounts, consumer-grade tools, or browser extensions that have entirely different data handling characteristics.
Training Data Re-use:
Is Your Candidates' Data Building Someone Else's Model?
This is perhaps the least understood risk in the stack and in some respects, the most consequential from a long-term governance perspective.
When organisations use third-party Generative AI tools, many of those tools (particularly consumer-grade or standard-tier enterprise offerings) include provisions in their terms of service permitting the use of user inputs to improve, fine-tune, or retrain their underlying models. In practical terms, this means that candidate CVs, assessment notes, interview summaries, and internal hiring commentary entered into these tools may become part of the training data that shapes future model behaviour for the vendor's other clients, or for publicly available versions of the model.
This creates several material uncertainties that remain unresolved in current legal and regulatory frameworks:
- Candidates provided their personal information for a specific and limited purpose. It was not collected with consent for AI model training. Using it as such is likely inconsistent with the purpose limitation principle under the Privacy Act 2020 and analogous obligations under GDPR for organisations with UK or EU applicants.
- Once candidate data has been absorbed into a model's training set, there is no reliable technical mechanism to remove it. The 'right to erasure' (which candidates may invoke) cannot currently be guaranteed in AI-trained systems. As one leading data governance analysis notes, it is 'technically impossible to fully remove data from training datasets without influencing deep learning models in unpredictable ways.'
- The vendor's obligations around training data are often opaque. Enterprise agreements may include 'zero-shot' or 'no training' provisions but verifying compliance with those provisions is rarely straightforward. Standard consumer or SME tiers frequently do not include these protections.
- If your AI tool vendor is acquired, restructured, or undergoes a change in terms of service, the protections you believed were in place may no longer apply and candidate data already absorbed into prior training runs cannot be recalled.
The Challenge:
That Which We Do Not Yet Know
Responsible governance requires acknowledging the boundaries of current certainty. Several material questions remain genuinely unresolved:
- At what threshold does AI-assisted screening become an 'automated decision' under privacy and employment law, triggering the right to human review? The line between decision support and decision-making is blurred in most current deployments, and legal precedent is still forming.
- How will courts in Aotearoa and the UK / EU / US interpret employer liability for discrimination arising from AI hallucination, particularly where the employer cannot demonstrate what data the model used or why it produced a particular output? The Workday litigation in the United States, alleging discriminatory screening of candidates from protected groups, is an early signal of the legal risk landscape.
- What constitutes meaningful consent to AI processing in a recruitment context? Candidates who apply for roles are not in a position to negotiate the terms of data processing. Whether standard privacy notices adequately capture AI-specific processing remains contested.
- How will AI vendor terms of service evolve as regulatory pressure increases? The current patchwork of enterprise agreements, consumer terms, and jurisdiction-specific data processing addenda creates genuine uncertainty for procurement and legal teams.
What This Means for Your Organisation
None of this is an argument against Generative AI in HR. The efficiency gains are real, and the technology will continue to develop. But the current deployment environment is characterised by speed of adoption outpacing speed of governance and the organisations most exposed are those that have treated AI tool selection as an operational decision rather than a risk decision.
The questions that boards and executives should now be asking of their HR and technology leadership include:
- Has a Privacy Impact Assessment been completed for each AI tool used in the recruitment workflow, including informal or supplementary tools used by individual staff members?
- Do we have a clear contractual position on whether our candidates' data is used to train our vendor's models and can we verify it?
- What human oversight exists between AI-generated assessments and hiring decisions? Is it documented?
- Do we have a complete picture of where candidate data goes once it enters our AI-integrated systems, including cross-border transfers?
- Have we assessed our exposure under IPP 3A (effective May 2026) which now requires disclosure when personal information is collected indirectly, as it routinely is through AI-integrated applicant tracking system platforms?
These are not IT questions. They are governance questions. And in an unpredictable regulatory and legal landscape, they belong on the board agenda.
If you are unsure how your organisation currently sits across any of these four risk areas, CyberForensics works with boards and leadership teams to map that exposure clearly and practically. A conversation costs nothing. Uncertainty, left unexamined, often costs considerably more.
Contact Us