New York Senator Raises Concerns Over Identity Verification Vendor Socure’s AI Data Practices

On July 10, New York State Senator Jeremy Cooney wrote to Dru Rai, the state’s chief information officer, expressing serious concerns about the state’s use of Socure, an AI-powered identity verification firm.

The letter examines Socure’s data practices, specifically the collection and unauthorized use of Personally Identifiable Information (PII). It demands transparency and ethics in handling this data and highlights the potential risk of artificial intelligence in the public sector. It also inquired whether the state had tested Socure’s fraud prediction models for bias and if they comply with state privacy laws.

Socure’s technology is used to predict fraud for resident services, such as by scanning benefits applications. Fraud has become a growing concern for state agencies, particularly since the COVID-19 pandemic, which saw heightened levels of fraudulent activities across many government functions. Socure is actively “pressure testing” its AI models to detect biases across demographics and ensure fairness.

The New York Senator’s main issue is how Socure collects and uses data points. He described the company as a “data broker” that collects, buys, and stores billions of data points, including PII without the knowledge or consent of New Yorkers. He argues that while identity verification is key to equal access to services, uncontrolled use of AI could prevent people from accessing essential government services.

However, Socure disputes these claims. Jordan Burris, vice president of public sector policy at Socure emphasized that the company is not a data broker and does not sell data to third parties or use it for commercial purposes. He further responded that the sole purpose of Socure is identity verification and that it uses data from various public and private sources for this purpose only.  He added that human reviewers are involved throughout the verification process to mitigate errors and biases.

“We are only focused on verifying identity and rooting out fraud,” Burris said, adding that Cooney’s letter misunderstands the company’s operations.

This is not the first time Socure has faced criticism. Earlier this year, New York leaders, including Rep. Ritchie Torres and the Rev. Al Sharpton, expressed similar concerns about the company’s practices regarding transparency and data. Torres’ letter to Socure’s CEO highlighted fears that the software might lead to discrimination and misuse of private data. Sharpton also questioned Socure’s transparency regarding data acquisition, storage, and use.