At Armilla AI, our mission is to ensure that AI-powered solutions meet the highest standards of performance, fairness, transparency, and robustness. We are pleased to announce that Private AI's PII de-identification models are now backed by Armilla Guaranteed, our third party warranty for AI performance.
Private AI specializes in data redaction software designed to detect, de-identify, and replace personal information (PII) in various types of documents, including chat transcripts, medical notes, and more. The core of their software is a machine learning model trained to identify PII entities across diverse and often unstructured datasets.
Given the critical nature of PII redaction in maintaining privacy and compliance with global regulations, including GDPR, CPRA, and HIPAA, Private AI engaged Armilla AI to perform an independent assessment of their PII identification model. This assessment aimed to validate the model’s effectiveness, fairness, and robustness in real-world conditions.
Clients using Private AI’s PII de-identification solution now benefit from a comprehensive performance warranty backed by global insurers Swiss Re, Chaucer and Greenlight Re. The warranty certifies that Private AI’s solution meets critical KPIs for accuracy, fairness, and robustness, and guarantees financial compensation to Private AI’s customers in the unlikely event of underperformance.
After a thorough evaluation, Armilla AI is pleased to report that Private AI’s PII identification model passed all areas of assessment, exceeding target metrics across all relevant scenarios. Conducted in line with the NIST AI Risk Management Framework, Armilla verifies the performance, fairness and robustness of the model.
Key Highlights:
Our assessment process involved testing the model on datasets provided by Private AI, which included a variety of PII-rich documents. We also conducted additional testing using external and Armilla’s own datasets from the different domains and structured data formats to verify the model’s robustness in different use cases, industry domains and envrionments.
The performance was evaluated using key metrics such as precision, recall, and the primary metric of missed PII percentage. We also examined the model’s sensitivity to various perturbations like text casing changes and introduced OCR errors to simulate real-world data conditions.
Overall, Armilla AI’s independent validation confirms that Private AI’s PII identification model is highly effective and reliable for PII redaction. With the added security of Armilla’s AI model warranty, clients can trust that their privacy protection tools are not only top-tier but also backed by a robust performance guarantee.
We are proud to have partnered with Private AI in this critical validation effort and look forward to supporting their continued commitment to privacy and security in AI-driven solutions.
Patricia Thaine, Private AI’s CEO, shared the following thoughts about the impact of the warranty on Private AI’s business: "Armilla AI’s third party verification and warranty on the performance and accuracy of our AI-powered de-identification solution has been a powerful asset in sales cycles, helping accelerate discussions with large enterprises. Customers are thrilled that we’re able to guarantee the value of our product, backed by trusted third parties like Swiss Re."
