The security of AI remains one of the top concerns of governments, companies, and consumers — and yet I am constantly surprised at the lax security standards and poor understanding of AI security threats within the AI community.
We're continuing our series of blog posts, videos, and webinars focused on AI and security because I believe we can influence how companies approach data privacy and data security within AI systems for the better. Companies that build security into the core of their AI systems are going to thrive in 2024, and we're committed to supporting our customers who choose this path.
We have three blogs already in 2024 starting with our AI predictions and then demonstrating different inversion attacks, including one on facial recognition. I hope you enjoy them!
Patrick Walsh CEO, IronCore
The Hidden Dangers of Face Embeddings: Unmasking the Privacy Risks
By Murph Murphy
One of the classic applications of machine learning is facial recognition, where a model is trained to find faces in images or videos and capture identifying information about them. Facial recognition has many applications, but some of the most useful ones involve taking facial features and matching them against other useful data previously collected and tied to the same face, which requires creating an embedding. Unless your facial recognition system is simply marking where to find faces in pictures or video, it’s probably generating an embedding from each face it identifies.
It may be a bit of a fool’s game, but it’s also rather important for decision making and planning. Any executive worth their salt is looking ahead at least a year to understand what changes are coming and how that might impact opportunities, threats, and where to spend time and money in the coming months.
Text embeddings contain sensitive data that's easily extractable but also easily protected
Text embeddings are used to power retrieval augmented generation (RAG) workflows, which feed relevant data to an AI model as context so it can produce more accurate, citable, grounded answers and can optionally operate on private data.
In this post I will show one very easy way people can reverse text embeddings, which often contain sensitive information, back into sentences similar to the original input. This demonstrates that even though they might not seem sensitive, vectors encode a ton of private and sensitive information, and they should be handled with care.
