Hi there,

In the past few months, I've heard a lot of myths and misconceptions about sensitive data within AI systems. And some of the misconceptions are being perpetuated by respected influencers. 

If you're going to use sensitive data in an AI system, you need to know how to do so safely. It's never been more important to build systems that are secure by design and by default. Below is a taste of some of the myths making the rounds. We debunk several more and cite academic research in our recent article in Hackernoon, "Embeddings Aren't Human Readable" And Other Nonsense. 

Myth 1 - Most of the security and privacy vulnerabilities are in the AI model

Not anymore. Traditionally all the sensitive data went into training a model, but today we have generally intelligent models like large language models and large vision models that are trained entirely on public data. When using these models, the private data shifts into the prompts, the inferences, and the embeddings. The embeddings are often referred to as the memory of AI.

I walk through the landscape of AI security problems and solutions in my short video, The Landscape of AI Security.

Myth 2 - Attackers don't know how to attack these new AI systems

Not all hackers are up to speed, but the academic literature and tutorials around how to attack AI systems and data are exploding, along with open source projects and tools to help attackers without deep AI knowledge attack these systems.

Don't assume the lack of any major breaches of AI systems means that these systems are either safe or being ignored by hackers. In fact, they're neither.

One example of an attack that isn't broadly known but is eye-opening is an embedding inversion attack. This takes the AI memory and uses it to reconstruct the original inputs that led to it. And we've been playing here internally with open-source programs that make this attack straightforward. 

I go into more detail in the blog mentioned earlier, but the main takeaway is this: the best attacks currently recover 92% of words and meaning from embeddings including personal details like people's names.

Myth 3 - Data stored in the AI memory (aka vector database) is meaningless to humans

Again, no. At a low level, an embedding is merely an array of real numbers with values that look something like this:

[0.123, -0.345, 0.567, -0.008, ....., -1.284]

When you or I look at those numbers, they're pretty meaningless. Does that mean embeddings are hashed or anonymized? Not so fast...

If you look at a GIF in a text editor instead of in your browser, it would look meaningless to you. That's because a gif, just like vector embeddings, are stored in a machine representation that requires interpretation before a human can understand it.

I mentioned embedding inversion attacks above and how much original text can be recovered, but the same is also true of embeddings representing other things like faces, voices, and more. Attacks on these models can generate photos and audio that look and sound like the original source.

Not only that, but there are other attacks on embeddings including membership inference and attribute inference attacks. And even if you ignore these attacks, embeddings are incredibly useful in and of themselves since they can be used for potent purposes like semantic search. Imagine a hacker that can just query your data asking for the most sensitive bits. Embeddings help the attacker do just that.

Protect Your AI Data 

If all of this is alarming to you, it should be. And that's why we are building Cloaked AI. Vector embeddings are incredibly powerful and the benefits are numerous, including constraining hallucinations. However, the risks are high if the data is sensitive. If you're storing embeddings in a vector database, you should be protecting them with Cloaked AI.

And as of last week, you can get your hands on the Cloaked AI beta. We're looking for feedback and input and we'd love to hear about your use cases. Fill out the form on the Cloaked AI page, and you'll be emailed the full instructions to access the beta. The team is busily adding a lot of features and functionality to Cloaked AI, and in the next beta release, you'll see an option to deploy it as a Pinecone proxy for a zero-friction experience. Reply to this email and let me know what else you'd like to see.

Thanks for reading along. Until next month,

Patrick Walsh
CEO, IronCore