In the rapidly evolving world of generative AI, managing AI system inputs and unpredictable outputs—such as hallucinations—is crucial for deploying secure, compliant enterprise AI solutions. By now, we’re all familiar with public examples of hallucinations – buying a Chevy Tahoe for $1, Google’s stock slipping after Bard hallucinates, and Air Canada misleading a customer – and their consequences can range from mild to dire.
Last year, to help teams detect anomalies and hallucinations, we released Galileo Evaluate and Galileo Observe, our evaluation and real-time monitoring solutions which are already helping teams accelerate development and reduce response times from days to minutes. But in our conversations with customers and AI teams, it became clear that many teams want more than monitoring. These teams are not content with simply reacting to prompt attacks and hallucinations; they want to stop them altogether. With this in mind, we set out to create a solution that can intercept AI inputs and responses in real-time, effectively stopping potential problems before they impact operations.
When setting out to build a real-time interception solution, we interviewed dozens of AI teams about their current practices for safeguarding AI inputs and outputs. We found teams faced three common challenges:
We’re excited to share that we’ve overcome these challenges and more!
Today, we are thrilled to unveil Galileo Protect, an advanced GenAI firewall solution that intercepts hallucinations, prompt attacks, security threats, and more in real-time. Building on the success of Galileo Evaluate and Galileo Observe, Galileo Protect leverages our suite of research-backed metrics to deliver real-time protection through a centralized, user-friendly interface. To ensure this works at enterprise-scale, we focused on aggressively improving accuracy and latency while dramatically reducing cost (more on this soon! 😉).
Customers such as Clearwater Analytics are already benefiting from Galileo, experiencing a 30% boost in efficiency and a dramatic reduction in response times—from days to minutes.
"Using Galileo Observe, we’ve significantly enhanced our visibility and reduced response times to any potential issues with LLMs from days to minutes. Galileo Protect will take this a step further by allowing us to automatically monitor and intercept AI responses in real-time, enabling us to provide guardrails around our AI products and bring them to customers faster." - Darrel Cherry, Distinguished Engineer, Clearwater Analytics.
Teams can safeguard their AI applications by following 3 easy steps:
1. Configure a ‘Stage’ of Rules: Easily configure a collection of rules and associated actions (collectively called a Stage) using Galileo's research-backed metrics or custom metrics of your own. Stages can be configured directly within the application code for ease or centrally for greater oversight.
1"ruleset":{
2 "rules":[{"metric":"pii"
3 "operator":"contains"
4 "target_value":"address"
5 }]
6 "action":{
7 "type":"OVERRIDE"
8 "choices":[
9 "Personal address detected in the model output. Sorry, I cannot answer that question."
10 ]
11 }
12}
2. Monitor your Stages and Rules: Keep an eye on active Stages and receive alerts when they are triggered. Use Observe to identify why a Rule was triggered and Evaluate to troubleshoot and resolve the issue.
3. Iterate and Optimize: Refine and adjust Stages and Rules as needed. Galileo helps by versioning Rules, which simplifies the process of adjusting them to enhance user experience.
Protect comes packed with features designed to help enterprise teams quickly configure and manage a portfolio of GenAI safeguards and interceptions:
We believe real-time interception is the next step towards building more secure, compliant, and efficient enterprise GenAI systems. We’re extremely excited to see what customers and future customers will accomplish with Galileo Protect.
If you’d like to learn more, watch Protect in action below, read our docs, or get started with Galileo today!