Senserva Blog

Teaching AI to Know When It Doesn't Know

Written by Clay Babcock | Dec 8, 2025

We've been thinking a lot about trust lately. Not the "do you trust this vendor" kind of trust, but the deeper question: when should you trust AI to make decisions on its own, and when should it ask for help?

That question has led us to what our founder Mark Shavlik has been calling "Guard Rails" - and it represents where Senserva is headed next.

The Problem We Keep Seeing

Here's what we've learned after years of helping organizations manage configuration drift: detection is the easy part. Our platform already catches misconfigurations that would take security teams months to find manually. We're good at that.

The hard part? Knowing what to do about them.

Some configuration changes are clear-cut threats that need immediate remediation - like someone disabling MFA on your critical systems. Others are ambiguous - they could be legitimate business changes, or they could be the early stages of a sophisticated attack. And a few require human judgment because the stakes are so high that even 99% confidence isn't enough.

Traditional security automation treats these the same: alert on everything, let humans sort it out. That's why security teams are drowning in alerts, and why real threats get missed in the noise.

We think there's a better way.

Meta-Cognitive AI: Systems That Know Their Limits

The innovation we're pursuing is what we call "meta-cognitive AI" - systems that don't just make decisions, but evaluate their own confidence in those decisions.

Think about how a skilled engineer approaches a problem. They don't just execute based on pattern matching. They consider context. They assess their confidence. They know when something is straightforward enough to handle quickly, and when it's complex enough to warrant escalation or collaboration.

That's what we're building into our next-generation platform.

Guard Rails works like this:

  1. Detection confidence: Our existing platform already does this well - pattern matching, anomaly detection, historical learning give us a confidence score on whether something is a threat.
  2. Context assessment: Here's the new part. The system evaluates the business context, system criticality, regulatory implications, and temporal patterns. A configuration change at 3 AM is different from the same change during a scheduled maintenance window.
  3. Impact classification: Not all threats are equal. A compromise of an emergency services system is fundamentally different from a non-critical dev environment, even if the technical indicators are identical.
  4. Graduated autonomy: Based on confidence + context + impact, the system makes a graduated decision:
    • High confidence + time-critical + clear context: Auto-remediate in 60-90 seconds
    • Medium-high confidence + critical infrastructure + ambiguous context: Escalate to human judgment in 5-15 minutes
    • Lower confidence or unclear context: Alert and monitor, gather more data

The key innovation isn't just faster detection. It's AI that knows when it doesn't know, and adjusts its behavior accordingly.

Progressive Learning: Building Trust Over Time

The second piece of the puzzle is progressive autonomy. Systems don't start at 95% autonomous, they earn it.

Month 1-2: Cautious (30% autonomous) The system operates mostly in advisory mode. It makes recommendations, humans approve them, and the AI learns from those decisions. What was approved? What was rejected? In what contexts?

Month 3-6: Confident (70% autonomous) For scenarios where the system has built confidence and the outcomes have been consistently correct, it starts acting autonomously. But it still escalates ambiguous cases and tracks its accuracy.

Month 6+: Mature (95% autonomous) The system handles most scenarios independently, but - and this is critical - it has learned to recognize the 5% where human judgment adds value. It doesn't just become more aggressive, it becomes wiser about its limitations.

This isn't just about efficiency. It's about building a sustainable security posture where automation handles what it's genuinely good at, and human expertise gets applied where it actually matters.

Why Telecommunications? Why Now?

We've been focused on Microsoft environments - and that's still our core business. But telecommunications represents both a natural expansion and a validation of where this technology needs to go.

Carriers operate life-safety systems (E911), national security infrastructure (CALEA lawful intercept), and critical 5G networks serving hundreds of millions of subscribers. And billions of calls, messages, and data packets. The stakes couldn't be higher, and the operational constraints are extreme.

Traditional security automation doesn't work in these environments because the false positive cost is too high. Accidentally block a legitimate E911 configuration change? You've just created a life-safety incident. But wait for manual review of every change? That's how attacks like Salt Typhoon stay undetected for 18 months.

Telecommunications needs exactly what we're building: AI that can act with confidence when the situation is clear, but knows when to pause and escalate when the stakes are high or the context is ambiguous.

The timing is interesting too. The Salt Typhoon breach - where nation-state actors compromised nine U.S. carriers including T-Mobile through configuration changes - has created industry-wide urgency around configuration security. The attack succeeded precisely because the changes looked authorized to traditional security tools.

The T-Challenge Connection

When we learned about Deutsche Telekom's T-Challenge 2025 - their annual innovation competition focused on AI-driven security - it felt like the right moment to formalize and share this direction.

We're one of hundreds of submissions, and honestly, that's fine. Whether we win or not, the process of articulating this vision for a sophisticated technical audience (and getting their feedback) is valuable. And if Deutsche Telekom sees value in partnering with us to develop this for telecommunications, that's a force multiplier.

But the direction we're headed isn't dependent on winning a competition. Guard Rails represents where security automation needs to go: systems that are smart enough to act autonomously when appropriate, and wise enough to know when they're not.

What This Means for Senserva Customers

If you're a current Senserva customer, you're probably wondering: "Does this mean you're abandoning Microsoft environments for telecommunications?"

Not even close.

The meta-cognitive framework we're building works across any complex, high-stakes environment. Microsoft 365, Azure, Entra ID, Intune - these all have the same fundamental challenge: lots of configuration changes, some legitimate and some malicious, and the need to respond quickly without creating false positive chaos.

Everything we're learning about progressive autonomy and context-aware decision-making will flow back into our core platform. Telecommunications is an expansion opportunity and a forcing function to make the technology even better, but the innovations benefit all our customers.

Think of it this way: if we can build AI that's trusted to manage life-safety telecommunications infrastructure, that same AI is going to be incredibly good at managing your Microsoft security stack.

Looking Forward

We're still early in this journey. The Guard Rails framework is about 70% developed as a decision engine, and the telecommunications-specific integrations are in the co-development phase (which is why a partner like Deutsche Telekom would be so valuable).

But the direction is clear. Security automation has been stuck in a binary mindset: either alert on everything and overwhelm humans, or auto-remediate everything and create unacceptable false positive risk.

The future is graduated autonomy. AI that understands context, evaluates its own confidence, and makes appropriate decisions ranging from immediate action to thoughtful escalation.

That's what we're building. And whether it's through telecommunications, Microsoft environments, or eventually other critical infrastructure sectors, the goal is the same: security systems that are both more effective and more trustworthy than what exists today.

We'll keep you posted on where this goes.

Clay Babcock is President of Senserva, where he focuses on partner strategy and market development. Before Senserva, he spent 15+ years in telecommunications operations and infrastructure, earning Nokia innovation awards for work on carrier systems. He's passionate about turning complex security problems into practical, deployable solutions.

 

Note: Senserva's Guard Rails concept was submitted to T-Challenge 2025, Deutsche Telekom's annual innovation competition. The competition runs through April 2026, with the top 12 teams selected in January.