“AI-powered” is doing a lot of heavy lifting right now.
Walk the floor at any network industry event and count how many booths have those two words somewhere on their signage. I’ll wait. The answer is all of them, or close enough that it doesn’t matter. The term has become what “cloud-based” was in 2012 and “next-generation” was before that: a marketing modifier that tells you almost nothing about what the product actually does.
Most of these AI stories look the same when you peel them back. You type a question into a chat interface. The tool queries its own data. You get an answer in plain English. The demo looks smooth. The presenter is polished. You walk away thinking, “Okay, that was pretty cool,” and then you go back to your hotel room and wonder what you actually just saw.
At Network Field Day, Thomas Scheibe from Aviz Networks said something on stage that I haven’t been able to stop thinking about. He was describing the current state of AI in network management, and he put it this way: “I have a controller today, I put an LLM in front of it, and now I can talk to my tools. That is 10% of the problem.”
The rest of this post is about what the other 90% looks like.
The 10% Is Real Value. Just Not the Whole Story.
Before I go any further, I want to be fair to the vendors doing the 10%. A chat interface that can answer “what VLANs are on this switch” or “show me BGP neighbor states across my fabric” in plain English is genuinely useful. It lowers the barrier for L1 and L2 staff. It makes it possible for a NOC engineer who has never touched Arista CLI to pull information off an Arista switch without filing a ticket or calling someone. That matters in real shops where people are spread thin.
But the 10% reveals itself fast when you push harder. What happens when answering the question requires pulling data from three different systems? What happens when the AI needs to actually do something instead of just reporting something? What happens when a junior tech asks a question that, if acted on, would take down a production VLAN? A chat wrapper on top of a single vendor’s controller doesn’t have answers to those questions. It just has answers.
RAG (Retrieval Augmented Generation) RAG is how AI tools pull in data from external sources to answer questions. Instead of relying only on what the model was trained on, a RAG-enabled tool reaches out to live data sources — your CMDB, your ticketing system, your network controller — and uses that information as context when generating a response. Without RAG, your “AI-powered” network tool is answering questions based on general training data. With it, the tool can reason about your network, your tickets, and your devices. The quality of the RAG implementation matters as much as the quality of the model.
The 90%: What Enterprise-Grade AI NetOps Actually Requires
This is the part the demos don’t show you. Not because vendors are hiding it, but because it’s hard to make a stage presentation about access control policies and schema validation feel exciting. It’s the infrastructure that has to exist before the magic works. Here’s what it breaks down to.
Multi-Source Data Connectors
Real networks are not a single-vendor environment with clean, consistent data in one place. Real networks have ServiceNow, NetBox, a CMDB that may or may not be accurate, flow data, syslog, maybe a SIEM, controllers from two or three different vendors, and at least one spreadsheet that someone has been maintaining since the Clinton administration.
An AI tool that only sees its own data is flying blind. It can tell you what it knows. It cannot tell you what matters.
In the Aviz demo, Network Copilot was simultaneously connected to ServiceNow, Fortinet, Palo Alto, Arista, and Dell SONiC. When a ticket came in about an unreachable server, the agent pulled the ticket details, identified the affected IP, checked ACLs on the adjacent switch, and then checked upstream firewall rules. No single one of those systems had the full picture. The AI had to span all of them to get to an answer. That is the actual problem being solved. A chat interface that only talks to your controller cannot do that.
Private, On-Prem LLM Hosting
This one matters a lot and does not get nearly enough airtime in vendor pitches.
Your network data includes ACLs, firewall rules, IP addressing schemes, topology diagrams, change ticket history, and device configurations. You do not want that data training a shared cloud model. You do not want it sitting in someone else’s infrastructure. In many organizations, sending that data to a third-party cloud service would be a compliance violation before you even get to whether the AI gives good answers.
Aviz built their entire platform to run on-prem, with the customer’s choice of LLM, including models the security team has already approved. They also built the flexibility so that if your company has already standardized on a particular model, you can point the platform at it instead of adopting a new one. The LLM is modular. The guardrails and workflows are what Aviz is actually selling.
If a vendor’s AI story requires sending your config data to their cloud, that is a conversation your InfoSec team needs to be in before you sign anything.
Role-Based Access Control That Actually Means Something
In the Aviz demo, Cody McCain walked through an administrative interface where access to devices, data connectors, LLMs, and custom agents can all be scoped by role. A frontline support tech doesn’t get the same AI permissions as a senior network engineer. A user who can read ACL data may not be allowed to make changes. Even the LLMs themselves can be gated, which matters if you’re paying per-token on a more powerful model and you don’t want everyone in the NOC running complex queries against it all day.
None of this is a new concept. It is the same RBAC model you already apply to your network management tools. The point is that the AI layer has to honor it. A tool that bypasses your access controls because it’s “just the AI layer” is not a tool you can deploy in a regulated environment.
Guardrails Against Hallucination
When someone in the audience asked Network Copilot which NCAA basketball team was the best, it responded that it is a network operations tool and doesn’t have information on sports teams. That sounds like a trivial party trick, but it’s pointing at something real. The model is tuned to stay in its lane. More importantly, it is tuned to say it doesn’t know rather than confidently generating something plausible but wrong.
Aviz also built schema-level validation into its data connectors. The model can sanity-check its own outputs against known data shapes. If an IP address appears in an answer but doesn’t match any device in the inventory, that’s a signal that something went wrong. Is this a perfect solution to hallucination? No. LLMs are non-deterministic and always will be. But layering guardrails at the instruction level, the tool level, and the schema level is meaningfully different from hoping the base model gets it right.
Agentic Workflows, Not Just Q&A
This is the biggest gap between the 10% and the full picture.
There is a real difference between a question-answering interface and an agent that can do work. In the Aviz demo, here is what actually happened with a single incident ticket: the agent pulled the ticket from ServiceNow, identified the affected IP from the ticket description without being told explicitly, queried the ACLs on the adjacent switch, moved upstream to check the firewall rules, wrote a root cause analysis, and posted it back to the ServiceNow ticket. The engineer typed a few prompts in plain English. That was it.
That is not a chat interface. That is a workflow engine with an LLM as the reasoning layer. The distinction matters because a workflow engine can be tested, audited, bounded, and improved. A chat interface is only as good as the person asking the questions.
The Demo-to-Production Gap Is Real, and Vendors Know It
Here is a pattern I have seen enough times to recognize it on sight. A vendor demos an AI tool on a laptop, with clean data, a controlled four-device lab environment, and a presenter who knows exactly which prompt to type to make it work. It looks great. Then you get home, start scoping a POC, and discover that your actual environment has 200 devices across six sites, three different OS versions, a NetBox that nobody has updated since the last network refresh, and an L1 tech who is going to ask the AI something completely unexpected at 2 am.
To his credit, Thomas acknowledged this directly on stage. He said something to the effect that demos are great, but if you want to use this in production, you need to have the guardrails in place first. That kind of honesty from a vendor is worth noting, because it means they have thought about the gap instead of just hoping you won’t notice it until after you’ve signed.
The vendors who have done the work to bridge that gap have a specific story to tell. They can tell you about their LLM benchmarking process across releases. They can tell you how they test for quality across standard prompts before they ship. They can tell you how they handle the model, producing different results for the same question on different days. If you ask those questions and a vendor changes the subject, that tells you something, too.
Questions to Ask Before You POC Any AI Network Tool
The next time a vendor walks you through a shiny AI demo, here are the questions worth asking before you agree to a proof of concept.
Where does my network data go, and can this run fully on-prem? If the answer is “we have a cloud option,” and there is no on-prem path, find out what data leaves your environment and under what terms.
What happens when the tool doesn’t know the answer? Ask to see it fail. A tool that says “I don’t have enough information to answer that” is more trustworthy than one that always produces a confident-sounding response.
Can it connect to my existing tools, or only yours? ServiceNow, NetBox, your other vendor’s controller, your SIEM. If the integration story is “we support our own ecosystem,” you are not solving the multi-source data problem.
Who controls what actions the AI can take, and at what granularity? Read-only access and write access are not the same thing. Ask to see the RBAC model.
Can I build custom workflows, or am I limited to what you’ve pre-packaged? The out-of-the-box use cases get you started. Your specific change management process, your specific escalation paths, and your specific risk tolerance are what determine whether the tool actually fits your operation.
How do you test the LLM quality across releases, and can I see that process? If the answer is “we test it,” push for specifics. What prompts? What models? What does passing look like?
Final Thoughts
The AI NOC is not science fiction anymore. What Aviz showed at Network Field Day felt genuinely close to production-ready, not because the LLM was doing something magical, but because they had done the unglamorous work of building enterprise scaffolding around it. Multi-source data connectors. On-prem LLM hosting. Role-based access control. Schema-level validation. A custom agent SDK for operators to build their own workflows. That is what the 90% looks like. None of it makes for a slick two-minute demo. All of it determines whether the tool actually works when your most critical link goes down on a Friday night.
The vendors who are serious about this space have the full story. The ones who are AI-washing have a great demo and a lot of hand-waving. You now know enough to tell the difference.
Disclaimer: This post is based on Aviz Networks’ presentation at Network Field Day 40. As a Network Field Day delegate, my travel and accommodations were covered, but nobody reviewed or approved this content. These opinions are my own.
