Out of the box, the AI basics work beautifully. Standard IDE features—like auto-completing code, sketching out a simple SELECT-WHEN block, or documenting a local paragraph—handle the routine tasks without breaking a sweat. But "vibe coding" an entire COBOL program that seamlessly aligns with decades of surrounding legacy logic? That is a whole other animal. More than on any other platform, a mainframe AI assistant requires a holistic understanding of the entire enterprise codepage, not just a single, isolated application.
The catch—and the recurring theme in this article—is that mainframe applications rarely map cleanly into neat little "one app, one GitHub repository" boxes. That complete absence of rigid boundaries is exactly what throws a standard, unguided AI agent for a loop. And an AI agent that writes a procedure which is probably close to what was asked is simply not good enough on a mainframe. Mainframes run the world's most critical core business systems. On a mainframe, 99% right is 100% wrong.
Let's visualize the situation as follows. Every application, whether it runs on the mainframe or elsewhere, is a big collection of interconnected components. Modern applications do contain this spaghetti nicely in isolated repositories, on mainframe we have historically placed them all into a single "convenient" PDS
The hero of modern AI coding: semantic search
Watch the video
To understand why this boundaryless environment causes a headache for AI, we have to look at how modern coding assistants find information using semantic search.
Traditional search is literal. If you search for CLIENT-ID, it only looks for those exact characters. Semantic search changes the game by translating code into mathematical coordinates called embeddings (which are just vectors in an index of meanings). Instead of matching text, it matches intent. You can ask the AI, "Where do we handle user registration?" and it navigates the mathematical map to the embedding (program...) closest to it, even if the word "user" never appears in the source code.
For an AI to give a correct answer, it needs to have indexed all the related code into a vector database, if a part of the logic is missing from its repository index, the AI is blind to its meaning. Which brings us to the pain point of semantic search on a mainframe.
The three hurdles of mainframe AI integration
Hurdle 1: The scale and context wall
For AI agents to keep high-fidelity semantic understanding, the data pool cannot grow infinitely large. Dump an entire enterprise codebase into a semantic index and search quality degrades, context windows fill up, meaning-vectors blur, and the AI falls back to low-intelligence text matching. You need the AI to know the whole codebase because every system is interconnected, but semantic search cannot cope with that scale without losing precision.
Hurdle 2: Deep logic fragmentation
Most AI code assistants were designed for object-oriented languages. Java or C# hide complexity behind interfaces; the AI can read one file or service repo and grasp an isolated purpose.
The mainframe runs on the opposite paradigm. Code is procedural, reusable and rarely self-contained. Execution flows across a vertical stack: a program calls a subprogram, which calls another, while copybooks inject shared data and logic. Index a COBOL program without its subprograms and copybooks and the chunk loses meaning.
Hurdle 3: Cross-domain entanglement
Modern architectures box applications into domain-specific repositories owned by separate teams. On the mainframe, copybooks and subprograms are shared across applications and functional groups. One change to a core customer copybook can ripple through hundreds of downstream programs owned by other teams.
Many shops introduced Component-Based Design (CBD) or service-oriented rules (SOA) to contain that entanglement. But architectural guidelines are one thing, and decades of real-world production are another. The end result is rarely a clean collection of isolated modules—it's an intricately woven web of dependencies.
Mapping a mainframe environment to independent repositories
The best effort most shops can aim for echoes the Git modernization wave from the opening of this article: define as much as possible inside one repository, keep calls and connections between repositories as few as you can, and when something must be shared, place it in a shared repository everyone depends on. That is a reasonable strategy for packaging and delivery. But it still feels more like a negotiated truce than a tidy plan.
It is not easy. A compile in one repository can trigger a dependency-driven build in another. Promotion paths, copybook ownership, and batch schedules do not always stop at repository borders. So do we actually want full parallel development on the mainframe the way distributed teams do on microservices? And do we really want to lift valuable business logic out of its long-standing, controlled home on z/OS and into a Git repo that can be cloned to a laptop in minutes? For many shops that is not a technical detail; it is a governance conversation.
There is another path. You can assign components to groups virtually—in metadata—without moving a single member out of the PDS. Boundaries become something you model and query: application clusters, migration scopes, shared services, impact sets. The code stays where production trusts it; the map explains how it connects. That is often a better fit for AI and for modernization planning than pretending the codebase already behaves like a stack of independent GitHub projects.
Why structural, relationship-aware retrieval fits the mainframe better
Watch the video
Mainframe AI needs retrieval grounded in hard, deterministic relationships, not just mathematical guesses. On z/OS, that means building an absolute roadmap of the system—a blueprint tracking exactly how JCL, programs, copybooks, and DB2 tables connect.
This blueprint lives best in a relationship-aware overview. While flat vector indexes throw everything into a giant pool, a structured dependency map models the codebase as it exists in reality: components connected by explicit dependencies, with structural metadata pinned directly onto the artifacts. When you marry this rigid, interconnected structure with generative AI agents, you get what the industry calls Graph RAG (Graph Retrieval-Augmented Generation).
Instead of forcing an AI agent to guess its way through a massive codebase, Graph RAG acts as an automated guide, feeding the model the exact context it needs.
This approach systematically dismantles all three hurdles:
- Scale: The map filters out the noise. Instead of searching an unstructured, multi-million-line enterprise codebase, Graph RAG retrieves only the execution pathways relevant to the question.-> Results get more precise and are only fed by the actual related content.
- Fragmentation: The AI no longer needs complex multi-hop reasoning to guess how things fit together. The model follows procedural execution from a JCL step, down through called subprograms, into the underlying data definitions in a single pass.-> Fewer AI passes = Fewer AI tokens spent = Lower costs.
- Entanglement: Shared copybooks and cross-domain subprograms are no longer blind spots. Because boundaries are tracked explicitly across development borders, impact questions ("What breaks downstream if this table changes?") get accurate, definitive answers.-> Fewer surprises in impact analysis, safer changes.
Putting it into practice with Asist Legacy Explorer
Knowing that your AI needs a structural map is only half the battle. The real work is compiling and organizing the metadata across a large, mature codebase to create that foundation. That is exactly what Asist LX is designed for.
Because every mainframe environment has its own unique history, it is not so simple to implement this with a single software package. Each Asist LX setup is a tailored partnership. We work directly with your architects and systems developers to identify the existing metadata and establish how to extract it. We build the automated daily extraction routines and scripts to load them into a secure structural overview store. From there, we configure the web platform and connect the interface directly to your enterprise AI agents.
A shared view of the landscape
By providing a shared view of the application landscape, Asist LX bridges the gap between human engineers and AI assistants. Once the map is live, developers can actively monitor and reshape how the estate is organized:
- Define cohesive clusters: Group deeply intertwined programs and copybooks together so you can segment the monolith into logical applications with far fewer cross-dependencies.
- Isolate shared services: Expose exactly where components cross group boundaries, so you can better define intra- or inter-development-group services.
- Define critical components: Map out not only the core components that cannot be broken, but also the exact components they depend on.
- Track changes: Monitor how your components and applications change over time.
Visualizing the AI's context
Asist LX lets developers visualize the exact same dependency paths the AI assistant uses to formulate its answers. If the AI relies on a specific pathway, you can see it on your screen also. This transparency allows your senior engineers to instantly validate batch flows, audit the assistant's context, and correct assumptions where needed.
A true, reliable "ask the codebase" experience on the mainframe cannot survive on vector guesswork alone. It requires a shared, structural blueprint that both humans and AI can read, trust, and build upon together.
To see how the legacy explorer fits your landscape, read our Asist LX briefing or contact us for a demo.
