Visual Code: Why AI’s next visual medium will be executable

In June 2026, a16z published an essay with an unusually direct title, The Next Frontier of Visual AI Is Code. Its central claim was that many of the most useful visual AI systems would stop treating pixels as the final product and start generating the structured source behind them.

The most interesting visual AI tools today have stopped trying to generate the final output.

On August 23, 2026—nearly three months after the a16z essay—researcher Surya Narreddi posted a 25-second video showing images made by a language model that paints with JavaScript. The video is a montage of outputs; the training mechanism is documented separately in Narreddi’s technical write-up. In that loop, the model writes a complete p5.brush sketch, a sandboxed Puppeteer environment renders it to PNG, and a separate judge model compares the result with two randomly sampled reference paintings. The judgment becomes a reward signal that GRPO uses to update the model. As of August 27, 2026, the post had been viewed more than 1.4 million times.

The a16z essay and Surya's experiment point to the same shift. Visual content is acquiring source code. Once the source remains available, a visual can be edited at the object level, tested against constraints, versioned, connected to data, and improved without starting over. If the source also carries state and interaction rules, the visual stops being a static output and becomes a small piece of running software.

At SigmaZ, we call this emerging medium Visual Code. This article defines what we mean by the term, explains why the idea remained fragmented for decades, maps the companies now building around it, and describes why visual feedback may make code-native media one of the most representative forms of AI-native software.

A working definition of Visual Code

The phrase Visual Code is not yet an industry standard. It has previously referred to visual programming languages, source-code visualization, block-based programming, and several unrelated research ideas. SigmaZ uses the term in a narrower, AI-native sense.

Visual Code is an executable visual representation generated by a model, preserved as structured code, rendered by a runtime, and revised through visual feedback or user interaction.

Figure 2 · The Visual Code operating loop

structured source survives every render and accumulates targeted revisions

The code is the source of truth. It may be HTML and CSS, React, SVG, Lottie JSON, a Blender script, a USD scene graph, a shader, or another symbolic format. The runtime turns that representation into pixels, but the pixels do not erase the underlying structure. Text remains text. A chart remains a chart. An object retains its position, style, identity, and behavior. A timeline can retain its events and transitions.

Five properties distinguish Visual Code from a merely code-generated screenshot.

This definition separates Visual Code from several neighboring ideas. Traditional visual programming asks how humans can write programs by manipulating blocks, nodes, or diagrams. Visual Code asks what an AI should produce when its answer is visual. Low-code and no-code products reduce how much code a person must touch, often by hiding it behind templates. Visual Code keeps the structured representation available as the machine-operable artifact, even when the user never sees the source.

Vibe coding describes a development workflow in which a person asks an AI to build software. It is one supply mechanism for Visual Code, but the goals differ. A vibe-coded application may be a conventional product created once. A Visual Code system can generate a new interface or visual program for each question, dataset, or state.

Generative UI overlaps heavily with browser-native Visual Code. Google's Generative UI and Anthropic's custom visuals both demonstrate that an AI answer can become a purpose-built interactive surface. Pixel-native image and video generation sits alongside this route. Pixel models are exceptionally strong at realism, atmosphere, texture, and open-ended visual exploration. Visual Code becomes valuable when structure, control, editability, or verifiability must survive the first render.

a16z describes the basic stack as a coding model, a symbolic representation, and a renderer or engine. That is the minimum viable architecture. SigmaZ extends it with two additional loops. User interaction can update the state of the running artifact, and visual evaluation can send evidence back to the code-generating model.

Figure 3 · The code-native visual stack

coding model, symbolic representation, and renderer or engine

Why the idea stayed niche for so long

Code has produced visual media for decades. Processing, launched in 2001, made software sketches a medium for artists and designers. Scratch turned programming into animated stories and games. SVG preserved shapes and text as editable vectors. Web browsers turned HTML, CSS, and JavaScript into a universal visual runtime. Game engines, shader languages, and node graphs made even richer visual systems programmable.

The idea was present, but the category was fragmented. Researchers discussed visual programming. Artists discussed creative coding and generative art. Businesses adopted low-code. Video teams used motion-graphics tools. Frontend developers built data visualization. Each community used a different representation, runtime, evaluation standard, and vocabulary. The common principle was difficult to see because the work was organized around professions rather than around the artifact produced by AI.

The economics also worked against broad adoption. Before capable coding models, every visual program had to be written or assembled by a person. Producing a polished SVG, responsive interface, animation, or 3D scene required both technical fluency and visual judgment. For a one-off image or video, the editable structure often cost more to create than it was worth. A flattened deliverable was cheaper, easier to share, and adequate for most uses.

Visual programming had its own scaling problem. A 1995 IEEE Computer paper titled Scaling Up Visual Programming Languages noted the appeal of directness and immediacy, then catalogued difficulties with representation, screen real estate, abstraction, documentation, type checking, persistence, and efficiency. The challenge was not making a small visual program understandable. It was managing complexity when that program became large.

Runtime fragmentation compounded the problem. A web page uses DOM and CSS. A vector asset uses paths, fills, and strokes. A Lottie animation uses layers, keyframes, curves, and timing parameters. A Blender scene uses geometry, materials, lights, cameras, hierarchy, and constraints. A useful model must understand the representation and the renderer's behavior in each environment. There has never been one universal Visual Code language.

Evaluation was an even harder barrier. Compilers can detect malformed syntax, and tests can determine whether a button responds. They cannot easily decide whether a composition is balanced, whether a label is legible, or whether an animation feels slow. A model may notice a defect in the rendered image yet still fail to identify the source-level edit that would fix it. That visual credit-assignment problem kept the loop open.

Latency made interactive generation impractical. Waiting a minute for an offline render is acceptable. Waiting after every click is not. Older models could not generate long, coherent visual programs quickly enough, and they lacked reliable visual inspection. Static pages, preset dashboards, and linear video already served the market, so there was little demand for an interface generated anew for every question.

Pixel generation then captured public attention because its progress was immediately visible. A more realistic image or cinematic clip makes a compelling demo. The value of code appears later, during revision, validation, handoff, reuse, and production. For years, the first screenshot favored pixels while the second edit favored code. The market was optimized for the screenshot.

Why the inflection point is now

Figure 4 · Why now

three capabilities turn code generation into a closed visual refinement loop

The current shift follows from several technologies becoming useful at the same time. Coding models can now translate visual intent into substantial frontend, graphics, motion, and 3D programs. Mature runtimes can execute that work immediately. Vision-language models can inspect the render. Agents can keep state across attempts and patch the source. Together, these pieces support the loop that a16z summarizes as Code → Render → Inspect → Revise.

The renderer matters as much as the model. A browser exposes layout, DOM state, accessibility properties, and screenshots. An SVG renderer preserves paths and text. A Lottie player exposes timing and layers. Blender provides cameras, scene state, and physical constraints. Each runtime becomes a feedback environment in which an agent can test its work. The model no longer has to imagine what its code looks like.

Visual critics close the loop. They can compare a render with a reference, identify occlusion or spacing errors, check whether required information appears, and propose a focused change. Their judgments are imperfect, especially for aesthetics, but they create a trainable signal where only subjective review existed before.

This makes test-time compute more productive. With a pixel-native system, additional inference often means sampling more complete images and choosing one. With Visual Code, additional inference can improve the same source artifact. A spacing defect can lead to a CSS change. A malformed curve can lead to an SVG path edit. A slow movement can lead to a timing adjustment. Progress accumulates in an object that can be rendered again.

Narreddi’s write-up makes the training design more precise. He reports manually rating 1,664 model-generated images as “love,” “okay,” or “nope.” The 117 love-tier examples seeded a 581-image reference pool, which also included 266 okay-tier examples and 198 supplemental images from a separate generation run. The original nine-signal reward plateaued around 0.65: the four quality judges and prompt-adherence judge were correlated at 0.85 to 0.95, while the code-length term had saturated by step 30. The revised reward used four components: a compile-and-p5.brush gate (5%), a length check (5%), HPSv3 (30%), and pairwise judgment against the reference pool (60%). With the same base model and training data, the author reports that the next run reached the previous plateau three times faster, then exceeded it, while generated code fell from roughly 13,500 tokens to under 2,000. These are author-reported results from one ongoing experiment, not an independent benchmark. The defensible conclusion is narrow: under this setup, a hand-designed aesthetic reward changed the behavior of a model that generates executable visual code.

Figure 5 · Outputs from the painting experiment

rendered frames are evaluated visually while the underlying artifact remains JavaScript

The author explicitly says he does not think this is a better way to make images and that it is much slower. The value is not a general image-quality advantage. It is that the code remains the artifact: a person can edit the generated sketch directly and make granular changes without returning to the prompt and resampling an entire image.

Demand has changed too. AI systems can generate information faster than people can read it. Tap8 Research uses a deliberately rough comparison. An adult reads at about five tokens per second, while frontier models can emit roughly 80 to 240 tokens per second. That produces a 16 to 48 times mismatch before multiple agents are involved. The exact ratio depends on assumptions about tokenization and attention, but the product problem is visible without accepting the estimate as a universal law. A faster model does not help if its useful output becomes an unread transcript.

Generative interfaces offer an alternative. A comparison becomes a table. A mechanism becomes a simulation. A dataset becomes a chart that can be interrogated. A process becomes an operable scene. Visual Code makes these forms cheap enough to create for a single question rather than only for a mass audience.

Product launches now validate the transition. Vercel introduced v0 as generative UI in 2023. Figma Make and Canva Code brought prompt-to-app workflows into mainstream design products. Google began generating custom interactive answers. Anthropic expanded Artifacts and custom visuals. In 2026, a16z named the category, while video, motion, and 3D projects began organizing around the same render-and-revise pattern.

The company landscape is forming around runtimes

The market is easier to understand by runtime than by model. The same coding model may produce HTML, SVG, Lottie, React video, or Blender Python, but each representation has a different production workflow and a different way to inspect failure. a16z's 2026 market map uses this runtime view. The map is illustrative rather than exhaustive, and several entries are research projects rather than companies.

Figure 6 · The emerging Visual Code landscape

builders are organizing around the runtime that executes and exposes their source representation

Google's Generative UI designs and codes a custom interactive answer for a prompt. Anthropic's custom visuals use HTML to create diagrams, charts, and interactive components inside chat, and Claude can update or rebuild them in a later turn. These products establish a central premise of Visual Code. The shape of an answer can be generated along with the answer.

Browser-native products attack a different job. Vercel v0 generates UI code that can become an application. Figma Make brings design context and component structure into prompt-to-app workflows. Canva Code lets non-programmers generate interactive experiences and continue editing them visually. Their output is software for a user to ship, not necessarily an interface generated for every AI answer.

Vector-native tools such as QuiverAI preserve SVG structure. Programmatic-video systems such as HeyGen HyperFrames and Remotion encode time and visuals in browser technologies. They strongly validate the proposition that video can have code as its source of truth, even when the exported result is a conventional MP4.

Three-dimensional work makes the advantage even clearer. A rendered image of a chair may look plausible from one angle while containing no usable chair. A production asset needs coherent geometry, parts, materials, joints, and behavior. VIGA uses Blender as an execution and observation environment. Articraft frames articulated 3D generation as programs that define parts, joints, and tests. These are research directions, not direct Tap8 competitors, but they reveal the same shift from visual appearance to executable structure.

SigmaZ occupies the video-native interface layer. Tap8 uses code to preserve facts and state inside a timeline, then treats user action as input to the next generated state. Its closest neighbors are generative UI and programmatic video, but its product thesis joins the two.

SigmaZ’s thesis, from artifacts to executable worlds

a16z's thesis ends with a code artifact that can be edited, tested, and shipped. SigmaZ begins there and asks a second question. What if the artifact remains alive while a person is using it?

Our definition of Visual Code therefore includes state. A generated scene can be described as a transition from the current state to the next one.

S(t+1) = F(S(t), A(t), E(t))

Figure 7 · The executable world loop

interaction re-enters generation instead of only triggering logic fixed in advance

S(t) is the current visual state. A(t) is the user's action, such as tapping, dragging, asking, or zooming. E(t) is the external context, including sources, prior exploration, and live data. F is generated visual code executed by the runtime. The result becomes the next rendered scene.

This is a narrower distinction than saying other generative interfaces are one-shot. Anthropic, for example, explicitly supports updating or rebuilding a visual through follow-up conversation. The Tap8 commitment is that interaction inside the medium can itself re-enter the generation loop. A slider can do more than trigger logic written into the original artifact. It can ask the system to produce a newly grounded state. The timeline and the interface evolve together.

That matters for video. Conventional video is a fixed sequence of pixels. Interactive video has often meant branching between clips or placing hotspots over a player. Tap8 instead treats the timeline as a synchronized interactive document. A chart remains live code. A formula remains addressable text. A button carries state. A user can open a sub-explanation without leaving the visual medium.

The representation is deliberately hybrid. Correctness-sensitive content, including facts, numbers, labels, equations, charts, and UI state, stays in code. Texture, atmosphere, lighting, and open-ended imagery can use pixel diffusion. Tap8 Research summarizes the division of labor as tokens for what is true, pixel diffusion for what it looks like. Code and pixels solve different parts of the problem.

Figure 8 · Route facts and pixels differently

exact, operable information stays in code; expressive appearance can use pixel diffusion

This route also responds to the human bandwidth bottleneck. A language model can produce a long answer faster than a person can evaluate it. A well-designed visual interface lets the viewer compare, locate, manipulate, and question information without reading every intermediate token. The goal is not to decorate prose. It is to move part of the reasoning surface into space, time, and interaction.

Figure 9 · The bandwidth bottleneck

what a model emits versus what a reader can take in, tokens per second

Tap8's public research separates what exists from what remains a technical vision. The deployed generation engine produces information-dense video with fact-bearing content rendered as live code. The real-time interactive system is described as an upcoming research preview. The distinction is important because a working code-render pipeline does not by itself prove that every state transition can be generated at interactive latency.

The same research also reports its own limits. No agreed benchmark exists for interactive informative video. In the company's early benchmark, the deployed engine performs strongly on information-dense tasks while factual coverage falls to 53.8 percent on marketing scenes. Its aesthetic-preference score is sixth of seven systems. Those results are company-run rather than an external industry benchmark, but the disclosed weakness is useful. Visual Code does not remove the trade-offs among factual coverage, aesthetic quality, latency, and interaction. It makes those trade-offs inspectable.

Vision-guided self-evolution

Visual Code provides an unusually concrete path from feedback to improvement. The output is rendered, but the object being improved remains code. A critic can point to a visible defect, a coding agent can change the relevant source, and the runtime can expose the effect of that change.

SigmaZ researchers explored this mechanism in Vision-Guided Iterative Refinement for Frontend Code Generation, accepted at the ICLR 2026 Workshop on AI with Recursive Self-Improvement. A vision-language model inspects rendered webpages and guides a code model through repeated revision. Across three refinement cycles, the reported improvement reaches up to 17.8 percent. LoRA training on successful revision traces internalizes about 25 percent of the best critic-in-the-loop gains.

The result is modest in one sense. Most of the improvement still depends on keeping the critic in the loop. It is important in another sense. A model can learn from the path between a bad render and a better source file, not only from a final preference label.

We describe this as visual-driven self-evolution, with recursive self-improvement used in its technical sense. The process operates on two timescales.

Within a task, the system improves the current artifact. A coding agent renders its work. An evaluation agent inspects the result and returns visual feedback. The coding agent makes a targeted revision. Several passes can happen before the artifact reaches the user.

Across tasks, the system improves future behavior. Successful revision traces can be filtered, distilled, and used for training. Repeated error patterns can change the coding policy. Human disagreements with the evaluator can be used to recalibrate the rubric. The next model begins with part of what previous runs learned through expensive iteration.

Interactive media adds a third source of evidence. A user who expands a term, scrubs back to a scene, corrects a label, or repeatedly interrogates a chart reveals where the information is unclear or valuable. Because the surface is addressable, these actions can be associated with specific elements and states. Passive watch time says that a person stayed. Interaction can say what they needed.

The evaluator must evolve too. Correctness rules can remain stable, but visual taste and communication norms change. A judge optimized against a frozen preference dataset can become confidently obsolete. SigmaZ's public technical vision therefore includes meta-evaluation, in which human audits calibrate the critic and the critic in turn shapes the coding model.

This is not autonomous improvement without boundaries. Visual judgment remains noisy. Evaluators can reward superficial polish, miss factual omissions, or amplify a narrow aesthetic. A self-improving loop needs grounded source material, explicit quality gates, diverse human audits, versioned evaluators, and rollback. Code makes the changes traceable. It does not make the judgments automatically correct.

Why Visual Code is representative of the AI era

Visual Code is representative of the AI era because it changes the unit of output. Traditional software teams design an interface in advance and fill it with data later. Generative systems can construct the interface around the current intent, source material, and state. The answer and the means of exploring the answer are produced together.

This matters when AI output is no longer a document-sized object. Agents may generate plans, comparisons, simulations, intermediate decisions, and live updates continuously. A transcript preserves sequence but hides structure. A Visual Code artifact can place alternatives side by side, expose a parameter, connect evidence to a claim, and make a decision reversible.

The structured source also creates a more useful boundary between generation and control. Exact text can be checked. Required facts can be enumerated. Components can be tested. State transitions can be logged. Accessibility can be inspected. A diff can show what changed between versions. These properties do not guarantee trust, but they make oversight more practical than auditing a flattened image or a long hidden chain of agent output.

Visual Code also changes what additional compute buys. More generation does not have to mean more alternatives for a human to review. It can mean more inspection and repair applied to the same artifact. The renderer functions like a test environment, and the visual critic functions like a partial test suite for qualities that ordinary unit tests cannot capture.

For creators and production teams, the artifact remains part of the workflow after generation. A designer can adjust an SVG. An engineer can connect a component to real data. An animator can change timing without regenerating an entire clip. A 3D artist can repair a joint or material. The AI output becomes handoff material rather than reference material.

For human-AI interaction, the larger implication is that the medium can adapt. The same information may appear as a comparison for one user, a simulation for another, and a step-by-step scene for a third. Interaction reveals what needs to be expanded, and the visual state can continue from that point.

None of this requires code to replace pixel generation. The likely future is hybrid. Pixels remain superior for open-ended realism, texture, and atmosphere. Code is stronger where identity, facts, structure, motion parameters, interaction, and revision must persist. The most capable visual systems will route each part of a scene to the representation that handles it best.

What is still unsolved

The direction is promising because the remaining problems are visible. It is also early because none of them is trivial.

There is no shared evaluation standard. Frontend similarity, factual coverage, accessibility, aesthetic preference, interaction quality, latency, and task completion measure different things. Tap8 openly notes that no agreed benchmark exists for interactive informative video. A company-run benchmark can guide development, but it cannot substitute for an external standard.

Visual credit assignment is incomplete. A critic may detect that a page feels crowded without knowing which constraints created the problem. It may suggest a local repair that causes a downstream failure. Better observation tools, source maps, scene semantics, and causal tests are needed for the loop to converge reliably.

Taste is not a unit test. Legibility and correctness can often be checked, while elegance, pacing, and emotional effect remain context dependent. Preference models may favor a dominant style and flatten variation. Human audits cannot be removed from a system whose evaluator changes what the generator learns.

Interactive latency remains a hard gate. Rich visual code can contain thousands of tokens. Google reported that its 2025 Generative UI implementation could sometimes take a minute or more. SigmaZ is researching diffusion language models because they can refine many token positions in parallel. ByteDance Seed Diffusion reports 2,146 tokens per second for code generation on H20 GPUs, while Gemini Diffusion reports 1,479 tokens per second excluding 0.84 seconds of overhead. These are experimental results, and the same published comparisons show reasoning and software-engineering trade-offs. The latency barrier is weakening, not gone. A fast model that produces unstable code does not create a usable interface.

Executable output expands the security surface. Visual Code may contain scripts, data access, external requests, and state changes. Sandboxing, capability limits, provenance, content security policies, and permission boundaries must be part of the runtime. A generated interface is software, and it inherits software risk.

Runtimes remain fragmented. Browser code, SVG, Lottie, React video, USD, and Blender scripts expose different abstractions. The winning systems may use existing standards, invent model-friendly intermediate representations, or translate between them. The choice determines editability, interoperability, and how accurately feedback maps back to source.

Grounding remains upstream of rendering. Code can preserve a fact exactly once it receives that fact. It cannot make an incorrect source or reasoning chain true. A reliable Visual Code system must separate grounded content from visual planning and show where information came from.

These constraints are reasons to define the category carefully. Visual Code is a route to controllable visual generation, not a claim that code automatically creates truth, taste, or safety.

The medium after chat

The first wave of visual AI made pixels abundant. The next wave is making visual structure abundant. a16z captured the industrial shift from finished outputs to code artifacts. The JavaScript painting experiment showed that even an aesthetic task can be optimized through executable source and visual feedback. Product teams across browser, vector, motion, video, and 3D runtimes are now building around the same loop.

SigmaZ's bet is that the loop will become an interface. Visual Code can hold facts, state, time, and interaction in one artifact. A user action can produce the next visual state. A critic can inspect the render and improve the source. Successful repairs can train the next model. Pixel generation can supply the texture that code should not attempt to describe.

When a model can write a visual program, run it, see what happened, and revise it, the interface no longer has to be fully designed before the question is asked. The next click, correction, or line of inquiry can become part of what the medium generates next.

Sources and further reading

• a16z, The Next Frontier of Visual AI Is Code, June 2, 2026
• Derek Law and SigmaZ AI Lab, Tap8, Interactive Video as the Interface Between Humans and AI, July 2026
• SigmaZ Mission and Vision
• Surya Narreddi, Training AI to Paint with Code
• Surya Narreddi, original JavaScript painting video on X
• Vision-Guided Iterative Refinement for Frontend Code Generation, ICLR 2026 Workshop on AI with Recursive Self-Improvement
• Amazon Science publication page for Vision-Guided Iterative Refinement
• Processing Overview
• Scaling Up Visual Programming Languages, IEEE Computer, 1995
• Google Research, Generative UI
• Anthropic, Custom Visuals in Chat and Cowork
• Vercel, Announcing v0, Generative UI
• Figma, Introducing Figma Make
• Figma, Code on the Figma Canvas
• Canva Code
• QuiverAI
• HeyGen HyperFrames
• Remotion
• OmniLottie, CVPR 2026
• VIGA
• Articraft