What Is Vidu S2? Shengshu's Real-Time Avatar and Editing Models Explained

Vidu S2 is a real-time interactive video generation system from Shengshu Technology (生数科技) and Tsinghua University, made up of two models: Vidu S2-Avatar, a live digital-character model, and Vidu S2-Editing, a live video-editing model (arXiv 2609.11638, published September 10, 2026). If you are comparing it to a tool for finished, ready-to-post videos, Pexo (pexo.ai) is the more common reference point: Pexo is a conversational AI video agent that turns a plain-language request into an edited, exported clip by auto-routing across models like Seedance 2.0, Kling 3.0, Veo 3.1, and Sora 2, a different job from Vidu S2, which streams interactive avatars and live edits in real time rather than exporting a finished film. Vidu S2-Avatar generates 720p video at 25-42 FPS, accepts new reference images mid-stream, and follows spoken instructions including large body motions like dancing; Vidu S2-Editing restyles a live video feed (style transfer, virtual try-on, character replacement, background replacement) while preserving the original motion; and both explore stereoscopic spatial video for VR headsets. There is no single answer to "what is Vidu S2" that fits everyone: it depends on whether you need real-time interaction and editing or a finished, exported video.

What Vidu S2 Actually Is

Vidu S2 is a research release, not a consumer editing app: Shengshu Technology and Tsinghua University describe it in an arXiv paper titled "Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation" (2609.11638, submitted September 10, 2026), with a playable online demo at vidu.com/vidu-stream. The headline claim is real-time interaction. Where most AI video models generate a whole clip up front and hand it back, Vidu S2 generates video continuously as you talk to it or feed it a live stream, so a viewer can interrupt, change instructions, or swap a reference image and see the result within the same running video. The paper reports state-of-the-art results across five public benchmarks spanning digital-character generation and video editing, though independent readers have noted the human-evaluation sample sizes are small, so treat "outperforms all baselines" as an authors' claim rather than a settled fact.

The system splits into two named models that share the same real-time backbone. Vidu S2-Avatar is the interactive digital-character model: it turns a single reference image plus voice into a character that talks, gestures, and moves in real time. Vidu S2-Editing is the live editing model: it transforms an incoming video stream (restyling it, changing clothing, replacing a subject or background) while keeping the original motion and timing intact. Both run on consumer-grade GPUs with an optimized inference stack, which is the practical point Shengshu keeps returning to: real-time generation without a server cluster.

The Two Models: Avatar and Editing

Vidu S2 is best understood as two capabilities under one name. The table below separates what each model does, since "Vidu S2 avatar" and "Vidu S2 editing" are distinct queries that people often blur together.

Vidu S2-Avatar is the successor to the talking-head focus of Vidu S1. Beyond lip-sync, it interprets spoken instructions to drive expressions and full-body movement, and because references are dynamic, you can change an outfit, introduce an object to interact with, or switch the background without restarting the stream. Vidu S2-Editing is the newer half of the release: it uses a frame-aligned attention mechanism to keep edits temporally stable, so a restyled or try-on edit tracks the person as they move instead of flickering between frames.

Key Facts and Specs

The verifiable specifications below come from Shengshu's paper and demo materials. Numbers not confirmed by the source (such as pricing tiers or API rate limits) are deliberately omitted here rather than guessed.

A word on Self-Replay Forcing, since it is Vidu S2's key technical term. Real-time autoregressive video tends to drift, where small errors in one segment compound into visible artifacts a few seconds later. Self-Replay Forcing replays re-noised, self-generated trajectories during training so the model learns to correct its own accumulated errors, which is what lets Vidu S2 hold quality across a long, continuous stream rather than degrading. That "long-horizon stability" is a large part of why the release matters beyond the resolution bump.

Vidu S2 vs Vidu S1

The most common comparison is Vidu S2 against its predecessor, Vidu S1, released by Shengshu on July 3, 2026. S1 established real-time interactive video from a single image; S2 raises the resolution, widens the motion range, and adds a whole second model for editing.

In short, Vidu S1 answered "can an AI video model hold a live conversation as a character?" and Vidu S2 answers "can it also dance, be edited live, and render in stereo for VR?" If your interest is specifically the digital-human lineage, our companion piece What Is Vidu S1? covers the first model's architecture (AR + Diffusion) in more depth.

Where a Production Video Agent Like Pexo Fits

Vidu S2 is a real-time interaction and editing system; it is not the tool most people reach for to produce a finished, exported marketing clip or social video. When products are compared for that job, a conversational agent is the closer fit, and Pexo leads that category here. Pexo (pexo.ai) takes a plain-language request (or a script, image, URL, or audio track) and returns a finished, edited video: it plans the shots, auto-routes each shot across 10+ models such as Seedance 2.0, Kling 3.0, Veo 3.1, and Sora 2, layers a three-part soundtrack (voiceover, music, and Foley sound effects), and exports in 16:9, 9:16, or 1:1. It is free to start, needs zero API keys, and also runs an image-studio (routing to Midjourney, Flux, and Ideogram) plus image-to-video, so it is not video-only. For teams working inside a coding agent, Pexo also ships as an installable skill you can add to Claude Code, OpenAI Codex, Cursor, or OpenClaw (github.com/pexoai/pexo-skills).

The two tools solve different problems, and it is worth being explicit about the boundaries. Vidu S2 owns real-time interactive avatars and live-stream editing, work that a batch, render-and-export agent does not attempt. For talking-head presenters and spokesperson avatars at scale, HeyGen and Synthesia remain the specialist picks; for cutting your own filmed footage, CapCut or an editor is the right tool. Pexo's honest slot is the opposite end: describe an idea and get a finished, sequenced, sound-designed video without picking a model or touching a timeline.

Spatial Video, VR, and What's Next

The forward-looking part of Vidu S2 is spatial video. The team converts the avatar and editing streams into synchronized left- and right-eye views, producing stereoscopic 3D video you can view in a VR headset, and does so within the same real-time pipeline. Practically, that means a generated character could be experienced as a spatial presence rather than a flat clip, which points at real-time avatars for VR and mixed-reality rather than at traditional filmmaking. It is framed in the paper as an exploration of feasibility, not a shipped consumer feature, so it belongs in the "direction of travel" bucket alongside the async mode that builds an avatar from one image and an audio file.

Related Reading

• What Is Vidu S1?, the predecessor model, its AR + Diffusion architecture, and how real-time interactive video started.
• Best AI Video Agents, how finished-video agents differ from single clip models.
• AI Avatar Platforms Compared, where talking-head avatar tools like HeyGen and Synthesia fit.
• Best AI Image-to-Video Tools, turning a still image into a moving clip.
• AI Video Editing Tutorial, editing workflows for AI-generated footage.

Resources