# Build your own video platform > The long-form guide to shipping an agentic or editor-style video product on the API: the document format, the command reducer, previews, renders, costs and both auth models. A guide for a developer who wants to ship a video product without building a renderer. Everything below was checked against the live API before it was written. Field names, credit costs, scope names and status strings are what the server actually returns, not what a spec says it should. Where something is not built yet, it says so. Base URL: `https://api.vidmoat.com/v1`. (`/api/v1` resolves to the same place, but use `/v1`: it is the supported path, and it avoids a redirect, which is where `Authorization` headers get dropped.) --- ## 1. What Vidmoat gives you, and what it does not ### You get - **A document format.** A project is one JSON document: settings, tracks, clips. Every clip field that affects the output is in it: transforms, crops, keyframes, colour grade, effects, masks, audio DSP, transitions. It is readable, diffable, and you can hold it in your own database if you want to. - **A command reducer.** ~80 named operations (`addClip`, `sliceClip`, `addCaptions`, `cutRanges`, `autoDuck`, `addKeyframe`, `applyMotionPreset`, `addAdjustmentLayer`, …) that take a document and produce a new one. Published verbatim at `GET /v1/schema/commands`. - **A render farm.** `POST /v1/renders` queues a job; a worker composites the document in Chromium, mixes audio through ffmpeg, and hands you an MP4/WebM/GIF URL. You never install ffmpeg. - **Generative media.** Text-to-image, text-to-video, text-to-speech, AI stickers, behind a credit meter, with refunds when the provider fails. xAI Grok is the default; images can also route to OpenAI or Vertex, speech to OpenAI, and video to HappyHorse or Wan. - **Transcription.** Word-level timings in exactly the shape the `addCaptions` command consumes. Plus a free mode that distributes a script you already have across a known duration, with no provider call at all. - **Preview.** Three representations: cheap JSON metadata + layout lint, a single rendered JPEG frame, and the whole interactive composition as HTML you can scrub client-side at zero server cost. - **A planner.** `POST /v1/ai/agent` turns plain English into a validated command program and applies it. ### You bring The product. The UI. The users. The onboarding, the pricing page, the retention loop, the reason anyone opens your app instead of CapCut. ### What is not there Say these out loud before you design around them: - **Render webhooks are available in the developer console.** Subscribe to `render.completed` and `render.failed` (plus credit, app and Telegram bot events), verify signatures and inspect delivery logs at `/developer/webhooks`. Public `/v1/webhooks` management is not shipped. Poll `/v1/ai/jobs/{id}` for video generation jobs and keep bounded render polling as a recovery path. See the [webhook guide](https://developer.vidmoat.com/developer/docs/webhooks). - **`ai.analyze` is not exposed.** The scope exists. The analysis (shot boundaries, loudness, voice activity) runs internally. There is no `POST /v1/ai/analyze` route. Do not build a cut-decision engine that assumes you can ask Vidmoat what is in a frame. - **No idempotency keys.** There is no `Idempotency-Key` header. Commands are not idempotent: posting `addClip` twice gives you two clips. §8 explains what to do instead. - **No chunked upload.** `POST /v1/media/uploads` buffers the whole file in memory and refuses anything over 32 MB. Bigger files go through `POST /v1/media/import`, which streams from a URL you host (up to your plan's upload cap or 512 MB, whichever is smaller, with a 120-second timeout). - **No browser-facing API.** An API key is a server credential. It must never reach a browser, and CORS on `api.vidmoat.com` is configured for Vidmoat's own hosts, not for arbitrary third-party origins. Your web UI talks to your server; your server talks to Vidmoat. §4 shows the one place this matters architecturally. - **No frame-exact local scrubbing.** The interactive preview is a real browser playing your media over the network, driven by the page's own clock. It is excellent for composition and timing decisions. It is not a broadcast monitor. If your users are colour-grading 4K ProRes and need guaranteed frame accuracy with no network, **Vidmoat is the wrong choice**: you want a local NLE, and no amount of API is going to fix that. - **No social publishing.** No scope grants posting to Vidmoat Social, and none will. - **Browser recorder, inpainting and Flows are deliberately not in v1.** Each would let one API caller starve everyone else on the box. --- ## 2. The mental model This is the one section worth reading twice, because everything else follows from it. **A project is a JSON document.** ```jsonc { "version": 2, "settings": { "name": "…", "aspectRatio": "9:16", "width": 1080, "height": 1920, "fps": 30, "background": "#000000" }, "tracks": [ { "id": 0, "label": "V1", "type": "video", … }, … ], "clips": [ { "id": "…", "type": "video", "start": 0, "duration": 6, "trackIndex": 0, "src": "…", … }, … ], "markers": [ … ] } ``` Nothing is derived and stored. The timeline's duration is `max(clip.start + clip.duration)`, computed on read. Compositing order is the position of a clip's track in the `tracks` **array**: a clip whose track is listed later draws on top. That is a real gotcha: `trackIndex` is a track *id*, not a layer number, and the two only coincide in the default document. **Every edit is a command through one reducer.** ``` { "op": "addClip", "type": "video", "src": "…", "start": 0, "duration": 6, "trackIndex": 0 } ``` One reducer is the only thing in the system that mutates a document. It generates ids, allowlists patch fields, normalises clips, resolves collisions, and applies the auto-fixes that model output actually needs. **The same reducer serves the editor, the agent and the API.** The editor's `window.vidmoat.dispatch` calls it. The chat agent calls it. The MCP server's `edit_project` calls it. `POST /v1/projects/{id}/commands` calls it directly, as a library import, not over HTTP. `PATCH /v1/projects/{id}` calls it, even for a rename, because `Project.name` on the database row and `document.settings.name` are two copies of one fact and `setProjectSettings` is the only thing that keeps them equal. **Why this matters to you.** An API edit and a hand edit cannot diverge. If your user opens the same project in Vidmoat's own editor, they see exactly what your API calls produced, with the same ids, the same normalisation and the same semantics. There is no "API version of the document" and no import/export step where fidelity leaks. That is the guarantee you are buying, and it is the reason a third party can build a real editor on this at all: you are not driving a sanitised subset, you are driving the actual thing. Two consequences to design around: 1. **Read the document you are about to mutate.** `GET /v1/projects/{id}?view=document` returns it verbatim: the same JSON the reducer consumes. Nothing is reshaped, so your read and your write agree about field names. 2. **The reducer may not do exactly what you asked.** An overlapping clip is pushed to the next track index, and the track is created if it is missing. `addClip` defaults `mediaDuration` to `duration` if you omit it, which quietly breaks waveform windowing and looping. Always read the returned `results[].data.clipId` rather than assuming, and pass `mediaDuration` explicitly when you pass `mediaStart`. --- ## 3. Hello, render ### 3.1 Get a key At `https://developer.vidmoat.com/apps`: create an app, accept the developer agreement, mint a key with the scopes you need. - Key format: `vmk_live_<48 hex>` or `vmk_test_<48 hex>`. Older unprefixed `vmk_` keys still work. - **Live keys need three things**: the developer agreement accepted, the app approved in review, and a plan with `api_access` (Studio, Team or Enterprise). **Test keys are free on every plan**, including free. Whether a key is test or live is fixed by how it was minted. Max 10 active keys per app. - Scopes are chosen at mint time and clamped to the app's approved ceiling, and clamped again on every request, so narrowing an app's ceiling immediately narrows every key it already issued. - Keys cannot mint keys. `/api/developer/**` is session-authenticated only. Everything is `Authorization: Bearer `. ### 3.2 Who am I ```bash export VM=https://api.vidmoat.com/v1 export KEY=vmk_live_… curl -s $VM/me -H "Authorization: Bearer $KEY" ``` ```jsonc { "account": { "id": "…", "email": "…", "name": "…", "plan": { "id": "studio", "name": "Studio", "features": ["ai_console", "no_watermark", …] }, "credits": { "remaining": 5840, "perMonth": 6000, "resetsAt": "2026-09-01T…" }, "exports": { "used": 12, "limit": null, "resetsAt": "…" }, "limits": { "maxProjects": null, "maxExportHeight": 2160, "maxUploadMb": 8192, "maxConcurrentRenders": 5 } }, "credential": { "type": "api_key", "keyId": "…", "appId": "…", "environment": "live", "scopes": ["account.read", "projects.read", "projects.write", "render.read", "render.write"] } } ``` `credential.scopes` is the answer to every 403 you are about to get. Read it first. ### 3.3 Create a project, already edited `POST /v1/projects` accepts a `commands` array, so a whole video can be one call. Returns `201`. ```bash PROJECT=$(curl -s -X POST $VM/projects \ -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \ -d '{ "name": "Hello render", "commands": [ { "op": "setProjectSettings", "aspectRatio": "9:16", "fps": 30, "background": "#0b0b0f" }, { "op": "addTextClip", "text": "Hello, render", "start": 0.2, "duration": 3, "style": { "fontSize": 96, "fontWeight": 800, "color": "#ffffff" } }, { "op": "addShapeClip", "kind": "capsule", "start": 0, "duration": 3.5, "trackIndex": 0, "style": { "fill": "#FF4D2E", "width": 900, "height": 260 } } ] }' | jq -r '.project.id') ``` Response: ```jsonc { "project": { "id": "clx…", "name": "Hello render", "createdAt": "…", "updatedAt": "…", "durationSec": 3.5, "clipCount": 2, "settings": { "aspectRatio": "9:16", "width": 1080, "height": 1920, "fps": 30, "background": "#0b0b0f" } }, "results": [ { "op": "setProjectSettings", "ok": true }, { "op": "addTextClip", "ok": true, "data": { "clipId": "…" } }, … ], "blocked": [], "lint": [ { "severity": "note", "time": 0.2, "message": "…" } ], "suggestedPreviewTimes": [0.2, 1.75] } ``` If a command in the program fails, the project is **not** rolled back. Each command succeeds or fails on its own, and the ones that succeeded are kept. You get the project plus a `results` array telling you which command was wrong: a debuggable state, rather than burning a project-quota slot on every retry. ### 3.4 Apply more commands ```bash curl -s -X POST $VM/projects/$PROJECT/commands \ -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \ -d '{ "commands": [ { "op": "applyMotionPreset", "clipId": "CLIP_ID", "preset": "bounce-in" }, { "op": "setTransition", "clipId": "CLIP_ID", "type": "dissolve", "duration": 0.4 } ] }' ``` ```jsonc { "projectId": "clx…", "ok": true, "dryRun": false, "results": [ { "op": "applyMotionPreset", "ok": true }, … ], "blocked": [], "lint": [ { "severity": "warn", "time": 1.2, "message": "Text \"Hello, render\" runs off the right at ~1.2s. Reduce fontSize/scale or move it inward." } ], "suggestedPreviewTimes": [1.2, 0.2], "timeline": { "clipCount": 3, "durationSec": 3.5 }, "hint": "1 layout warning(s): see lint[] …" } ``` Notes that will save you an afternoon: - **1 to 200 commands per request.** Split larger programs; they apply in order. - **There is no all-or-nothing rollback.** Each command succeeds or fails on its own in `results[]` (`ok: false` plus an `error`), and the batch is saved if at least one command succeeded. The top-level `ok` is true only when every command succeeded. - **`blocked` is not an error.** Plan-gated ops (Pro effects/filters/ transitions, `runScript`) are *dropped* from the batch and named in `blocked`; everything else applies. A Creator-plan caller with one Studio-only effect in a fifty-command program gets forty-nine applied. - **`lint` comes back on every call and you should read it.** Overlapping text, off-canvas elements, font sizes unreadable at output resolution. `severity` is `"warn"` or `"note"`, capped at 10 entries. Numeric x/y/fontSize choices routinely look wrong on the real canvas, and this is the only cheap way to find out. - **`dryRun: true`** runs the whole reducer and returns results + lint without persisting: nothing is saved. ### 3.5 Render and poll ```bash RENDER=$(curl -s -X POST $VM/renders \ -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \ -d "{\"projectId\":\"$PROJECT\",\"format\":\"mp4\",\"quality\":\"standard\"}" \ | jq -r '.render.id') ``` `202`, and note what is *not* in that request body: the document. You name a project you own and the server reads it. The internal `/api/render` takes the whole timeline inline; v1 deliberately does not, because that would weld the clip format into a public contract. ```jsonc { "render": { "id": "clx…", "projectId": "clx…", "status": "PENDING", "progress": 0, "format": "mp4", "resolution": "full", "quality": "standard", "url": null, "error": null, "createdAt": "…", "updatedAt": "…" }, "applied": { "resolution": "full", "watermark": false, "priority": true }, "poll": "/api/v1/renders/clx…" } ``` `applied` tells you the two things that would otherwise be silent surprises: a plan-capped resolution (`maxHeight` + `cappedByPlan: true`) and a baked-in watermark (free plans lack `no_watermark`). ```bash while :; do sleep 4 S=$(curl -s $VM/renders/$RENDER -H "Authorization: Bearer $KEY") echo "$S" | jq -r '.render | "\(.status) \(.progress)%"' case $(echo "$S" | jq -r '.render.status') in COMPLETED) echo "$S" | jq -r '.render.url'; break ;; FAILED|CANCELLED) echo "$S" | jq -r '.render.error'; exit 1 ;; esac done ``` **Status strings are uppercase**: `PENDING`, `PROCESSING`, `COMPLETED`, `FAILED`, `CANCELLED`. The failure reason is on `render.error` (not `errorMsg`, which is the database column name; the serializer renames it). Request-shape validation happens for accepted bodies: `format ∈ {mp4, webm, gif}`, `quality ∈ {draft, standard, high}`, `resolution` is `"full"` or anything else (which means half). A project with no clips is a `400`. --- ## 4. Building an editor UI This is the section that decides whether you have a product or a script. The answer is yes, you can build a real editor UI on this, and here is exactly how. ### 4.0 The architecture constraint, first Your API key is a server credential. It cannot go in a browser, and `api.vidmoat.com`'s CORS configuration is not written for arbitrary third-party origins. So: ``` browser (your UI) ──► your server ──► api.vidmoat.com/v1 │ │ └──────── media bytes, directly ─────────┘ /uploads/* /exports/* ACAO: * ``` The **control plane** goes through your server. The **media plane** does not: `/uploads/`, `/exports/` and `/fixtures/` on `api.vidmoat.com` are all served with `Access-Control-Allow-Origin: *`. That single header is what makes a genuine third-party editor possible, because it means your browser can `fetch()` the raw media, decode it, and read pixels and samples out of it without tainting a canvas or proxying gigabytes through your own box. Give yourself a thin proxy and never think about it again: ```ts // server: app/api/vm/[...path]/route.ts (or the equivalent in your stack) export async function POST(req: Request, { params }: { params: Promise<{ path: string[] }> }) { const { path } = await params; const res = await fetch(`https://api.vidmoat.com/v1/${path.join('/')}`, { method: 'POST', headers: { Authorization: `Bearer ${process.env.VIDMOAT_KEY!}`, 'Content-Type': 'application/json' }, body: await req.text(), }); // Pass the body and status through untouched: the error envelope's `code` is // what your client branches on, and flattening it to "request failed" throws // away the only stable machine-readable part. return new Response(await res.text(), { status: res.status, headers: { 'Content-Type': 'application/json' }, }); } ``` ### 4.1 Read the document, draw a timeline ```ts type Clip = { id: string; type: 'video' | 'audio' | 'text' | 'image' | 'shape' | 'sticker' | 'html' | 'adjustment'; name: string; src: string; start: number; // timeline seconds duration: number; // timeline seconds trackIndex: number; // a TRACK ID, not a layer index mediaStart: number; // in-point in the source media mediaDuration: number; // length of the source media speed: number; reverse: boolean; freezeAt: number | null; loop: boolean; x: number; y: number; scale: number; rotation: number; opacity: number; textStyle?: { content: string; fontSize: number; /* … */ }; isCaption?: boolean; kfX: { time: number; value: number; easing?: string }[]; // … one array per keyframable prop volume: number; muted: boolean; fadeIn: number; fadeOut: number; }; type TrackDef = { id: number; label: string; type: 'video' | 'audio' | 'text'; color: string; bg: string; muted: boolean; locked: boolean; visible: boolean }; type EditorDocument = { version: 2; settings: { name: string; aspectRatio: string; width: number; height: number; fps: number; background: string }; tracks: TrackDef[]; clips: Clip[]; markers?: { id: string; time: number; label?: string; color?: string }[]; }; const { project } = await vm(`/projects/${id}?view=document`); const doc: EditorDocument = project.document; ``` Three views exist and the difference is size, not taste: | `view` | Returns | Use for | |---|---|---| | `summary` | id, name, timestamps, `durationSec`, `clipCount`, `settings` | project lists | | `clips` (default on a single GET) | + `tracks`, + per-clip summaries (id, type, start, duration, trackIndex, src, first 120 chars of text, effect types, `keyframedProps`) | a timeline overview | | `document` | + the whole document verbatim | anything you intend to edit | `view=document` is refused on the *list* endpoint (`GET /v1/projects`): a page of 20 full documents is megabytes. You get `view=clips` and a `note` saying so. Drawing the timeline is then arithmetic. Pixels-per-second is your zoom; the Vidmoat editor uses 10 to 300 px/s with a 46 px track height: ```tsx const PX_PER_SEC = 60; function Timeline({ doc }: { doc: EditorDocument }) { const total = Math.max(...doc.clips.map(c => c.start + c.duration), 1); return (
{/* tracks in ARRAY order: index 0 is the bottom layer */} {doc.tracks.map((track, layer) => (
{doc.clips .filter(c => c.trackIndex === track.id) .map(c => (
{c.type === 'text' ? c.textStyle?.content : c.name} {(c.type === 'audio' || c.type === 'video') && ( )}
))}
))}
); } ``` ### 4.2 Waveforms, client-side This is the part people assume needs a server. It does not, and Vidmoat's own editor does not use one either: it decodes the media in the browser with `AudioContext.decodeAudioData` and reduces it to peaks. Because `/uploads/` sends `Access-Control-Allow-Origin: *`, **your** origin can do the identical thing. ```ts const peakCache = new Map(); /** ~300 MB ceiling: decodeAudioData needs the ENTIRE file in memory. Past that, * the tab OOMs, so fail fast and fall back to a flat waveform instead. */ const MAX_DECODE_BYTES = 300 * 1024 * 1024; export async function computePeaks(url: string, count = 400): Promise { const hit = peakCache.get(url); if (hit) return hit; try { const res = await fetch(url); // cross-origin, allowed by ACAO: * if (!res.ok) throw new Error(`fetch ${res.status}`); if (Number(res.headers.get('content-length') || 0) > MAX_DECODE_BYTES) { res.body?.cancel(); throw new Error('too large to decode'); } const buf = await res.arrayBuffer(); const ctx = new AudioContext(); const audio = await ctx.decodeAudioData(buf.slice(0)); // slice(0): decodeAudioData detaches the buffer const chan = audio.getChannelData(0); const block = Math.floor(chan.length / count) || 1; const peaks: number[] = []; let max = 0.0001; for (let i = 0; i < count; i++) { let peak = 0; const start = i * block; for (let j = 0; j < block; j += 8) { // stride: 8x faster, visually identical const v = Math.abs(chan[start + j] || 0); if (v > peak) peak = v; } peaks.push(peak); if (peak > max) max = peak; } const norm = peaks.map(p => p / max); peakCache.set(url, norm); return norm; } catch { // A non-audio file or a decode failure is normal, not exceptional. Draw // something and keep the editor usable. const flat = Array.from({ length: count }, (_, i) => 0.2 + 0.12 * Math.abs(Math.sin(i * 0.4))); peakCache.set(url, flat); return flat; } } ``` A trimmed clip must show *its* region of the file, not the whole file's. The window is a pair of fractions derived from `mediaStart`, `duration` and `speed`: ```ts function mediaWindow(clip: Clip): { from: number; to: number } | undefined { const len = clip.mediaDuration; if (!len || len <= 0) return undefined; // this is why you always pass mediaDuration const from = Math.max(0, Math.min(1, clip.mediaStart / len)); const to = Math.max(from, Math.min(1, (clip.mediaStart + clip.duration * clip.speed) / len)); return { from, to }; } function Waveform({ src, clip, width, color, height = 26 }: { src: string; clip: Clip; width: number; color: string; height?: number; }) { const [peaks, setPeaks] = useState(null); useEffect(() => { let alive = true; if (src) computePeaks(src, 400).then(p => { if (alive) setPeaks(p); }).catch(() => {}); return () => { alive = false; }; }, [src]); if (!peaks) return null; const win = mediaWindow(clip); const view = win ? peaks.slice( Math.floor(win.from * peaks.length), Math.max(Math.floor(win.from * peaks.length) + 1, Math.ceil(win.to * peaks.length)), ) : peaks; const bars = Math.max(12, Math.min(400, Math.floor(width / 2))); const step = view.length / bars; const mid = height / 2; return ( {Array.from({ length: bars }, (_, i) => { const h = Math.max(1.2, (view[Math.floor(i * step)] ?? 0) * (height - 2)); return ; })} ); } ``` The same technique gives you the other analyses Vidmoat runs client-side, off the same decoded buffer: silence detection (RMS per 20 ms window, merge, drop regions under 0.4 s) and energy-flux beat detection. Both are a few dozen lines over the same peaks array. The point here is that **none of it costs you a server round-trip or a credit**, and it is the input to the `cutRanges`, `trimSilence`, `addMarkers` and `sliceClip` commands. ### 4.3 The composition preview `GET /v1/projects/{id}/preview?format=html` returns the whole composition: a self-contained HTML document with every clip as a positioned element, driven by **one paused GSAP timeline** registered at `window.__timelines.main`. This is the exact HTML the render worker drives, not an approximation of it. ``` GET /v1/projects/{id}/preview?format=html → text/html, Cache-Control: private, no-store X-Vidmoat-Media-Count: 7 ``` Two things stop you from putting that URL straight in an `