Capability

Video to SFX

Generate sound effects that land on what happens in a video — footsteps, impacts, ambience, whooshes — as one full SFX bed or one clip per detected sound.

Versions: v1 and v2
Modes
video_to_sfx — native payload
standard — a full SFX bed
plan → magic — one clip per sound
Response
Pricing

Native only — new capability

No compat route exists for Video to SFX. Call POST /v1/generations with the tool_id for model="v1", operation="video_to_sfx" (see Tool Catalog).

Versions: v1 and v2

v1 (model="v1") — standard, plan and magic. keep_dialogue returns the effects only, placed around the clip’s speech.
v2 (model="v2") — the newer sound model, 48 kHz. standard only: its payload has no mode and no plan_token. keep_dialogue returns one mix of the new effects with the clip’s dialogue (its music dropped), to play instead of the clip’s own audio, at no extra cost. prompt (up to 2000 characters) and negative_prompt (up to 1000) are guidance: the model reads the video itself. Each file’s metadata carries includes_dialogue.

Each version has its own tool_id; the payload below is v1’s, and v2 takes the same fields without mode and plan_token.

Modes

standard — 1–4 takes of one continuous SFX bed for the whole window, each as long as the window.
plan — detects the separate sounds in the window and quotes what generating them costs. Free; generates nothing.
magic — one clip per detected sound, each timed to where it happens in the video. Call plan first, then magic with its plan_token.

Every time in the output is in milliseconds into the source video, so a clip can be placed on your timeline at its start_ms without any offset math.

video_to_sfx — native payload

POST/v1/generationstool_id for (model="v1", "video_to_sfx")

Field	Type	Required	Default	Description
video	file input	Yes	—	The video to score, up to 1 GiB, as a URL, inline base64 or a Library file ref (see Providing audio/file input). A URL is the practical choice for video.
mode	string	No	"standard"	"standard", "plan" or "magic".
start_seconds	float	No	0	Where the scored window starts, in seconds into the video.
duration_seconds	float	No	to the end	How much to score. Up to 300 s per call; score a longer video in several windows.
takes	int	No	1	1–4. Standard: takes of the bed. Plan/magic: takes per detected sound.
prompt	string	No	—	Standard only. Optional steer, up to 5000 characters.
negative_prompt	string	No	—	Standard only. Sounds to avoid (may be ignored).
seed	int	No	—	Standard only. 0–4294967295.
keep_dialogue	bool	No	false	Standard only. Keeps the clip's own dialogue and places effects around it. Turned off, with a note, when the video has no audio.
plan_token	string	No	—	Magic only. The plan_token a plan call on the same video returned.
budget_tokens	int	No	0	The most you agree to pay, in cents. v1 standard: ceil(seconds × takes × 3), plus seconds × 2.4 with keep_dialogue. v2: ceil(seconds × takes × 4.5). Magic: the plan's quote_tokens. Plan: 0.

budget_tokens is a ceiling, and it is enforced

The balance hold is exactly budget_tokens. A budget below the price is refused before anything is generated, and the error message names the exact price to send. You are charged only for what was delivered — a take or sound that doesn’t come back isn’t charged — and never more than budget_tokens.

standard — a full SFX bed

Two takes of a 20 s window: budget_tokens = 20 × 2 × 3 = 120 ($1.20).

curl -X POST https://apiv2.soundverse.ai/v1/generations \
  -H "Authorization: Bearer sksoundverse_..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: vsfx-req-001" \
  -d '{
    "tool_id": "<video_to_sfx tool_id>",
    "payload_json": "{\"video\": \"https://example.com/clip.mp4\", \"mode\": \"standard\", \"start_seconds\": 5, \"duration_seconds\": 20, \"takes\": 2, \"budget_tokens\": 120}"
  }'

plan → magic — one clip per sound

Call with mode: "plan" and budget_tokens: 0. The completed output lists the detected sounds and returns quote_tokens (the magic price, in cents), a signed plan_token and plan_expires_at.
Before plan_expires_at, call with mode: "magic", the same video, window and takes, that plan_token, and budget_tokens = quote_tokens. Magic generates exactly the sounds the plan listed.

import json, time, requests

API = "https://apiv2.soundverse.ai/v1"
HEADERS = {"Authorization": "Bearer sksoundverse_..."}
TOOL_ID = "<video_to_sfx tool_id>"
video = {"video": "https://example.com/clip.mp4", "start_seconds": 0, "duration_seconds": 30}

def run(payload, key):
    task = requests.post(f"{API}/generations", headers={**HEADERS, "Idempotency-Key": key},
                         json={"tool_id": TOOL_ID, "payload_json": json.dumps(payload)}).json()
    while True:
        t = requests.get(f"{API}/generations/{task['task_id']}", headers=HEADERS).json()
        if t["status"] in ("completed", "failed"):
            return t
        time.sleep(3)

plan = run({**video, "mode": "plan", "budget_tokens": 0}, "vsfx-plan-001")["output"]
print(len(plan["sounds"]), "sounds, magic costs", plan["quote_tokens"], "cents")

magic = run({**video, "mode": "magic", "plan_token": plan["plan_token"],
             "budget_tokens": plan["quote_tokens"]}, "vsfx-magic-001")["output"]

A plan_token is bound to the API key’s account and to the API: it can’t be spent by another account or in the Soundverse app. Past plan_expires_at, run plan again.

Response

Once the task completes, its output has this shape (magic shown):

{
  "text": "Generated 2 sounds (2 files).",
  "mode": "magic",
  "model": "v1",
  "window_start_ms": 0,
  "window_duration_ms": 30000,
  "source_duration_ms": 42000,
  "tokens_charged": 27,
  "notes": [],
  "takes": [],
  "sounds": [
    {"sound_index": 0, "label": "door slam", "type": "sfx", "category": "foreground",
     "start_ms": 2000, "end_ms": 5000, "sound_start_ms": 2500, "sound_end_ms": 3100,
     "gain_db": -3.0, "credits": 30, "asset_indices": [0], "file_ids": ["018f..."]},
    {"sound_index": 1, "label": "room tone", "type": "sfx", "category": "background",
     "start_ms": 0, "end_ms": 6000, "sound_start_ms": 0, "sound_end_ms": 6000,
     "gain_db": -12.0, "credits": 60, "asset_indices": [1], "file_ids": ["018f..."]}
  ]
}

Each file is FLAC and is also in output.assets. Place a sound’s file at its start_ms; sound_start_ms/sound_end_ms mark the sound itself inside it, and gain_db is the suggested mix level.
standard fills takes instead: one entry per delivered take with its file_id, duration_ms and gaps ([start_ms, end_ms] ranges where that take is silent).
plan fills sounds without files, plus plan_token, quote_tokens and plan_expires_at.
notes explains ignored fields and automatic adjustments, one sentence each.

Pricing

Model	Mode	Price	Counted on
`v1`	`standard`	$0.03 / s / take	Seconds of video scored × takes.
`v1`	`keep_dialogue`	+$0.024 / s	Standard only. Once per call, however many takes.
`v1`	`magic`	$0.03 / s / take	Seconds of each detected sound × takes. plan returns the exact quote.
`v1`	`plan`	Free	Detects the sounds and quotes magic. Generates nothing.
`v2`	`standard`	$0.045 / s / take	Seconds of video scored × takes. keep_dialogue is included.

USD, the same on every license tier, rounded up to the whole cent per call. Rate limit: 20 calls an hour, 100 a day.

← Sound EffectsAudio Effect →