Composite blocks for realtime motion capture camera and fusion applications.
Load the provider extensions you use first, then load the version-pinned bundle as an unsandboxed custom extension:
https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-realtime-motion-capture@0.7.0/dist/turbowarp-realtime-motion-capture.js
All feature groups are OFF by default. Set the complete startup configuration before loading the bundle:
globalThis.__TWMP_FEATURE_FLAGS__ = {
qrCourierPairing: true,
webgpuMoveNetMultiPose: true,
protocolV1Codec: true,
cameraCalibrationV1: true,
avatarRetargetV1: true,
frameSyncPatternV1: true,
poseFusion3D: true,
glowStickMarkers: true,
};
globalThis.__TWMP_QR_CONFIG__ = {errorCorrectionLevel: "M"};Omit features you do not use. Reload the project and extension after changing this startup-fixed object.
prepare offer QR for peer [PEER]Creates a named WebRTC offer and prepares one or more QR courier parts.
show offer QR part [INDEX] on this spriteDisplays a one-based part as a temporary, lossless SVG sprite skin.
end offer QR displayRestores original skins and discards sensitive temporary session data.
The extension exposes blocks across eight independently enabled feature groups. See the generated block reference for every opcode, argument, type, and default.
The startup-fixed qrCourierPairing flag is OFF by default. The extension requires the version 2 TurboWarp WebRTC runtime capability and must run unsandboxed.
Pairing QR codes can expose local addressing and ICE credentials. Display them only inside a trusted venue and remove courier photographs after pairing.
The independent webgpuMoveNetMultiPose flag is OFF by default. When enabled, the extension leases frames from TurboWarp Camera Source, explicitly requires the TensorFlow.js webgpu backend, and returns up to six tracked COCO-17 poses as PoseFrame2D version 1 JSON.
No CPU, WASM, or WebGL inference fallback is provided. The first model load generally requires network access; prepare the browser cache before an offline venue run.
The independent protocolV1Codec flag is OFF by default. When enabled, blocks validate and round-trip SessionPolicy, CameraCalibration, PoseFrame2D (v1 and v2), PoseFrame3D, and Performance DSL JSON. This extension owns those contracts and publishes their generated JSON Schemas; applications depend on the extension, never the reverse.
Unknown schemas, versions, fields, malformed COCO-17 tuples, and pairing credential keys fail closed. Diagnostic reporters expose the JSON Pointer path and message.
Pose timestamps are opaque values supplied by a separate synchronized local time service. This extension carries them unchanged and implements no clock probes, offset estimation, ping, or pong logic.
The independent cameraCalibrationV1 flag is OFF by default. A high-level state machine leases the named Camera Source stream, fixes its real resolution, accepts diverse chessboard samples, and exports an exact CameraCalibration v1 profile.
The bundled, exact-pinned OpenCV.js 4.12 WebAssembly backend is the sole solver. It estimates intrinsic, distortion, and world-from-camera values and rejects insufficient samples, resolution changes, weak views, excessive reprojection RMS, unknown profile versions, and pairing credentials.
The independent avatarRetargetV1 flag is OFF by default. The adapter requires TurboWarp-A-Frame 0.5.0 scene capability v2, refuses any other version, and uses only its public template, transform, event, count, delete, and VRM operations.
Corresponding external PoseFrame3D and PoseFrame2D v1 values can drive up to six VRM avatars. The adapter matches person IDs, expands COCO-17 to BlazePose-33, and uses exact-pinned Kalidokit 1.1.5 as its only rotation solver before turning the VRM humanoid bones on the performer's own side. A block sets VRM expressions on the avatar bound to a person, so the application can show Performance DSL effects on it.
The deterministic expansion uses reduced-visibility duplicates for unavailable hand, foot, and face landmarks and is less precise than native BlazePose input. Kalidokit is deprecated upstream, so retargeting requires validation with real recordings. This extension provides no custom solver fallback and does not align frames, query history, triangulate, or solve 3D data.
The independent poseFusion3D flag is OFF by default. When enabled, blocks buffer PoseFrame2D JSON per camera in timestamp-ordered ring buffers, resample every camera at one past instant behind a configured delay, and triangulate the synchronized 2D sets into PoseFrame3D version 1 JSON.
Out-of-order frames inside the jitter window are reordered, while duplicates, late arrivals, and frames that no loaded calibration profile describes are counted as dropped. Occluded keypoints are filled from the visible side of their bracket, disagreeing views are resolved by the largest consensus set, and a keypoint without two confident views holds its last triangulated position with score zero.
The independent glowStickMarkers flag is OFF by default. When enabled, the camera app samples the keypoints you name for a uniquely colored glow stick on the same video frame as the pose and reports PoseFrame2D version 2.
The fusion app maps those colors to performers with the Performance DSL palette. An identified person keeps the performer's identifier across occlusion and re-entry, views of different performers never merge, and a color seen on the mirrored keypoint tells the fusion that the camera read a back view as a front view, so that view's left and right labels are swapped before triangulation.
The public API is the TurboWarp extension and its generated manifest. Source-level controllers are internal and are not npm exports. See the repository README and architecture documentation for build, test, release, and compatibility details.