API Reference

DUSTrack: Semi-automated point tracking in ultrasound videos.

There are three main components in the DUSTrack workflow, namely the DUSTrack GUI for initial manual annotation, the module for deep learning based tracking using DLC project management, and the annotation class with LK-RSTC post-processing.

open()

Unified entry point. Pass a video to start fresh, or a DLC project (or a video inside one, or its config.yaml) to resume in place.

DUSTrack

Interactive GUI for manual annotation and refinement of tracking points.

DLCProject

Interface for training and managing DeepLabCut pose estimation models. Requires the deeplabcut package to be installed.

VideoAnnotation

Annotation-data container with DeepLabCut HDF5 interop. Use directly for programmatic loads – dustrack.VideoAnnotation(json_path, video).to_signals().

lk_moving_average_filter()

Lucas-Kanade optical flow algorithm with Reverse Sigmoid Tracking Correction for reducing frame-to-frame jitter in tracked points. In practice, access this via dustrack.VideoAnnotation.postprocess() method.

Typical workflow:

1. Create annotations using DUSTrack GUI

2. Create DLC project from annotations

3. Train and refine models iteratively

4. Apply optical flow post-processing to reduce jitter

Example

>>> import dustrack
>>>
>>> # Step 1: start fresh on a video
>>> tracker = dustrack.open('video.mp4', 'manual')
>>> # Use the GUI to annotate points and save (writes video_annotations_manual.json),
>>> # then click "Create DLC Project" + "Train DLC model" from the sidebar.
>>>
>>> # If you close the UI mid-workflow, resume with the same call:
>>> tracker = dustrack.open('video.mp4')             # auto-detects the project
>>> tracker = dustrack.open('path/to/project/')      # equivalent
>>> tracker = dustrack.open('path/to/config.yaml')   # equivalent
>>>
>>> # DUSTrack also works without deeplabcut installed -- the LK-RSTC
>>> # post-processing alone gets you usable tracks:
>>> tracker = dustrack.open('video.mp4', 'manual')
>>> tracker.process_with_lk()
>>>
>>> # Programmatic load (no UI):
>>> va = dustrack.VideoAnnotation('video_annotations_pn.json', 'video.mp4')
>>> signals = va.to_signals()

class dustrack.Blip(label: str, start: int, end: int, anchor_before: tuple[float, float], anchor_after: tuple[float, float], threshold: float)

Bases: object

A single detected outlier run for one label.

Variables:

• label (str) – The label key (string digit, matching VideoAnnotation.labels).

• start (int) – First blip frame, inclusive.

• end (int) – Last blip frame, inclusive. start == end for a single-frame blip.

• anchor_before (tuple[float, float]) – (x, y) at frame start - 1 (the last-known-good).

• anchor_after (tuple[float, float]) – (x, y) at frame end + 1 (the first-known-good after the run).

• threshold (float) – The per-label displacement threshold that flagged the entry.

label: str

start: int

end: int

anchor_before: tuple[float, float]

anchor_after: tuple[float, float]

threshold: float

property length: int

Number of blip frames (inclusive on both ends).

__init__(label: str, start: int, end: int, anchor_before: tuple[float, float], anchor_after: tuple[float, float], threshold: float) → None

class dustrack.BlipReport(blips: list[dustrack.blip.Blip] = <factory>, per_label_stats: dict[str, dict[str, float | int]] = <factory>, params: dict[str, typing.Any] = <factory>, n_frames: int = 0)

Bases: object

Detection result for one VideoAnnotation.

Variables:

• blips (list[dustrack.blip.Blip]) – All detected blips across all labels, in label-then-frame order.

• per_label_stats (dict[str, dict[str, float | int]]) – Per-label diagnostic dict. Keys: median_d, mad_d, threshold, n_blips, n_skipped_edge (spikes at frame 0 / last frame that can’t be anchor-bracketed), n_skipped_long (runs longer than max_blip_length), n_skipped_noreturn (spikes whose forward scan never finds a return-position match).

• params (dict[str, Any]) – The detection knobs used (echoed for traceability).

blips: list[dustrack.blip.Blip]

per_label_stats: dict[str, dict[str, float | int]]

params: dict[str, Any]

n_frames: int = 0

by_label() → Mapping[str, list[dustrack.blip.Blip]]

Group blips by label.

length_histogram() → dict[int, int]

Count blips by run length.

min_coverage() → float

Smallest per-label fraction of finite frames in the source annotation.

min_coverage() == 1.0 means every label is densely populated across all frames (typical DLC predicted trace); lower values mean at least one label has gaps. Used by the UI gate to decide whether the active layer is dense enough for blip detection to be meaningful (sparse manual layers can be scanned, but the per-label MAD threshold isn’t well-conditioned on a handful of points).

Returns 0.0 when there are no labels, no per-label stats recorded, or n_frames == 0.

__init__(blips: list[dustrack.blip.Blip] = <factory>, per_label_stats: dict[str, dict[str, float | int]] = <factory>, params: dict[str, typing.Any] = <factory>, n_frames: int = 0) → None

class dustrack.DUSTrack(vid_name, annotation_names='iteration-0', *args, clahe_clip=1.0, clahe_grid=8, gamma=1.0, brightness=0, dark_mode=False, n_labels: int = 1, titlefunc: Callable | None = None, height_ratios: tuple = (10, 1, 1), fast_render: bool = True, **kwargs)

Bases: VideoBrowser

Interactive video point annotator with DeepLabCut integration.

DUSTrack provides a GUI for manual annotation of points in videos, with integrated support for creating DeepLabCut projects, training models, and post-processing results using optical flow algorithms.

Features:

• Manual point annotation with event marking

• Real-time trajectory visualization

• DeepLabCut project creation and training

• Optical flow-based jitter reduction

• Multiple annotation layer management

Variables:

• _dlcproject (DLCProject) – Associated DeepLabCut project instance.

• _ax_lims (dict) – Stores axis limits when plot axes are frozen.

Example

>>> # Most users should call :func:`dustrack.open` instead of
>>> # constructing DUSTrack directly -- it dispatches to either
>>> # this class (bare video) or :meth:`DLCProject.annotate`
>>> # (resume in a DLC project) based on the path you pass.
>>> # Direct construction below is for advanced / scripted use.
>>>
>>> # Basic usage -- annotation_names defaults to "iteration-0",
>>> # the canonical seed name for the DLC pipeline (the next
>>> # DLC training round lands as iteration-1).
>>> tracker = DUSTrack('video.mp4')
>>>
>>> # Explicit layer name -- saved as {video_name}_annotations_pn.json
>>> tracker = DUSTrack('video.mp4', "pn")
>>>
>>> # With multiple annotation layers
>>> tracker = DUSTrack('video.mp4', {
...     'manual': 'manual_labels.json',
...     'dlc_iter1': 'dlc_predictions.h5'
... })

CORRECTIONS_LAYER_NAME = 'dlccorr'

PREDICT_LAYER_NAME = 'dlc_predict'

__init__(vid_name, annotation_names='iteration-0', *args, clahe_clip=1.0, clahe_grid=8, gamma=1.0, brightness=0, dark_mode=False, n_labels: int = 1, titlefunc: Callable | None = None, height_ratios: tuple = (10, 1, 1), fast_render: bool = True, **kwargs)

Parameters:

• vid_name (str) – Path to the video file.

• titlefunc (Optional[Callable]) – Function to generate the title for the plot.

• figure_or_ax_handle (Optional[Union[plt.Axes, plt.Figure]]) – Handle to the figure or axis.

• image_process_func (Callable) – Function to process the image.

• fast_render (bool) – If True, render the video frame and any annotation scatter overlays Qt-native (QGraphicsView + QPixmapItem) instead of through matplotlib’s imshow. Bypasses the canvas pixmap-upload cost that dominates per-frame budget on QtAgg (see BENCHMARKING.md probe 13 / 14 for the architecture rationale). Requires a Qt matplotlib backend; silently falls back to Tier 1 if no Qt window is found. In fast_render mode, user additions to _ax_image via mpl plotting calls are NOT supported (the image region is no longer an mpl Axes); _ax and _im are set to None.

discard_unsaved_annotations(event=None)

Drop in-memory edits on the active layer and reload from disk.

Inverse of save: confirms via ConfirmOverlay, then calls VideoAnnotation.reload() on self.ann and triggers self.update() so the trace pane + scatter re-render. The confirm body branches on whether the layer’s backing file exists:

• File exists -> “Reload from disk” semantic.

• File missing -> “Reset to empty” semantic.

Refuses on the dlccorr splice and any layer matching _is_dense_layer_name() – those are regenerated from the active + overlay layers (via apply_manual_corrections or process_with_lk), not authored by hand, so “discard and reload” has no meaningful semantic. Points the user at Remove layer instead.

freeze_plot_axes(event=None)

Lock the axis limits of trajectory plots to current view.

Useful for comparing tracking quality across different frames without automatic axis rescaling distracting from the comparison.

Parameters:

event – Mouse/keyboard event (unused, for button compatibility).

unfreeze_plot_axes(event=None)

Restore automatic axis scaling for trajectory plots.

Parameters:

event – Mouse/keyboard event (unused, for button compatibility).

create_dlc_project(event=None, name=None, path=None, experimenter='x', seed_bundle_path=None, link_videos: bool | None = None) → DLCProject

Create a new DeepLabCut project using current annotations as training labels.

rc2 (1.1.0rc2): on a Qt backend, project creation runs off the GUI thread under a modal overlay (no progress bar – it’s a fast op, but the overlay surfaces DLC’s stdout and a Done button so the user can confirm the project location and any warnings before continuing). On non-Qt backends the call runs synchronously and returns the new DLCProject.

1.2.0a2: if the active manual layer is empty when the user clicks Create DLC Project on the Qt path, a multi-step Seed-from-bundle modal opens (_prompt_seed_bundle()). Picking a valid bundle routes through a seeding flow that: (a) creates the project as usual, (b) calls import_seed_bundle_into_project() to install the bundle’s snapshot as iteration-0’s trained model (overwriting the project’s bodyparts with the bundle’s), and (c) runs analyze_videos(iteration_num=0) to produce a dense reference layer the user can refine into iteration-1. seed_bundle_path may also be passed programmatically to bypass the modal.

Parameters:

• event – Mouse/keyboard event (unused, for button compatibility).

• name (str, optional) – Project name. Defaults to “{video_name}_{annotation_layer}”.

• path (str, optional) – Directory for project. Defaults to video’s parent directory.

• experimenter (str, optional) – Experimenter name. Defaults to config value.

• seed_bundle_path (str, optional) – Path to a seed bundle folder (as produced by extract_snapshot_for_seeding()). When supplied, the seeding flow runs unconditionally (Qt or non-Qt), bypassing the empty-layer-triggered modal.

• link_videos – Forwarded to DLCProject to control how the source video is placed inside <project>/videos/. None (default) hard-links on same-volume sources and falls back to copy on cross-volume. False forces a deep copy (the pre-1.3.0a2 behavior).

Returns:

The newly created project instance on the sync path. None on the Qt async path – read self._dlcproject after the Done button is clicked.

Return type:

DLCProject

Note

Project names must contain an underscore for proper DLC configuration handling.

process_dlc_project(event=None, *args, **kwargs)

Train the DeepLabCut model without leaving the annotation UI.

rc2 (1.1.0rc2): training runs off the GUI thread, the DUSTrack window stays open under a “Training in progress” overlay (progress bar + scrolling stdout tail), and on success the new DLC prediction layers are added to the live session via add_annotation_layers() – no relaunch. The overlay transitions to a “Complete” / “Failed” state with a Done button so the user can review the final stdout before the predictions swap in (or read the error before retrying); failure paths fold into the overlay rather than popping a separate QMessageBox.

1.2.0a2: a Training options modal runs before everything else (Qt path only). It surfaces DLCProject.train_iteration()’s arg surface in the UI: refine_mode (scratch / in-project / external), source iteration/snapshot picker for in-project, Browse… for external .pt, training epochs (DLC3) / iterations (DLC2), and a create-labeled-video toggle. Cancel returns to the UI without kicking off training. On accept, the underlying call routes through DLCProject.train_iteration() (the explicit-args sibling of DLCProject.process()) – so the DLC2 silent-drop bug on refine=<str> is gone, and external snapshots work on both DLC versions (DLC3 via train_network(snapshot_path=...), DLC2 via a pose_cfg init_weights edit).

A unified pre-flight runs before the overlay starts, scanning every manual annotation layer in the session (file-pattern detection – .json alongside the video, <video_stem>_annotations*.json, minus dlccorr / buffer / dlc*; agnostic to which layer is active, the overlay, or a placeholder, and to whether the user renamed iteration-N to something else). For each manual layer we report (a) in-memory diffs vs the on-disk JSON and (b) frames missing one or more bodyparts. If any layer has either kind of issue, a single modal lists the per-layer breakdown and offers two actions: Save and clean (write per-layer recovery sidecars for the dropped frames, drop the incomplete frames from every affected layer, then save every affected layer, then train) and Cancel (return to the UI). Layers without issues are not touched; DLC trace layers and dlccorr are not in scope. Recovery sidecars are written as <fstem>.dustrack-dropped-incomplete-<YYYYMMDDTHHMMSS>; the composite extension avoids .json so the annotation-discovery glob does not re-ingest them on subsequent training runs.

DLC’s stdout/stderr are also teed to the launching terminal (sys.__stdout__) so callers who launched from a shell can watch progress there as before.

On non-Qt backends (or if the QMainWindow can’t be located), the method falls back to the pre-rc2 behavior: close the figure, run training synchronously on the calling thread, and return a fresh DUSTrack via DLCProject.annotate(). The pre-flight modal is skipped on this path (no GUI to host it).

Parameters:

• event – Mouse/keyboard event (unused, for button compatibility).

• *args – Forwarded to DLCProject.process() on the non-Qt fallback path; ignored on the Qt path (the Training options modal owns the kwarg surface there).

• **kwargs – Same – non-Qt fallback only. create_video defaults to False on the non-Qt path (vs. True for direct DLCProject.process() calls) so the annotate -> train -> review -> annotate loop doesn’t write a labeled mp4 each pass; the Qt path’s create-labeled-video checkbox owns this on its own.

Returns:

self on the Qt path (training is asynchronous; the same DUSTrack will refresh in place when the user clicks Done). Also self if the user cancels either the Training options modal or the pre-flight modal – the UI is left intact. On the fallback path, the freshly-launched DUSTrack from DLCProject.annotate().

Return type:

DUSTrack

Raises:

• ImportError – If deeplabcut isn’t installed.

• ValueError – If no DLCProject has been created yet.

process_with_lk(event=None, *args, **kwargs)

Apply Lucas-Kanade optical flow post-processing to reduce tracking jitter.

rc2 (1.1.0rc2): on a Qt backend, the LK-RSTC pass runs off the GUI thread under a progress overlay that mirrors the Train DLC flow (tqdm bars drive the progress widget; phase label reflects “Submitting” vs “Processing”; Done button lets the user confirm before the smoothed layer swaps in). On non-Qt backends the call runs synchronously and returns the new VideoAnnotation.

Uses the Lucas-Kanade RSTC (Reverse Sigmoid Tracking Correction) algorithm to smooth trajectories. The processed annotation is saved and added as a new layer, with the original set as overlay for comparison.

rc2 layer-naming harmonisation: the new layer is adopted via _adopt_layer(), which derives its name from the output filepath via VideoFileManager.canonical_layer_name() – identical to what a close + reopen would show. Replaces the earlier behaviour where the in-session layer briefly carried the "noname" fallback until reload. When the source layer is a DLC trace, the smoothed output is also plot-type-normalised to "line" so it looks identical to other dlc_* layers.

The source layer is save()-ed to disk right before LK kicks off (mirroring the pre-train save in process_dlc_project()), so the on-disk state matches what LK sees. In the typical workflow the source is the dlccorr layer (active after apply_manual_corrections()) and the save persists any in-memory manual edits. Sources without a .json filename (e.g. raw DLC traces loaded from .h5) are read-only inputs; the save is skipped for them with a one-line note.

Parameters:

• event – Mouse/keyboard event (unused, for button compatibility).

• *args – Additional arguments passed to lk_moving_average_filter.

• **kwargs – Additional keyword arguments (e.g., window_size) passed to filter.

Returns:

The smoothed annotation layer on the sync path. None on the Qt async path – the new layer is added to self.annotations when the Done button is clicked.

Return type:

VideoAnnotation

See also

dustrack.lk_filter.lk_moving_average_filter: The filtering algorithm.

detect_blips_workflow(event=None)

Detect blip outliers on the active layer + remove them.

Two-stage flow that mirrors process_with_lk()’s Qt-vs-mpl dispatch:

1. On Qt: pop the BlipOptionsDialog modal (knob tuning + in-modal Detect button + results pane + drop-frame checkbox). Cancel returns; on Remove blips, save a without-blip copy of the source layer via dustrack.blip.remove_blips() and adopt it.

2. On mpl-fallback: run detect + remove synchronously with module defaults; print a one-line summary.

Either way, the without-blip layer adopts via _adopt_layer() with the source layer pinned as overlay (mirrors Reduce jitter’s adoption shape so the user sees the source DLC trace + cleaned version side-by-side immediately).

The LK-interpolation alternative (per-blip RSTC re-track, producing a sparse corrections layer) shipped first but turned out to be less useful in the pia02 workflow than just dropping the contaminating frames – the model trains on a cleaner subset rather than on synthesized positions. The LK function dustrack.blip.interpolate_blips() stays available for headless callers who explicitly want it.

remove_current_layer(event=None)

Remove the active annotation layer from the DUSTrack session.

Session-only: the underlying JSON / HDF5 file on disk is not touched – so on next launch the layer reappears if its file is still next to the video. Pair with a manual file delete (or Save annotation as... to a different name + delete the original) when the intent is “undo manual corrections”.

Refuses with a notice if only one removable layer remains in the session (excluding the implicit "buffer" layer, which is never user-visible but always present). Otherwise confirms via ConfirmOverlay; the confirm body is severity- aware via _is_dense_layer_name():

• Dense / derived (dlc_*, dlccorr, *lkmovavg*): regenerable, default button = Remove.

• Sparse / authored (manual labels): irreversible, default button = Cancel.

copy_existing_annotations_from_overlay(event=None)

Copy overlay annotation points to the current annotation layer for selected frames.

Useful when DLC predictions are more accurate than manual labels. Typically used with manual annotations in the primary annotation layer, and the model predictions in the overlay layer. Data is copied only for frames that exist in the primary annotation layer. Perform this action within a specified frame range by selecting an interval.

Parameters:

event – Mouse/keyboard event (unused, for button compatibility).

Raises:

ValueError – If no annotation overlay is currently selected.

Note

Only affects the current label. Other labels remain unchanged.

MANUAL_CORRECTIONS_SUFFIX = '_manual_corrections'

apply_manual_corrections(event=None)

Splice the active layer’s manual entries into the overlay to produce/refresh the dlccorr layer.

Workflow context (step 4 of the DUSTrack pipeline): after iterating DLC training (steps 2-3), you flip into a manual-correction mode. The active annotation layer holds sparse hand-edits (a few frames where DLC was wrong); the annotation overlay points at the DLC trace you want to correct. Clicking this button produces a layer named CORRECTIONS_LAYER_NAME ("dlccorr") that is the overlay’s data with your manual entries spliced in wherever they exist. The file lives next to the video as <video>_annotations_dlccorr.json and is automatically excluded from DLC training input (the _dlccorr filter in _extract_frames()) – this is a terminal output.

Preflight save. If the active (patch) layer has any unsaved in-memory edits, they are written to disk before the splice runs. The user has explicitly clicked Apply, so “intent to commit” is signalled; saving the source as a side effect keeps the on-disk state coherent with dlccorr (which is always saved by this method).

Source-layer rename. On a successful splice the patch layer is renamed in place from <old_name> to <old_name>_manual_corrections (the file is moved on disk too, old file deleted). The rename marks “this is the layer whose manual entries the on-disk dlccorr was spliced from” so the relationship survives reload. No-op if the patch is already named *_manual_corrections (idempotent re-apply).

Post-apply state. The corrections layer becomes the active annotation layer and the (now-renamed) manual layer becomes the overlay so you can see where your hand was. To iterate, switch the active layer back to your manual layer, set the overlay back to the DLC trace, add more points, click again. Each click regenerates the corrections layer from the current (overlay, active) pair, so adding annotations directly to the corrections layer is not recommended – they’ll be discarded on the next apply.

Idempotency. If a dlccorr layer is already present in the session, its in-memory data is replaced wholesale (with a _revision bump so the trace cache invalidates) and the file overwritten. Otherwise a fresh VideoAnnotation is built, saved, and adopted under the canonical machinery.

Parameters:

event – Mouse/keyboard event (unused, for button compat).

Raises:

ValueError – If no annotation overlay is currently set, or if the active annotation layer is already the corrections layer (would create a circular splice).

save_annotation_as(event=None)

Save the active annotation layer to a user-chosen path.

Opens a Qt save-file dialog seeded with the video’s folder and a suggested filename of <video_stem>_annotations_<layer>.json. Falls back to self.ann.save() (which writes to the layer’s existing .fname) when no Qt window is available – e.g. headless / Agg backend.

swap_active_and_overlay(event=None)

Swap the active annotation layer with the overlay layer.

No-op if no overlay is currently selected.

update()

Update the display with current frame and maintain frozen axis limits if set.

Returns:

Result from parent class update() method.

swap_to(index: int) → bool

Switch the active video to self._bundles[index]. See _bundle_swap.swap_to().

swap_prev(event=None) → bool

Move to the previous bundle (no-op at index 0).

Connected to the sidebar’s ◀ button and the Alt+Left keybinding. Returns the underlying swap_to() result so keybinding-handler callers can short-circuit if desired.

swap_next(event=None) → bool

Move to the next bundle (no-op at last index).

Connected to the sidebar’s ▶ button and the Alt+Right keybinding.

add_video(path_or_paths, *, layer_name=None, set_active=False, **dustrack_kwargs) → list[int]

Append one or more videos to this tracker’s bundle list. See _bundle_swap.add_video().

remove_video(index: int) → bool

Drop a bundle from the tracker’s bundle list. See _bundle_swap.remove_video().

replace_active_with(path_or_paths, *, layer_name=None, **dustrack_kwargs) → list[int]

Swap the active bundle for newly-picked video(s). See _bundle_swap.replace_active_with().

classmethod from_annotations(annotations: list[dustrack.annotations.VideoAnnotation], *args, **kwargs) → DUSTrack

add_annotation_layers(annotation_names: list[str] | dict[str, pathlib.Path] | list[dustrack.annotations.VideoAnnotation], n_labels: int = 1) → None

Load data from annotation files if they exist, otherwise initialize annotation layers.

remove_annotation_layer(name: str) → None

Remove an annotation layer from the active session.

Tears down the layer’s plot artists (scatter + per-label trace lines on x/y trace axes) via VideoAnnotation.clear_display(), drops the layer from annotations via AssetContainer.remove(), then resyncs the annotation_layer / annotation_overlay statevariables through _refresh_annotation_state_lists() so the dropdown rotations + current selections stay valid.

Pre-flight: - name must be an existing layer name. - Refuses if it would leave the container empty (callers needing

a “reset to single empty layer” semantic should use VideoAnnotation.reload() on the surviving layer instead).

Active-layer handoff: if name is currently the primary layer (annotation_layer.current_state), the previous layer in the rotation becomes the new primary (or the first one if the removed layer was at index 0). If name is currently the overlay, the overlay clears to None.

Note: "buffer" is not excluded here – the dnav layer treats every named layer the same. Consumers (DUSTrack) enforce buffer-exclusion at the UI layer.

set_key_bindings() → None

Set the keyboard actions.

Groups mirror the 5-step workflow in docs/source/resources/ keyboard_shortcuts.png: layer selection -> label selection -> frame navigation -> edit -> refine. self._section_order pins the cheatsheet’s section order to the workflow order (section 3 first, then 1, 2, 4, 5a, 5b, 5c) regardless of when each binding was originally registered. Bindings not on the PNG (save, refresh, reset view, pan, keep-overlapping, toggle-num-keys mode) fall through to the “Other” section.

refresh(event: Any | None = None) → None

Force-refresh the UI from the current annotation .data.

Drops the trace-display cache on every annotation layer and the frame-marker cache, then calls update() so the next draw re-reads every value from the backing dicts.

Recovery path for the rare case where .data was mutated directly (e.g. from an IPython prompt: v.ann.data["0"][42] = [x, y]) bypassing the public add() / remove() / add_at_frame() API and therefore skipping the _revision bump that VideoAnnotation.update_display_trace() and update_frame_marker() cache on. In normal in-code mutation paths the public API bumps _revision and the caches invalidate automatically; calling refresh() is always safe but rarely necessary.

add_events() → None

Add an event to specify time intervals for interpolating with lucas-kanade.

property ann: VideoAnnotation

Return current annotation layer.

set_image_background_color(color: Any) → None

Set the background color of the image region.

Tier 1 routes to self._ax_image.set_facecolor (matplotlib); Tier 2 routes to the Qt image pane’s set_background_color. The single-method surface lets consumers (DUSTrack’s _apply_dark_theme) issue one call regardless of which tier is active.

update_annotation_visibility(draw: bool = False) → None

Update the visibility of all annotation layers, for example, when the layer is changed.

update_frame_marker(draw: bool = False) → None

Update the current frame location in the trace plots.

The trace ylim / FOI tick positions are a pure function of the active label + per-annotation contents + frames_of_interest, none of which change when only _current_idx moves. Cache them keyed on a cheap tuple of (label, annotation revisions, frames_of_interest) so per-frame navigation only does the frame-marker set_data calls.

copy_annotations_from_overlay() → None

Copy annotations from the overlay layer into the current layer in the current frame.

copy_current_annotation_from_overlay() → None

Copy annotations from the overlay layer into the current layer.

copy_frames_of_interest_from_overlay() → None

copy annotations at frames of interest from buffer into the current layer. If there is no buffer, then copy from the overlay layer.

copy_frames_in_interval_from_overlay() → None

For the current label only.

add_annotation(event: Any) → None

Add annotation at frame. If it exists, it’ll get overwritten.

remove_annotation(event: Any | None = None) → None

remove annotation at the current frame if it exists

previous_frame_with_current_label(event: Any | None = None) → None

Go to the previous frame with the current label in the current annotation layer.

next_frame_with_current_label(event: Any | None = None) → None

Go to the next frame with the current label in the current annotation layer.

previous_frame_with_any_label(event: Any | None = None) → None

Go to the previous frame with any label in the current annotation layer.

next_frame_with_any_label(event: Any | None = None) → None

Go to the next frame with any label in the current annotation layer.

previous_frame_of_interest(event: Any | None = None) → None

Go to the previous frame of interest.

next_frame_of_interest(event: Any | None = None) → None

previous_annotation_layer() → None

Go to the previous annotation layer.

next_annotation_layer() → None

Go to the next annotation layer

previous_annotation_overlay() → None

Go to the previous annotation overlay layer.

next_annotation_overlay() → None

Go to the next annotation overlay layer.

previous_annotation_label() → None

Set current annotation label to the previous one.

next_annotation_label() → None

Set current annotation label to the next one.

update_annotation_label_states() → None

Update the states of the annotation label state variable.

cycle_number_keys_behavior() → None

Number keys can be used to either select labels, or place a specific label. Toggle between these two behaviors.

increment_label_range() → None

Increment the label range by 10.

decrement_label_range() → None

Increment the label range by 10.

increment_if_unannotated(event: Any | None = None) → None

Advance the frame if the current frame doesn’t have any annotations. Useful to pause at annotated frames when adding a new label.

decrement_if_unannotated(event: Any | None = None) → None

Go to the previous frame if the current frame doesn’t have any annotations. Useful to pause at annotated frames when adding a new label.

save() → None

Save current annotation layer json file.

.h5 layers (DLC predicted traces, labeled_data blocks) are not in-place editable through this key — the JSON-only contract on VideoAnnotation.save() would raise mid-keypress and look like a silent no-op. Short-circuit with a clear printed message that points the user to manual layers or to Save annotation as... for explicit copy-out.

select_label_with_mouse(event: Any) → None

Select a label by clicking on it with the left mousebutton.

place_label_with_mouse(event: Any) → None

Place the selected label with the right mousebutton.

go_to_frame(event: Any) → None

toggle_frame_of_interest(event: Any) → None

Mark/unmark the current frame as a frame of interest

keep_overlapping_continuous_frames() → None

keep_overlapping_frames() → None

predict_labels_with_lucas_kanade(labels: str | list[str] = 'all', start_frame: int | None = None, mode: str = 'full') → Any

Compute the location of labels at the current frame using Lucas-Kanade algorithm.

get_selected_interval() → tuple[int, int]

remove_labels_in_interval(all_labels: bool = False) → None

decimate_annotations_in_interval() → None

Prune incomplete frames, then drop every other complete frame in the selected interval.

Prep step for training: ensures the surviving in-interval frames are (a) fully annotated across the required-label set and (b) half as dense (even-stride sampling). Frame-level – every label is removed at each dropped frame.

Required-label set follows the same project-aware / project-unaware split the Train preflight uses (_preflight.scan_incomplete_frames()):

• DLC project loaded – required = project bodyparts (mapped through _dlc_bodyparts_to_layer_labels()). Stray non-bodypart labels on the layer are ignored from the required check, but any frame they sit on is still part of all_frames – if it’s missing a required label it still counts as incomplete, and pruning cleans the stray with it.

• No DLC project – required = labels with at least one annotation. Empty labels are treated as UI placeholders (user created a slot and abandoned it). This is the best inference available without external truth and matches the project-unaware Train preflight rule.

Starter form of the “general-model workflow” decimation feature; the DINOv3-feature farthest-point-sampling variant is deferred. Incomplete frames in the interval are always pruned. The every-other halving is a no-op when fewer than 2 complete frames remain in the interval after pruning.

interpolate_with_lk(all_labels: bool = False) → None

Interpolate with lk-rstc between selected interval. Use data in the overlay layer as start and end points. Add interpolated points to the current layer.

predict_range_with_dlc() → None

Run DLC inference over the selected interval (z z e).

The fast half of the refine loop: rather than a whole-video analyze_videos pass, this asks the current model what it predicts right here, in the interval just marked with z. Results land in the PREDICT_LAYER_NAME overlay so they render against the primary layer instead of overwriting it – the point is to see what changed, which is lost if the prediction clobbers the trace being compared.

Synchronous, like its a / x siblings in this group.

interpolate_with_lk_norstc(all_labels: bool = False) → None

Infer the motion of the current label using lucas-kanade algorithm in the selected interval.

check_labels_with_lk(mode: str = 'minimal') → None

Interpolate between all labeled frames. This only makes sense for sparse-labeled annotations. Use this when doing first-time annotations (as opposed to refinement).

I am testing if refining the start labels using this strategy, and augmenting the training data will improve deeplabcut tracking!

Parameters:

mode (str, optional) – “all” - LK-interpolation for all labels across all labeled frames “current” - LK-interpolation for current label across all labeled frames “minimal” - LK-interpolation for current label between labeled frames near the current frame (two previous to two next) Defaults to “minimal”.

render(start_frame: int, end_frame: int, out_vid_name: str | None = None) → None

Render the video with annotations.

class dustrack.DLCProject(path, videos=[], name='test_01', experimenter='x', annotation_suffix='', internal_to_dlc_labels: dict = None, link_videos: bool | None = None)

Bases: object

Interface to deeplabcut training and inference Current workflow:

1. Create a project with some videos. Videos will be copied.

   d = DLCProject(r’C:/data_opr02/004_02/ml_models/dlc’, name=’opr02_s004_muscles’, experimenter=’praneeth’, videos=[<video_list>])

2. Launch the initial annnotator for video 0, repeat if there are more videos

   d.annotate(0)

3. Extract frames, train network, evaluate network, analyze videos, and create labeled video

   d.process()

4. Refine the labels

   d.annotate(0, ‘praneeth_2’) # the second argument determines the suffix for the annotations file. CAUTION: Make sure that the files are read by extract_frames in the correct order! Pay attention to the output of this method.

5. Re-train network with refined labels

   d.process()

Repeat steps 4 and 5 until satisfied with the results.

__init__(path, videos=[], name='test_01', experimenter='x', annotation_suffix='', internal_to_dlc_labels: dict = None, link_videos: bool | None = None)

Initialize or load a DeepLabCut project.

If a config.yaml exists at the path, loads the existing project. Otherwise, creates a new project with the provided videos.

Parameters:

• path (str) – Directory containing or for the project.

• videos (list) – List of video file paths to include.

• name (str) – Project name (must contain underscore for proper config handling).

• experimenter (str) – Experimenter identifier.

• annotation_suffix (str) – Suffix for annotation files (e.g., ‘manual’, ‘refined’).

• internal_to_dlc_labels (dict, optional) – Custom label name mapping.

• link_videos – How to place each source video inside <project>/videos/. None (default) hard-links via os.link() on the same volume and falls back to a regular copy with a stderr warning on cross-volume. True requires hard-linking and raises OSError on failure. False always copies (the pre-1.3.0a2 behavior).

Note

Videos are hard-linked into the project folder by default (1.3.0a2+), saving disk space when the source and the project share a volume. Pre-1.3.0a2 behavior (deep copy) is recoverable via link_videos=False. Project names without underscores may cause config issues with network paths.

property paths: Mapping[str, Path]

Full paths to project folder and standard DLC subfolders.

Returns:

Mapping of folder names to Path objects with keys:

• ’project’: Main project directory

• ’models’: Trained model weights (dlc-models or dlc-models-pytorch)

• ’results’: Evaluation results

• ’labels’: Labeled frame data

• ’training_data’: Training datasets

• ’videos’: Video files

Return type:

dict

property config: dict

Current project configuration dictionary.

Returns:

Parsed contents of config.yaml.

Return type:

dict

property name: str

Project name from configuration.

property trackers: list

Names of tracked body parts as used internally by DLC.

Returns:

Body part names (e.g., [‘point0’, ‘point1’]).

Return type:

list

property label_names: list

Human-readable names for tracked points.

Returns meaningful names from dlc_trackermap.txt if available, otherwise returns the internal tracker names.

Returns:

Display names for body parts.

Return type:

list

property trackermap

Load meaningful label names from dlc_trackermap.txt.

This file maps internal names (point0, point1) to biological names (nose, left_ear, etc.) for better interpretability.

Returns:

Mapping from internal names to display names.

Return type:

dict

Example dlc_trackermap.txt content:

point0 - muscle_boundary point1 - fascia point2 - bone

edit_config(config_file=None, **kwargs)

Modify project configuration parameters.

Parameters:

• config_file (str, optional) – Path to config file. Defaults to main config.

• **kwargs – Configuration parameters to update (e.g., iteration=2, snapshotindex=5).

Returns:

Result of deeplabcut.auxiliaryfunctions.edit_config().

property video_list: list[pathlib.Path]

Full paths to videos in the project.

property video_names: list[str]

Video filenames without extensions.

property current_iteration: int

Model iteration number currently set in config.yaml.

property latest_iteration: int

Highest iteration number in dlc-models folder.

property latest_trained_iteration: int

Most recent iteration that has saved model snapshots.

property all_iterations: list

All iteration numbers found in dlc-models, sorted ascending.

property all_snapshots: Mapping[int, list[int]]

Training snapshots for each model iteration.

Returns:

Maps iteration number to list of training iteration numbers.

For DLC3, identifies .pt files; for DLC2, identifies .index files.

Return type:

dict

current_iteration_is_trained() → bool

Check if current iteration has any saved snapshots.

latest_iteration_is_trained() → bool

Check if latest iteration has any saved snapshots.

iteration_is_trained(iteration_num: int) → bool

Check if a specific iteration has been trained.

Parameters:

iteration_num (int) – Model iteration to check.

Returns:

True if snapshots exist for this iteration.

Return type:

bool

increment_iteration()

Advance to next iteration if current one is trained.

Returns:

For method chaining.

Return type:

self

add_videos(videos: list[pathlib.Path], link_videos: bool | None = None)

Add new videos to existing project and copy their annotations.

Parameters:

• videos – List of video file paths to add.

• link_videos – Placement mode for the new videos. None (default) hard-links with copy fallback on cross-volume. True requires hard-linking. False always copies. Same semantics as DLCProject.__init__().

Returns:

For method chaining.

Return type:

self

copy_annotations(video_name: Path | list)

Copy DUSTrack JSON files into project’s video folder.

Parameters:

video_name – Single video path or list of video paths.

Returns:

Path(s) to copied annotation file(s), or None if not found.

Return type:

str or list

Note

Looks for files matching {video_stem}_annotations_{suffix}.json

extract_frames(annotation_file_names=None, suffix_merged='merged', save_merged_json=False, check=False)

Extract labeled frames from videos and convert annotations to DLC format.

This method: 1. Finds all annotation JSON files for each video 2. Merges multiple annotation files if present 3. Extracts the annotated frames from videos 4. Converts annotations to DLC’s CSV/HDF5 format in labeled-data folder

Parameters:

• annotation_file_names (list, optional) – Specific annotation files to use. If None, automatically finds all matching files.

• suffix_merged (str) – Suffix for merged annotation file. Defaults to ‘merged’.

• save_merged_json (bool) – Whether to save the merged JSON. Defaults to False.

• check (bool) – Whether to run deeplabcut.check_labels(). Defaults to False.

Returns:

For method chaining.

Return type:

self

Note

Automatically excludes files with ‘_dlccorr’ suffix (correction files).

get_pose_cfg_file(iteration_num: int = None, type_: str = 'train') → Path

Get path to pose configuration file for an iteration.

Parameters:

• iteration_num (int, optional) – Iteration number. Defaults to current.

• type (str) – ‘train’ or ‘test’ subfolder. Defaults to ‘train’.

Returns:

Full path to pose_cfg.yaml (DLC2) or pytorch_config (DLC3).

Return type:

Path

get_best_snapshot(iteration_num: int = None) → int

Find training iteration with lowest test error.

For DLC3, uses the snapshot marked as ‘best’ unless DLC3_USE_LAST_SNAPSHOT is True in config, in which case returns the last snapshot. For DLC2, parses CombinedEvaluation-results.csv.

Parameters:

iteration_num (int, optional) – Model iteration. Defaults to current.

Returns:

Training iteration number of best snapshot.

Return type:

int

get_best_snapshot_test_error(iteration_num: int = None) → float

Get test error (RMSE in pixels) at best snapshot.

Parameters:

iteration_num (int, optional) – Model iteration. Defaults to latest trained.

Returns:

Test error in pixels, or -1.0 if evaluation file doesn’t exist.

Return type:

float

get_best_snapshot_idx(iteration_num: int = None) → int

Get snapshot index (not training iteration number) of best snapshot.

Parameters:

iteration_num (int, optional) – Model iteration. Defaults to current.

Returns:

Index in the all_snapshots list for this iteration.

Return type:

int

initialize_weights(source_iteration: int = None, source_snapshot: int = None, dest_iteration: int = None)

Initialize model weights from a previous iteration (transfer learning).

Used when refining a model with additional labels. Edits the pose_cfg file to set init_weights parameter.

Parameters:

• source_iteration (int, optional) – Iteration to copy from. Defaults to second-to-last iteration.

• source_snapshot (int, optional) – Training iteration within source_iteration. Defaults to best snapshot.

• dest_iteration (int, optional) – Iteration to initialize. Defaults to latest iteration.

Returns:

For method chaining.

Return type:

self

Note

Does nothing if there’s only one iteration (no source to copy from).

create_training_dataset(**kwargs)

Call deeplabcut.create_training_dataset.

train(**kwargs)

Train the neural network model.

Sets custom learning rate schedule and trains with more iterations than DLC defaults for better convergence.

Parameters:

**kwargs – Passed to deeplabcut.train_network(). - maxiters (int): Total training iterations. Default: 500000 (DLC2) or 1000 (DLC3 epochs). - max_snapshots_to_keep (int): Max saved checkpoints. Default: 20.

Returns:

For method chaining.

Return type:

self

Note

Custom learning rate schedule: [0.005@10k, 0.02@350k, 0.002@425k, 0.001@1M]

evaluate(**kwargs)

Evaluate all training snapshots on test set.

Temporarily sets snapshotindex to ‘all’ to evaluate every checkpoint, then restores original value.

Parameters:

**kwargs – Passed to deeplabcut.evaluate_network().

Returns:

For method chaining.

Return type:

self

analyze_videos(iteration_num=None, snapshotindex=None, create_video=True, **kwargs)

Run inference on videos and optionally create labeled output videos.

Parameters:

• iteration_num (int, optional) – Model iteration to use. Defaults to current.

• snapshotindex (int, optional) – Snapshot index to use. Defaults to best snapshot. Negative indices supported.

• create_video (bool) – Whether to create labeled video. Defaults to True.

• **kwargs – Additional arguments for deeplabcut.analyze_videos(). - videos: List of video paths or indices. If not provided, analyzes all videos.

Returns:

For method chaining.

Return type:

self

Note

Results saved to videos/iteration-{N}/ subfolder. If videos kwarg contains integers, they’re treated as indices into self.video_list.

process(iteration_num=None, maxiters=None, refine: bool | str = True, create_video=True, source_snapshot=None, **kwargs)

Automated workflow: extract frames, train, evaluate, and analyze.

This is the main method for handling the full DLC pipeline. It intelligently decides what steps to run based on the current project state: - If iteration already evaluated: just analyze videos - If frames need extraction: extract them - If not trained: train the model - If refining: initialize weights from previous iteration

Parameters:

• iteration_num (int or str, optional) – Iteration to process. Can be integer or ‘latest’. Defaults to ‘latest’.

• maxiters (int, optional) – Training iterations. Defaults: 500000 (DLC2) or 1000 epochs (DLC3).

• refine (bool) – Use transfer learning from previous iteration. Defaults to True.

• create_video (bool) – Create labeled output video. Defaults to True.

• source_snapshot (int, optional) – Specific snapshot for weight initialization.

• **kwargs – Additional arguments. - videos: List of videos to analyze (can be indices or paths).

Returns:

For method chaining.

Return type:

self

Example

>>> proj = DLCProject('path/to/project')
>>> proj.process()  # Full automated workflow

train_iteration(*, refine_mode: Literal['scratch', 'in_project', 'external'] = 'scratch', source_iteration: int = None, source_snapshot: int = None, external_snapshot_path: str = None, maxiters: int = None, create_video: bool = False, videos: list = None, analyze_batchsize: int = None)

Explicit-args training driver for UI-triggered flows.

Distinct from process() (auto-infer for CLI ergonomics). Caller decides everything: refine source, training duration, output options. Strict validation per refine_mode; no inference and no silent fallbacks.

The mechanics of advancing iterations (extract_frames → increment_iteration if latest is trained → create_training_dataset if needed) mirror process(); the only difference is how weights are initialised once the destination iteration is in place.

Parameters:

• refine_mode – How to initialise weights for the next training round. "scratch" starts from random init (no pose_cfg edit); "in_project" copies weights from a snapshot in this project (requires source_iteration, optional source_snapshot); "external" initialises from an arbitrary snapshot file (requires external_snapshot_path; supported on both DLC2 and DLC3 – DLC3 passes the path through train_network(snapshot_path=...), DLC2 edits pose_cfg’s init_weights via _initialize_weights_from_external_path()).

• source_iteration – in-project iteration to copy weights from. Only valid with refine_mode="in_project"; must point at a trained iteration.

• source_snapshot – specific snapshot within source_iteration. Only valid with refine_mode="in_project"; defaults to the best snapshot when None.

• external_snapshot_path – path to an external snapshot file (.pt on DLC3, .index on DLC2). Only valid with refine_mode="external"; the file must exist at call time.

• maxiters – training epochs (DLC3) or iterations (DLC2). Defaults to the same values process() uses (50 / 500000) so the two methods stay consistent until the UI exposes the field.

• create_video – write a labeled output video after analyze. Defaults to False (the UI ergonomics default; process() defaults to True for CLI parity).

• videos – list of videos (indices or paths) to analyze. Forwarded to analyze_videos. None analyses every video in the project.

• analyze_batchsize – batchsize for analyze_videos. Forwarded on if set; None lets analyze_videos pick its own default (post-2026-05-20: rc14 knee at 4).

Returns:

For method chaining.

Return type:

self

Raises:

• ValueError – on refine_mode / argument mismatch (see _validate_train_iteration_args()).

• TypeError – if source_iteration / source_snapshot aren’t int when given.

• FileNotFoundError – if external_snapshot_path is set but the file doesn’t exist.

annotate(video_index: int = 0, new_annotation_suffix=None, **dustrack_kwargs)

Launch interactive annotation GUI for a video.

Opens DUSTrack interface with existing annotation layers loaded, including any DLC predictions as line plot overlays.

Parameters:

• video_index (int) – Index of video in video_list. Defaults to 0. Negative indices supported.

• new_annotation_suffix (str, optional) – Suffix for new annotation layer. Defaults to ‘iteration-{N}’ where N is the next iteration number.

• **dustrack_kwargs – Forwarded to the DUSTrack constructor. Notable pass-through options: fast_render=True (datanavigator 1.5.0+ Tier 2 Qt-native video pane, ~3x speedup on the interosseous_pn24-x benchmark), dark_mode=True, clahe_clip, clahe_grid, gamma, brightness.

Returns:

Interactive annotation interface.

Return type:

DUSTrack

Note

Creates a ‘buffer’ layer for temporary annotations. Latest DLC predictions are automatically set as overlay.

get_trajectories(videos=None, iteration=None)

Load tracking results as DLCData objects.

Parameters:

• videos (list or str, optional) – Videos to load. Defaults to all videos.

• iteration (int, optional) – Model iteration. Defaults to current.

Returns:

Maps video stem to DLCData object.

Return type:

dict

Raises:

ValueError – If a requested video is not in the project.

open()

Open project folder in Windows Explorer.

class dustrack.VideoAnnotation(fname: str | None = None, vname: str | None = None, name: str | None = None, n_labels: int = 1, **kwargs)

Bases: object

Manage one point annotation layer in a video.

Each annotation layer can contain up to 10 labels, which are string representations of digits 0-9. Each label is a dictionary mapping a frame number to a 2D location on the video frame.

Parameters:

• fname (str, optional) – File name of the annotations (.json) file. If it doesn’t exist, it will be created when the save method is used. If this is a video file, fname will default to <video_name>_annotations.json. This can also be a DeepLabCut .h5 file (either labeled data OR predicted trace). Defaults to None.

• vname (str, optional) – Name of the video being annotated. Defaults to None.

• name (str, optional) – Name of the annotation (something meaningful, e.g., <muscle_name>_<scorer> such as brachialis_praneeth). Defaults to None.

• n_labels (int, optional) – Number of labels for annotation. Defaults to 10.

• **kwargs – Additional optional parameters: - palette_name (str, default=’Set2’): Color scheme to use. Defaults to ‘Set2’ from seaborn. - ax_list (list, default=[]): If specified, the annotation display will be initialized on these axes. Alternatively, use VideoAnnotation.setup_display() to specify the axis list and colors. - preloaded_json (dict, optional): The result of VideoAnnotation._load_json (in case you prefer to pickle the JSON files).

to_dlc()

Convert from JSON file format into a DeepLabCut DataFrame format, and optionally save the file.

__init__(fname: str | None = None, vname: str | None = None, name: str | None = None, n_labels: int = 1, **kwargs) → None

property data: dict[str, dustrack.annotations._TrackedFrameDict]

Per-label frame→location dictionary.

Reads behave like a normal dict[str, dict[int, list[float]]]. Writes through ann.data[label][frame] = ... automatically bump _revision, keeping consumers’ caches consistent. See _TrackedFrameDict.

classmethod from_multiple_files(fname_list: list[str], vname: str, name: str, fname_merged: str, **kwargs) → VideoAnnotation

Merge annotations from multiple files. If multiple files contain an annotation label for the same frame, values from the last file will be kept.

load(n_annotations: int = 1, **h5_kwargs) → dict

Load annotations from a json file, dlc h5 file, or initialize an annotation dictionary if a file doesn’t exist.

DUSTrack-shaped: the DeepLabCut .h5 branch exists in the generic class because datanavigator and DUSTrack co-developed. The DLC paths will migrate to dustrack.VideoAnnotation in 1.3.0 alongside the pointtracking.py split; the JSON path stays here.

property n_frames: int

Number of frames in the video being annotated

property n_annotations: int

Number of points being annotated in the video.

property labels: list[str]

Labels of the annotations.

property palette: list[tuple]

Color palette for the annotations.

property frames: list[int]

Frame numbers in the video that have annotations.

property frames_overlapping: list[int]

list of frames in the video where all the labels are annotated.

property plot_type: str

Type of plot to use for the annotations.

get_frames(label: str) → list[int]

Return a list of frames that are annotated with the current label.

reload() → None

Drop in-memory state and reload from disk (or start fresh).

Inverse of save(). If fname is None or the file doesn’t exist, restores the empty {str(i): {} for i in range(n)} shape via the existing load() fallback (see the file-missing branch). Wholesale- replaces self.data so the property setter rewraps every per-label inner dict as a _TrackedFrameDict, and bumps _revision explicitly so per-frame caches keyed on (label_list, _revision) invalidate – the outer setter rewraps but does not itself bump (mirrors sort_data() and remove_empty_labels()).

save(fname: str | None = None) → None

Save the annotations json file. self.fname should be a valid file path.

Empty-but-declared labels are preserved (written as "label": {}). 1.4.0rc2 promoted labels from a derived property of “which keys have data” to first-class schema; the previous implicit self.remove_empty_labels() call here is gone. Callers that want a lean export (e.g. DUSTrack pre-flight before DLC training) still drive remove_empty_labels() explicitly.

to_json() → None

Alias for save method.

sort_labels() → None

Sort labels in the data dictionary.

sort_data() → None

Sort annotations by the frame numbers.

clip_trailing_empty_labels() → None

Remove trailing empty labels from the annotation dictionary.

remove_empty_labels() → None

Remove empty labels from the annotation dictionary.

get_values_cv(frame_num: int) → ndarray

Return annotations at frame_num in a format for openCV’s optical flow algorithms

add_at_frame(frame_num: int, values: ndarray) → None

Add annotations at a frame, given the annotation values.

get_at_frame(frame_num: int) → list[list[float]]

Retrieve annotations at a given frame number. If an annotation is not present, nan values will be used.

to_dlc(scorer: str = 'praneeth', output_path: str | None = None, file_prefix: str | None = None, img_prefix: str = 'img', img_suffix: str = '.png', label_prefix: str = 'point', save: bool = True, internal_to_dlc_labels: dict[str, str] | None = None) → DataFrame

Save annotations in deeplabcut format.

DUSTrack-shaped: emits a DeepLabCut-shaped DataFrame (and writes an .h5 if save=True); slated to migrate to dustrack.VideoAnnotation in 1.3.0 alongside the pointtracking.py split.

to_trace(label: str) → ndarray

Return a 2d numpy array of n_frames x 2.

Parameters:

label (str) – Annotation label.

Returns:

2d numpy array of number of frames x 2.

xy position values of uannotated frames will be filled with np.nan.

Return type:

np.ndarray

Schema-tolerant: a label absent from data is treated as all-frames-unannotated and returns the full NaN array. Lets cross-layer trace consumers (e.g. _DUSTrackBase.update_frame_marker(), which iterates every annotation layer with one shared label) survive a layer that legitimately doesn’t carry the active label – either a freshly created layer or a layer where the user hasn’t placed any points for that label yet. 1.4.0rc2: prior to the schema-tolerant relaxation this asserted, which crashed the corrections-layer flow when the patch had been saved with one of its labels empty (save then pruned that label, so it disappeared from labels and the assert fired).

to_traces() → Mapping[str, ndarray]

Return annotations as traces (numpy arrays of size n_frames x 2).

Returns:

Dictionary mapping labels to 2d numpy arrays.

Return type:

Mapping[str, np.ndarray]

to_signal(label: str) → Data

Return an annotation as pysampled.Data at the frame rate of the video.

Parameters:

label (str) – Annotation label.

Returns:

Signal sampled at the frame rate of the video being annotated.

Return type:

pysampled.Data

to_signals() → Mapping[str, Data]

Return annotations as a dictionary of sampled Data signals sampled at the frame rate of the video being annotated.

Returns:

Dictionary mapping labels to pysampled.Data.

Return type:

Mapping[str, pysampled.Data]

to_pysampled() → Data

Return annotations as a pysampled.Data object

Returns:

pysampled.Data

add_label(label: str | None = None, color: tuple[float, float, float] | None = None) → None

add(location: list[float], label: str, frame_number: int) → None

Add a point annotation (location) of a given label at a frame number.

remove(label: str, frame_number: int) → None

Remove a point annotation of a given label at a frame number.

setup_display_scatter(ax_list_scatter=None) → None

Setup scatter plot display.

Each entry in ax_list_scatter is either: - a matplotlib Axes (Tier 1) – a PathCollection is

created via ax.scatter and stashed in plot_handles;

• a Qt QGraphicsItemGroup (Tier 2) – a datanavigator._qt._QtScatterArtist is built on the group and stashed instead. Both expose the same mpl-PathCollection-shaped API consumed downstream.

setup_display_trace(ax_list_trace_x: None | Axes | list[matplotlib.axes._axes.Axes] = None, ax_list_trace_y: None | Axes | list[matplotlib.axes._axes.Axes] = None) → None

Setup trace plot display.

setup_display(ax_list_scatter: None | Axes | list[matplotlib.axes._axes.Axes] = None, ax_list_trace_x: None | Axes | list[matplotlib.axes._axes.Axes] = None, ax_list_trace_y: None | Axes | list[matplotlib.axes._axes.Axes] = None) → None

Setup display for scatter and trace plots.

clear_display() → None

Clear the display.

re_setup_display() → None

re-establish display elements when adding a label

invalidate_caches() → None

Drop per-annotation render caches so the next update_display_trace re-runs the full ydata sweep.

Recovery hatch for direct .data mutations from the command line (which skip the _revision bump the trace cache keys on). Normal in-code mutations should go through add() / remove() / add_at_frame() instead.

update_display_scatter(frame_number: int, draw: bool = False) → None

Update scatter plot display.

update_display_trace(label: str | None = None, draw: bool = False) → None

Update trace plot display.

Trace contents are a pure function of (label_list, self.data); none of that changes when only the parent’s _current_idx moves. Cache on (label_list, self._revision) so per-frame navigation skips the to_trace / set_ydata sweep entirely.

update_display(frame_number: int, label: str | None = None, draw: bool = False) → None

Update scatter and trace plot display.

hide(draw: bool = True) → None

Hide all elements (scatter, traces) in this annotation.

show(draw: bool = True) → None

Show all elements (scatter, traces) in this annotation.

show_trace(label: str, draw: bool = True) → None

Show trace for a specific label.

hide_trace(label: str, draw: bool = True) → None

Hide trace for a specific label.

show_one_trace(label: str, draw: bool = True) → None

Show only one trace for a specific label.

postprocess(video_name: str = None, window_size: float = 0.5, use_parallel: bool = True, save_raw: bool = True) → VideoAnnotation

Apply Lucas-Kanade moving average filter to smooth tracking annotations.

This is the main post-processing function. It applies RSTC over overlapping time windows and averages the results to produce smooth, jitter-free trajectories.

Workflow:

1. Load video and annotations

2. Slide a time window through the video

3. For each window: - Apply RSTC to get smoothed trajectory - Store results indexed by window position

4. Average all trajectories that cover each frame

5. Save smoothed annotations

Parameters:

• tracked_points (Union[str, VideoAnnotation]) – Either: - Path to annotation JSON file - VideoAnnotation object with tracking data

• video_name (str, optional) – Video path. Defaults to None; required when tracked_points is a file path (asserted at call time). Ignored when tracked_points is already a VideoAnnotation.

• window_size (float, optional) – Time window in seconds for moving average. Larger windows provide more smoothing but increase computation. Typical range: 0.1 to 1.0 seconds. Defaults to 0.5.

• use_parallel (bool, optional) – Use ThreadPoolExecutor for parallel processing. Significantly faster for longer videos. Defaults to True.

• save_raw (bool, optional) – If True (default), allocate the full (n_window_frames, n_total_frames, n_labels, 2) per-window RSTC array and dill-pickle it to {stem}_lkmovavg_{window_size}.pkl alongside the averaged JSON. Per-window data is consumed downstream by pn-projects/wobble and gaitmusic (their .rawlk property → lk_gradients velocity estimation), so keep True for those flows. If False, stream a running (n_total_frames, n_labels, 2) sum and per-cell count instead, divide at the end, and skip the .pkl write. Cuts peak memory from W*N*L*16 bytes to N*L*12 bytes (~10x on typical W=15, L=4 cases). On the example-video bench wall time is essentially unchanged (post-processing is <0.3% of total) but on real-world long videos (N=36715+) the .pkl round-trip alone is a couple of seconds. FP summation order changes between modes so save_raw=False output is numerically close (np.allclose with atol~1e-10) rather than bit-exact vs save_raw=True.

Returns:

New annotation object with smoothed trajectories.

Saved to {original_stem}_lkmovavg_{window_size}.json

Return type:

VideoAnnotation

Output Files:

• {stem}_lkmovavg_{window_size}.pkl: Per-window RSTC results (only when save_raw=True; consumed by .rawlk downstream).

• {stem}_lkmovavg_{window_size}.json: Averaged smoothed annotations (always written).

Performance Tips:

• Use use_parallel=True for videos longer than a few seconds

• Smaller window_size is faster but provides less smoothing

• Results are cached: the .json is the primary cache, and when save_raw=True the .pkl is reused on subsequent calls regardless of the save_raw flag (a pre-existing .pkl is always read if the .json is missing). Delete either to recompute the corresponding stage.

Example

>>> # Post-process manual annotations
>>> ann = VideoAnnotation('video_annotations.json', 'video.mp4')
>>> ann_smooth = lk_moving_average_filter(ann, window_size=0.5)
>>>
>>> # Post-process DLC predictions with larger window
>>> ann_dlc = VideoAnnotation('video_dlc_predictions.h5', 'video.mp4')
>>> ann_smooth = lk_moving_average_filter(ann_dlc, window_size=1.0)
>>>
>>> # Non-parallel processing for short clips
>>> ann_smooth = lk_moving_average_filter(ann, window_size=0.3, use_parallel=False)
>>>
>>> # GUI-style: skip the .pkl sidecar (memory + I/O savings)
>>> ann_smooth = lk_moving_average_filter(ann, save_raw=False)

In practice, the postprocess shortcut on a VideoAnnotation is equivalent: >>> from dustrack import VideoAnnotation >>> ann = VideoAnnotation(‘video_dlc_predictions.h5’, ‘video.mp4’) >>> ann_rstc = ann.postprocess()

Raises:

• AssertionError – If video_name is not provided when tracked_points is a file path.

• AssertionError – If tracked_points is not a VideoAnnotation or valid file path.

Note

The algorithm uses a deque-based frame buffer to minimize memory usage and avoid repeated video decoding. Frames are read once and discarded after the window moves past them.

set_alpha(alpha: float = 0.4, label: str | None = None, draw: bool = True) → None

Set the transparency level of all (or one) the traces and labels in this annotation.

Parameters:

• alpha (float, optional) – alpha value between 0 and 1. Defaults to 0.4.

• label (_type_, optional) – Defaults to all labels.

• draw (bool, optional) – Update display if True. Defaults to True.

set_plot_type(type_: str = 'line', draw: bool = True) → None

Set the plot type for traces.

Records the choice on _plot_type and applies the visual style, so subsequent re_setup_display() / setup_display() calls (which read self.plot_type) don’t revert to the stale value. Symmetric with the plot_type property setter.

Parameters:

• type (str, optional) – “line” or “dot”. Defaults to “line”.

• draw (bool, optional) – Update display if True. Defaults to True.

clip_labels(start_frame: int, end_frame: int) → None

Remove annotations outside the clip range. Clip range includes start and end frame.

keep_overlapping_continuous_frames() → None

Keep data from consecutive frames that have all labels.

keep_overlapping_frames() → None

Keep data only from frames where every label is annotated.

Sibling of keep_overlapping_continuous_frames() without the consecutive-runs constraint: fully-labeled but isolated frames are preserved. Motivating use case is DLC training pre-flight, where partial frames degrade the trained model even though DLC tolerates per-bodypart NaN in its CSV.

get_area(labels: list[str] | str, lowpass: float | None = None) → Data | ndarray

Get the area in pixel squared.

export_video(out_file_name=None, start_frame=None, end_frame=None)

class dustrack.VideoAnnotations(parent: Any)

Bases: AssetContainer

add(name: str | VideoAnnotation, fname=None, vname=None, **kwargs) → VideoAnnotation

Create-and-add

reorder(names: list[str]) → None

Permute the underlying _list so layer names follow names.

names must be a permutation of the current AssetContainer.names. Callers that own the rotation of a annotation_layer / annotation_overlay StateVariable (i.e. _DUSTrackBase and subclasses) are responsible for resyncing those after this returns – reorder only manages the membership order.

dustrack.build_toc(sources: str | Path | Iterable[str | Path], *, extensions: Iterable[str] = ('.mp4', '.mov', '.mkv', '.avi', '.m4v'), recursive: bool = True, force: bool = False, show_progress: bool = True, progress_callback: Callable[[int, int, Path, str], None] | None = None, cancel_check: Callable[[], bool] | None = None) → dict

Pre-build the PyAV+TOC sidecar for every video under sources.

Thin pass-through to datanavigator.precompute_toc_folder() so DUSTrack callers can stay inside the dustrack namespace. First open of a video pays the per-file TOC build cost; pre-building means dustrack.open(...) returns essentially instantly afterward.

Example:

import dustrack
dustrack.batch.build_toc("M:/us_videos_for_tracking2")

Parameters:

• sources – A directory, a video file, or an iterable mixing both. Directories are walked for extensions.

• extensions – File extensions to include (case-insensitive). Default matches dnav’s DEFAULT_VIDEO_EXTENSIONS.

• recursive – If True (default), recurse into subdirectories.

• force – If True, rebuild even when a valid cache exists. Useful for upgrading pre-1.3 sidecars to schema v2 with per-frame timestamps.

• show_progress – If True (default), wrap iteration in a tqdm bar. Suppressed automatically when progress_callback is set.

• progress_callback – Optional fn(idx, total, video_path, status) invoked after each video. status is the per-file result string from datanavigator.precompute_toc() ("hit", "built", "built (uncached)", "error: ..."). Forwarded to dnav; the path is converted to a Path for consumer convenience (dnav itself emits a string path).

• cancel_check – Optional zero-arg callable polled at the top of each file. If truthy, the loop exits early and the partial result dict is returned.

Returns:

{path: status} per datanavigator.precompute_toc(), plus an "error: missing" entry for any explicitly-named path that doesn’t exist.

dustrack.convert_to_mono(sources: str | Path | Iterable[str | Path], *, crf: int = 22, preset: str = 'slow', suffix: str = '_mono', skip_existing: bool = True, skip_already_mono: bool = True, ffmpeg_bin: str | None = None, ffprobe_bin: str | None = None, verbose: bool = True, show_progress: bool = False, progress_callback: Callable[[int, int, Path, str], None] | None = None, cancel_check: Callable[[], bool] | None = None) → list[pathlib.Path]

Re-encode each source as h265 monochrome alongside the original.

Parameters:

• sources – A video path, an iterable of video paths, or a directory (recursive walk of supported extensions). Originals are never modified – each output is written as <source_stem><suffix>.mp4 in the same folder.

• crf – libx265 quality. Lower = better quality, bigger file. Default 22 lands within 6% of typical ultrasound capture file size on 706x558 clips; DLC inference parity at CRF 22 is sub-pixel (median 0.19 px). Pass a lower value (e.g. 18) for tighter pixel parity at a 50%+ file-size cost.

• preset – libx265 preset. Slower = better compression at same CRF. Default "slow" matches the immersionlab telemed convention.

• suffix – Appended to the source stem to form the output filename. Default "_mono".

• skip_existing – When True (default), skip sources whose output file already exists. Pass False to force re-encode.

• skip_already_mono – When True (default), skip sources whose encoded pix_fmt is already monochrome (no point re-encoding a mono file). Detected via ffprobe.

• ffmpeg_bin – Override ffmpeg path; default looks on PATH and at C:/ffmpeg/bin/ffmpeg.exe.

• ffprobe_bin – Override ffprobe path; same lookup rules as ffmpeg.

• verbose – Print one status line per source.

• show_progress – If True, wrap the per-file loop in a tqdm bar (per-file status lines are routed through tqdm.write so the bar stays clean). Matches the show_progress kwarg on build_toc(). Off by default to preserve the historical print-only behaviour for shell users.

• progress_callback – Optional fn(idx, total, src_path, status) invoked after each source. status is one of "ok", "skip_missing", "skip_overwrite", "skip_existing", "skip_already_mono", or "failed". Used by the Qt batch-process modal to drive its own progress UI without relying on tqdm.

• cancel_check – Optional zero-arg callable polled at the top of each source. If it returns truthy, the loop exits early. Used by the Qt batch-process modal’s Cancel button.

Returns:

List of output paths actually written (skipped sources omitted).

dustrack.detect_blips(ann: VideoAnnotation, *, threshold_factor: float = 5.0, max_blip_length: int = 5, return_position_factor: float = 3.0) → BlipReport

Find sparse blip runs in a dense-trace annotation, per label.

Pure-Python: does not decode the video.

Parameters:

• ann – A VideoAnnotation whose data[label] is densely populated across frames (typically a DLC .h5 predicted trace; partial coverage is tolerated — NaN frames are skipped).

• threshold_factor – Per-label entry threshold in units of robust sigma (1.4826 * MAD) above the median displacement. Default 5.0 is conservative; lower flags more candidates.

• max_blip_length – Maximum run length (in frames) the bracketing scan will consider before giving up on a spike. Long runs are usually real model failures, not blips.

• return_position_factor – Multiplier on the per-blip return-tolerance factor * median_d * (run_len + 1). Default 3.0 allows the return anchor to land within ~3 typical-per-frame displacements per run-frame of the pre-blip anchor.

Returns:

BlipReport.

dustrack.detect_and_interpolate_blips(ann: VideoAnnotation, *, save: bool = True, threshold_factor: float = 5.0, max_blip_length: int = 5, return_position_factor: float = 3.0, lk_config: dict | None = None, progress_callback: Callable[[int, int], None] | None = None) → tuple[dustrack.annotations.VideoAnnotation, dustrack.blip.BlipReport]

Convenience wrapper: detect, interpolate, optionally save.

Parameters:

• ann – Source dense-trace annotation.

• save – If True (default), write the sparse output to <source_stem>_blip_corrections.json next to the source. Refuses to overwrite an existing file (raises FileExistsError); delete or rename the existing file first.

• threshold_factor – See detect_blips().

• max_blip_length – See detect_blips().

• return_position_factor – See detect_blips().

• lk_config – See interpolate_blips().

Returns:

(sparse_annotation, report).

Raises:

FileExistsError – If save=True and the output file already exists. Per [[benchmark-cleanup-collision]]: never silently overwrite a user file.

dustrack.extract_snapshot_for_seeding(snapshot_path: str | Path, destination_path: str | Path, description: str = '') → Path

Copy a DLC3 snapshot + its pytorch_config.yaml + sibling test/pose_cfg.yaml into a seed bundle.

Parameters:

• snapshot_path – Path to a snapshot-*.pt file inside an existing DLC3 project’s dlc-models-pytorch/iteration-N/<modelfolder>/train/ folder.

• destination_path – Folder to write the bundle into. Created if missing; pre-existing unrelated files are preserved, but matching snapshot-*.pt / pytorch_config.yaml / pose_cfg.yaml / description.txt are overwritten.

• description – Optional human-readable description of the bundle (one paragraph; what the model tracks, training corpus, intended use). Written to description.txt in the bundle and surfaced by the seeding-modal picker. Empty string skips the file.

Returns:

The destination folder (as a pathlib.Path).

Return type:

Path

Raises:

FileNotFoundError – If snapshot_path does not point at an existing .pt file, or if pytorch_config.yaml / sibling test/pose_cfg.yaml are missing.

dustrack.get_seed_bundles_root() → Path | None

Read the remembered seed-bundles root from ~/.dustrack/config.json. Returns None if unset or the path no longer exists on disk.

dustrack.import_seed_bundle_into_project(dlc_project, bundle_path: str | Path, iteration: int = 0, shuffle: int = 1) → Path

Wire a seed bundle into a DLC project so DLC sees iteration-N as already trained with the bundled snapshot.

Side effects:

• dlc_project.edit_config(bodyparts=<from bundle>) – the project’s bodypart list is overwritten by the bundle’s metadata.bodyparts. The model literally expects those output channels, so the project must match.

• Creates <project>/dlc-models-pytorch/iteration-<N>/<modelfolder>/{train,test}/, where <modelfolder> is the DLC convention <Task><date>-trainset<int(frac*100)>shuffle<shuffle> (see deeplabcut/utils/auxiliaryfunctions.get_model_folder).

• Copies the snapshot into train/ and writes train/pytorch_config.yaml with metadata.project_path and metadata.pose_config_path rewritten to the destination.

• Writes test/pose_cfg.yaml with its dataset field rewritten to the new project root.

• Writes training-datasets/iteration-<N>/UnaugmentedDataSet_<Task><date>/metadata.yaml with a single shuffle entry matching the manufactured folder. DLC’s analyze_videos resolves shuffles through this file (TrainingDatasetMetadata.get), so without it inference raises Could not find a shuffle with trainingset fraction X and index N.

After this returns, dlc_project.analyze_videos(iteration_num=N) will pick up the bundled snapshot via all_snapshots (which globs *train/snapshot*.pt) and run inference against the new video without any training having happened.

Parameters:

• dlc_project – A dustrack.dlcinterface.DLCProject instance, just constructed (typically with an empty active manual layer – bodyparts: [] in config.yaml, no iteration-N folder yet).

• bundle_path – Folder produced by extract_snapshot_for_seeding().

• iteration – Iteration number to install the bundle under (default 0).

• shuffle – Shuffle number for the model folder name (default 1, matching DLC’s analyze_videos default).

Returns:

The created modelfolder (.../iteration-<N>/<modelfolder>).

Return type:

Path

Raises:

FileNotFoundError, ValueError – from inspect_seed_bundle().

dustrack.inspect_seed_bundle(bundle_path: str | Path) → dict

Validate a seed bundle and return its metadata.

Parameters:

bundle_path – Folder containing snapshot-*.pt, pytorch_config.yaml, and pose_cfg.yaml.

Returns:

{"snapshot": Path, "pytorch_config": Path, "pose_cfg": Path, "bodyparts": list[str], "net_type": str}.

Return type:

dict

Raises:

• FileNotFoundError – If any of the three required files is missing.

• ValueError – If multiple snapshot-*.pt files are present (bundle is ambiguous), or if pytorch_config.yaml lacks metadata.bodyparts.

dustrack.interpolate_blips(ann: VideoAnnotation, report: BlipReport, *, lk_config: dict | None = None, progress_callback: Callable[[int, int], None] | None = None) → VideoAnnotation

LK-RSTC re-track every blip and return a sparse corrections layer.

The returned VideoAnnotation shares the source video and label set, but data[label] is populated only at blip frames for blipped labels (other labels expose empty {}). Suitable for saving as a new annotation layer and consuming as DLC training data via the standard NaN-tolerant labeled-data path.

Parameters:

• ann – Source dense-trace annotation (the one passed to detect_blips()). Its video is reused for decode.

• report – Detection result from detect_blips().

• lk_config – Optional override for the per-pair LK config (passed through to dustrack.lk_opticalflow.lucas_kanade_rstc()).

• progress_callback – Optional (done_count, total_count) callback fired once per completed blip. Used by the UI ProgressOverlay to drive a progress bar; headless callers can leave it None. Fires after the per-blip LK has completed and the corrections are written into the sparse output dict.

Returns:

A sparse VideoAnnotation with corrections at blip frames. The fname field is set to <source_stem>_blip_corrections.json next to the source; the file is NOT written by this function (call .save() on the returned object if you want it persisted).

dustrack.list_seed_bundles(root: str | Path) → list[dict]

Scan root for valid seed bundles and return their metadata.

Each subdirectory that passes inspect_seed_bundle() is included; invalid folders are silently skipped (the picker modal is allowed to show “no bundles found” rather than surfacing a long list of validation errors).

Returns:

One entry per valid bundle, sorted by folder name. Each entry carries every field inspect_seed_bundle() returns plus name (the bundle folder’s name relative to root) and path (the absolute bundle folder path).

Return type:

list[dict]

dustrack.remove_blips(ann: VideoAnnotation, report: BlipReport, *, drop_frame_if_any_blip: bool = False) → VideoAnnotation

Build a without-blip copy of ann: every blip frame’s entry is dropped, all other frames carry their original (x, y) untouched.

The output is a dense VideoAnnotation (same labels + frame coverage as the source minus the dropped entries) — directly usable as DLC training data via the NaN-tolerant labeled-data path, since DLC simply skips frames with missing per-bodypart values.

Parameters:

• ann – Source dense-trace annotation passed to detect_blips().

• report – Detection result from detect_blips().

• drop_frame_if_any_blip – When False (default), only the blipped label’s entry is removed at each blip frame; other labels at the same frame are preserved. When True, the blip frame is dropped from every label — the strict “if any label was bad at this frame, the frame is suspect for all” policy. Useful when blips on one label tend to correlate with bad image quality (occlusion, motion blur) that compromises every label at the same instant.

Returns:

A new VideoAnnotation whose fname is set to <source_stem>_blip_removed.json next to the source; the file is NOT written by this function (call .save() on the returned object if you want it persisted).

dustrack.lk_moving_average_filter(tracked_points: str | VideoAnnotation, video_name: str = None, window_size: float = 0.5, use_parallel: bool = True, save_raw: bool = True) → VideoAnnotation

Apply Lucas-Kanade moving average filter to smooth tracking annotations.

This is the main post-processing function. It applies RSTC over overlapping time windows and averages the results to produce smooth, jitter-free trajectories.

Workflow:

1. Load video and annotations

2. Slide a time window through the video

3. For each window: - Apply RSTC to get smoothed trajectory - Store results indexed by window position

4. Average all trajectories that cover each frame

5. Save smoothed annotations

Parameters:

• tracked_points (Union[str, VideoAnnotation]) – Either: - Path to annotation JSON file - VideoAnnotation object with tracking data

• video_name (str, optional) – Video path. Defaults to None; required when tracked_points is a file path (asserted at call time). Ignored when tracked_points is already a VideoAnnotation.

• window_size (float, optional) – Time window in seconds for moving average. Larger windows provide more smoothing but increase computation. Typical range: 0.1 to 1.0 seconds. Defaults to 0.5.

• use_parallel (bool, optional) – Use ThreadPoolExecutor for parallel processing. Significantly faster for longer videos. Defaults to True.

• save_raw (bool, optional) – If True (default), allocate the full (n_window_frames, n_total_frames, n_labels, 2) per-window RSTC array and dill-pickle it to {stem}_lkmovavg_{window_size}.pkl alongside the averaged JSON. Per-window data is consumed downstream by pn-projects/wobble and gaitmusic (their .rawlk property → lk_gradients velocity estimation), so keep True for those flows. If False, stream a running (n_total_frames, n_labels, 2) sum and per-cell count instead, divide at the end, and skip the .pkl write. Cuts peak memory from W*N*L*16 bytes to N*L*12 bytes (~10x on typical W=15, L=4 cases). On the example-video bench wall time is essentially unchanged (post-processing is <0.3% of total) but on real-world long videos (N=36715+) the .pkl round-trip alone is a couple of seconds. FP summation order changes between modes so save_raw=False output is numerically close (np.allclose with atol~1e-10) rather than bit-exact vs save_raw=True.

Returns:

New annotation object with smoothed trajectories.

Saved to {original_stem}_lkmovavg_{window_size}.json

Return type:

VideoAnnotation

Output Files:

• {stem}_lkmovavg_{window_size}.pkl: Per-window RSTC results (only when save_raw=True; consumed by .rawlk downstream).

• {stem}_lkmovavg_{window_size}.json: Averaged smoothed annotations (always written).

Performance Tips:

• Use use_parallel=True for videos longer than a few seconds

• Smaller window_size is faster but provides less smoothing

• Results are cached: the .json is the primary cache, and when save_raw=True the .pkl is reused on subsequent calls regardless of the save_raw flag (a pre-existing .pkl is always read if the .json is missing). Delete either to recompute the corresponding stage.

Example

>>> # Post-process manual annotations
>>> ann = VideoAnnotation('video_annotations.json', 'video.mp4')
>>> ann_smooth = lk_moving_average_filter(ann, window_size=0.5)
>>>
>>> # Post-process DLC predictions with larger window
>>> ann_dlc = VideoAnnotation('video_dlc_predictions.h5', 'video.mp4')
>>> ann_smooth = lk_moving_average_filter(ann_dlc, window_size=1.0)
>>>
>>> # Non-parallel processing for short clips
>>> ann_smooth = lk_moving_average_filter(ann, window_size=0.3, use_parallel=False)
>>>
>>> # GUI-style: skip the .pkl sidecar (memory + I/O savings)
>>> ann_smooth = lk_moving_average_filter(ann, save_raw=False)

In practice, the postprocess shortcut on a VideoAnnotation is equivalent: >>> from dustrack import VideoAnnotation >>> ann = VideoAnnotation(‘video_dlc_predictions.h5’, ‘video.mp4’) >>> ann_rstc = ann.postprocess()

Raises:

• AssertionError – If video_name is not provided when tracked_points is a file path.

• AssertionError – If tracked_points is not a VideoAnnotation or valid file path.

Note

The algorithm uses a deque-based frame buffer to minimize memory usage and avoid repeated video decoding. Frames are read once and discarded after the window moves past them.

dustrack.lucas_kanade(video: Video | VideoReader | str | Path, start_frame: int, end_frame: int, init_points: ndarray, mode: str = 'full', **lk_config) → ndarray

Track points in a video using Lucas-Kanade algorithm.

Parameters:

• video (Union[utils.Video, VideoReader, str, Path]) – Video object or path to video file.

• start_frame (int) – Initial frame for tracking.

• end_frame (int) – Final frame (inclusive).

• init_points (np.ndarray) – n_points x 2. Locations to be tracked at start_frame.

• mode (str, optional) – ‘full’ tracks the points at every frame in the entire segment. ‘direct’ tracks the point at the last frame using the first frame. Defaults to ‘full’.

• **lk_config – Additional configuration for Lucas-Kanade algorithm.

Returns:

n_frames x n_points x 2, includes start and end frame for ‘full’,

and 1 x n_points x 2 for ‘direct’.

Return type:

np.ndarray

dustrack.lucas_kanade_rstc(video: Video | VideoReader | str | Path, start_frame: int, end_frame: int, start_points: ndarray, end_points: ndarray, target_frame: int = None, **lk_config) → ndarray

Track points in a video using Lucas-Kanade algorithm, and apply the reverse sigmoid tracking correction (RSTC) as described in Magana-Salgado et al., 2023.

Parameters:

• video (Union[utils.Video, VideoReader, str, Path]) – Video object or path to video file.

• start_frame (int) – Initial frame for tracking.

• end_frame (int) – Final frame (inclusive).

• start_points (np.ndarray) – n_points x 2. Locations to be tracked at start_frame.

• end_points (np.ndarray) – n_points x 2. Locations to be tracked at end_frame.

• target_frame (int, optional) – Target frame for direct mode. Defaults to None.

• **lk_config – Additional configuration for Lucas-Kanade algorithm.

Returns:

Corrected tracking path.

Return type:

np.ndarray

dustrack.open(path=None, layer_name=None, **dustrack_kwargs)

Open a DUSTrack annotation session; auto-resolves single- vs multi-video.

The unified entry point for the DUSTrack workflow. Users hand it a path (or a list of paths) and DUSTrack figures out whether they’re starting fresh on a standalone video (Phase 1), resuming inside a DLC project on one video (Phase 2 single), or queueing every video in a DLC project for in-session swap navigation (Phase 2 multi, 1.2.0a3).

Phase 1 – bare video, no DLC project context.

Equivalent to DUSTrack(path, layer_name, **kwargs). Works without deeplabcut installed – the GUI plus the LK-RSTC post-processing run standalone, which is the “Option 1” install path from the paper. Single-video only; multi-video requires a DLC project (see Phase 2 multi).

Phase 2 single – one video inside a DLC project.

Accepts a video inside a project’s videos/ folder. Resolves the DLCProject and dispatches to DLCProject.annotate() so a fresh DUSTrack opens with all existing annotation layers, DLC trace overlays, and a new iteration layer wired up.

Phase 2 multi – the whole project (or a subset). (1.2.0a3)

Three entry shapes:

• Project folder: dustrack.open('S:/path/to/project/') queues every video in project.config['video_sets'] (in project order). Behavior change vs <=1.2.0a2 (which opened only the first video).

• DLC config.yaml: dustrack.open('.../config.yaml') queues every video in project.config['video_sets'], same as the folder form. Behavior change vs <=1.2.0a3-pre (which opened video 0 only). DLCProject.__init__ runs rebase_to_config() on each video_sets key, so stale paths after a project-folder rename self-heal here.

• List of videos: dustrack.open([v0, v1, ...]) queues exactly those videos, in the given order. Every entry must resolve to the same DLC project; mismatches and bare-video entries raise. The list-form is the way to override the YAML-stored order (e.g. Ctrl+click in the file dialog).

The active session opens on the first queued video; the rest are background-hydrated and reachable via the sidebar’s arrow nav row (or Alt+Left / Alt+Right). Per-video state (active layer, overlay, frame, frozen axes, image-pane zoom, unsaved edits) persists across swaps – swap-back returns to the exact visual state the user left.

The two-phase split mirrors DUSTrack’s deliberate copy-on-project- creation design: once a DLC project exists, the project folder is the workspace and the original video becomes a frozen “rewind point” (delete the folder to start over). open() honors that boundary – pointing at the original video gives you Phase 1, pointing at the in-project copy gives you Phase 2.

Parameters:

• path – Video file, config.yaml, DLC project folder, a sequence of videos inside one DLC project, or None – None pops a Qt file picker and lets the user pick one or more videos.

• layer_name – Annotation layer name. Optional in both phases: Phase 1 defaults to 'iteration-0' (the canonical seed name for the rest of the DLC pipeline – the next DLC training iteration lands as iteration-1); Phase 2 defaults to iteration-{N+1} (the next-iteration suffix derived from the project’s training history). Callers can still pass an explicit name to override. Ignored for the project-folder multi-video form.

• **dustrack_kwargs – Forwarded to the underlying DUSTrack constructor (dark_mode, fast_render, clahe_clip, gamma, brightness, etc.).

Returns:

Live annotation UI, ready to use. None if the no-arg form’s file picker was cancelled.

Return type:

DUSTrack

Raises:

• FileNotFoundError – If path doesn’t exist (or any entry in a list form is missing).

• ValueError – Path is a directory that isn’t a DLC project, an empty sequence was supplied, a multi-video list mixes DLC projects, or a multi-video list includes any Phase 1 (bare-video) entry.

• ImportError – Phase 2 entry on a system without deeplabcut installed.

Examples

Zero-argument launch (pops a video picker):

import dustrack
tracker = dustrack.open()

Fresh annotation (default layer name 'iteration-0'):

tracker = dustrack.open('video.mp4')

Multi-video launch from a DLC project folder (queues every video in the project):

tracker = dustrack.open('S:/path/to/project/')

Multi-video launch from a subset of project videos:

tracker = dustrack.open(['S:/proj/videos/v0.mp4',
                         'S:/proj/videos/v3.mp4'])

Resume after closing the UI mid-workflow (any of these work as single-video entries):

tracker = dustrack.open('S:/path/to/project/videos/video.mp4')
tracker = dustrack.open('S:/path/to/project/config.yaml')

With UI options:

tracker = dustrack.open('video.mp4', 'manual', dark_mode=True)

Telemed ultrasound recordings (.tvd.h5 sidecar) – let the Log helper plan + encode the per-panel mp4 on demand, then hand the path here:

from immersionlab import telemed
lf = telemed.Log('M:/path/to/scan.tvd.h5')
tracker = dustrack.open(lf.ensure_mp4(panel=2))

dustrack.set_seed_bundles_root(path: str | Path | None) → None

Persist the seed-bundles root path so the Create-DLC-Project seeding modal can populate its bundle list without forcing the user to navigate from scratch each time. Pass None to forget the current value.

Entry point

The recommended way to start a DUSTrack session. Hand it a path and it figures out whether you’re starting fresh on a bare video or resuming inside a DLC project, dispatching to :class:~dustrack.gui.DUSTrack or :meth:~dustrack.dlcinterface.DLCProject.annotate accordingly. Direct construction of :class:~dustrack.gui.DUSTrack still works for advanced use, but dustrack.open(...) is the documented surface.

dustrack._open.open(path=None, layer_name=None, **dustrack_kwargs)

Open a DUSTrack annotation session; auto-resolves single- vs multi-video.

The unified entry point for the DUSTrack workflow. Users hand it a path (or a list of paths) and DUSTrack figures out whether they’re starting fresh on a standalone video (Phase 1), resuming inside a DLC project on one video (Phase 2 single), or queueing every video in a DLC project for in-session swap navigation (Phase 2 multi, 1.2.0a3).

Phase 1 – bare video, no DLC project context.

Equivalent to DUSTrack(path, layer_name, **kwargs). Works without deeplabcut installed – the GUI plus the LK-RSTC post-processing run standalone, which is the “Option 1” install path from the paper. Single-video only; multi-video requires a DLC project (see Phase 2 multi).

Phase 2 single – one video inside a DLC project.

Accepts a video inside a project’s videos/ folder. Resolves the DLCProject and dispatches to DLCProject.annotate() so a fresh DUSTrack opens with all existing annotation layers, DLC trace overlays, and a new iteration layer wired up.

Phase 2 multi – the whole project (or a subset). (1.2.0a3)

Three entry shapes:

• Project folder: dustrack.open('S:/path/to/project/') queues every video in project.config['video_sets'] (in project order). Behavior change vs <=1.2.0a2 (which opened only the first video).

• DLC config.yaml: dustrack.open('.../config.yaml') queues every video in project.config['video_sets'], same as the folder form. Behavior change vs <=1.2.0a3-pre (which opened video 0 only). DLCProject.__init__ runs rebase_to_config() on each video_sets key, so stale paths after a project-folder rename self-heal here.

• List of videos: dustrack.open([v0, v1, ...]) queues exactly those videos, in the given order. Every entry must resolve to the same DLC project; mismatches and bare-video entries raise. The list-form is the way to override the YAML-stored order (e.g. Ctrl+click in the file dialog).

The active session opens on the first queued video; the rest are background-hydrated and reachable via the sidebar’s arrow nav row (or Alt+Left / Alt+Right). Per-video state (active layer, overlay, frame, frozen axes, image-pane zoom, unsaved edits) persists across swaps – swap-back returns to the exact visual state the user left.

The two-phase split mirrors DUSTrack’s deliberate copy-on-project- creation design: once a DLC project exists, the project folder is the workspace and the original video becomes a frozen “rewind point” (delete the folder to start over). open() honors that boundary – pointing at the original video gives you Phase 1, pointing at the in-project copy gives you Phase 2.

Parameters:

• path – Video file, config.yaml, DLC project folder, a sequence of videos inside one DLC project, or None – None pops a Qt file picker and lets the user pick one or more videos.

• layer_name – Annotation layer name. Optional in both phases: Phase 1 defaults to 'iteration-0' (the canonical seed name for the rest of the DLC pipeline – the next DLC training iteration lands as iteration-1); Phase 2 defaults to iteration-{N+1} (the next-iteration suffix derived from the project’s training history). Callers can still pass an explicit name to override. Ignored for the project-folder multi-video form.

• **dustrack_kwargs – Forwarded to the underlying DUSTrack constructor (dark_mode, fast_render, clahe_clip, gamma, brightness, etc.).

Returns:

Live annotation UI, ready to use. None if the no-arg form’s file picker was cancelled.

Return type:

DUSTrack

Raises:

• FileNotFoundError – If path doesn’t exist (or any entry in a list form is missing).

• ValueError – Path is a directory that isn’t a DLC project, an empty sequence was supplied, a multi-video list mixes DLC projects, or a multi-video list includes any Phase 1 (bare-video) entry.

• ImportError – Phase 2 entry on a system without deeplabcut installed.

Examples

Zero-argument launch (pops a video picker):

import dustrack
tracker = dustrack.open()

Fresh annotation (default layer name 'iteration-0'):

tracker = dustrack.open('video.mp4')

Multi-video launch from a DLC project folder (queues every video in the project):

tracker = dustrack.open('S:/path/to/project/')

Multi-video launch from a subset of project videos:

tracker = dustrack.open(['S:/proj/videos/v0.mp4',
                         'S:/proj/videos/v3.mp4'])

Resume after closing the UI mid-workflow (any of these work as single-video entries):

tracker = dustrack.open('S:/path/to/project/videos/video.mp4')
tracker = dustrack.open('S:/path/to/project/config.yaml')

With UI options:

tracker = dustrack.open('video.mp4', 'manual', dark_mode=True)

Telemed ultrasound recordings (.tvd.h5 sidecar) – let the Log helper plan + encode the per-panel mp4 on demand, then hand the path here:

from immersionlab import telemed
lf = telemed.Log('M:/path/to/scan.tvd.h5')
tracker = dustrack.open(lf.ensure_mp4(panel=2))

DUSTrack GUI

class dustrack.gui.DUSTrack(vid_name, annotation_names='iteration-0', *args, clahe_clip=1.0, clahe_grid=8, gamma=1.0, brightness=0, dark_mode=False, n_labels: int = 1, titlefunc: Callable | None = None, height_ratios: tuple = (10, 1, 1), fast_render: bool = True, **kwargs)

Bases: VideoBrowser

Interactive video point annotator with DeepLabCut integration.

DUSTrack provides a GUI for manual annotation of points in videos, with integrated support for creating DeepLabCut projects, training models, and post-processing results using optical flow algorithms.

Features:

• Manual point annotation with event marking

• Real-time trajectory visualization

• DeepLabCut project creation and training

• Optical flow-based jitter reduction

• Multiple annotation layer management

Variables:

• _dlcproject (DLCProject) – Associated DeepLabCut project instance.

• _ax_lims (dict) – Stores axis limits when plot axes are frozen.

Example

>>> # Most users should call :func:`dustrack.open` instead of
>>> # constructing DUSTrack directly -- it dispatches to either
>>> # this class (bare video) or :meth:`DLCProject.annotate`
>>> # (resume in a DLC project) based on the path you pass.
>>> # Direct construction below is for advanced / scripted use.
>>>
>>> # Basic usage -- annotation_names defaults to "iteration-0",
>>> # the canonical seed name for the DLC pipeline (the next
>>> # DLC training round lands as iteration-1).
>>> tracker = DUSTrack('video.mp4')
>>>
>>> # Explicit layer name -- saved as {video_name}_annotations_pn.json
>>> tracker = DUSTrack('video.mp4', "pn")
>>>
>>> # With multiple annotation layers
>>> tracker = DUSTrack('video.mp4', {
...     'manual': 'manual_labels.json',
...     'dlc_iter1': 'dlc_predictions.h5'
... })

CORRECTIONS_LAYER_NAME = 'dlccorr'

PREDICT_LAYER_NAME = 'dlc_predict'

__init__(vid_name, annotation_names='iteration-0', *args, clahe_clip=1.0, clahe_grid=8, gamma=1.0, brightness=0, dark_mode=False, n_labels: int = 1, titlefunc: Callable | None = None, height_ratios: tuple = (10, 1, 1), fast_render: bool = True, **kwargs)

Parameters:

• vid_name (str) – Path to the video file.

• titlefunc (Optional[Callable]) – Function to generate the title for the plot.

• figure_or_ax_handle (Optional[Union[plt.Axes, plt.Figure]]) – Handle to the figure or axis.

• image_process_func (Callable) – Function to process the image.

• fast_render (bool) – If True, render the video frame and any annotation scatter overlays Qt-native (QGraphicsView + QPixmapItem) instead of through matplotlib’s imshow. Bypasses the canvas pixmap-upload cost that dominates per-frame budget on QtAgg (see BENCHMARKING.md probe 13 / 14 for the architecture rationale). Requires a Qt matplotlib backend; silently falls back to Tier 1 if no Qt window is found. In fast_render mode, user additions to _ax_image via mpl plotting calls are NOT supported (the image region is no longer an mpl Axes); _ax and _im are set to None.

discard_unsaved_annotations(event=None)

Drop in-memory edits on the active layer and reload from disk.

Inverse of save: confirms via ConfirmOverlay, then calls VideoAnnotation.reload() on self.ann and triggers self.update() so the trace pane + scatter re-render. The confirm body branches on whether the layer’s backing file exists:

• File exists -> “Reload from disk” semantic.

• File missing -> “Reset to empty” semantic.

Refuses on the dlccorr splice and any layer matching _is_dense_layer_name() – those are regenerated from the active + overlay layers (via apply_manual_corrections or process_with_lk), not authored by hand, so “discard and reload” has no meaningful semantic. Points the user at Remove layer instead.

freeze_plot_axes(event=None)

Lock the axis limits of trajectory plots to current view.

Useful for comparing tracking quality across different frames without automatic axis rescaling distracting from the comparison.

Parameters:

event – Mouse/keyboard event (unused, for button compatibility).

unfreeze_plot_axes(event=None)

Restore automatic axis scaling for trajectory plots.

Parameters:

event – Mouse/keyboard event (unused, for button compatibility).

create_dlc_project(event=None, name=None, path=None, experimenter='x', seed_bundle_path=None, link_videos: bool | None = None) → DLCProject

Create a new DeepLabCut project using current annotations as training labels.

rc2 (1.1.0rc2): on a Qt backend, project creation runs off the GUI thread under a modal overlay (no progress bar – it’s a fast op, but the overlay surfaces DLC’s stdout and a Done button so the user can confirm the project location and any warnings before continuing). On non-Qt backends the call runs synchronously and returns the new DLCProject.

1.2.0a2: if the active manual layer is empty when the user clicks Create DLC Project on the Qt path, a multi-step Seed-from-bundle modal opens (_prompt_seed_bundle()). Picking a valid bundle routes through a seeding flow that: (a) creates the project as usual, (b) calls import_seed_bundle_into_project() to install the bundle’s snapshot as iteration-0’s trained model (overwriting the project’s bodyparts with the bundle’s), and (c) runs analyze_videos(iteration_num=0) to produce a dense reference layer the user can refine into iteration-1. seed_bundle_path may also be passed programmatically to bypass the modal.

Parameters:

• event – Mouse/keyboard event (unused, for button compatibility).

• name (str, optional) – Project name. Defaults to “{video_name}_{annotation_layer}”.

• path (str, optional) – Directory for project. Defaults to video’s parent directory.

• experimenter (str, optional) – Experimenter name. Defaults to config value.

• seed_bundle_path (str, optional) – Path to a seed bundle folder (as produced by extract_snapshot_for_seeding()). When supplied, the seeding flow runs unconditionally (Qt or non-Qt), bypassing the empty-layer-triggered modal.

• link_videos – Forwarded to DLCProject to control how the source video is placed inside <project>/videos/. None (default) hard-links on same-volume sources and falls back to copy on cross-volume. False forces a deep copy (the pre-1.3.0a2 behavior).

Returns:

The newly created project instance on the sync path. None on the Qt async path – read self._dlcproject after the Done button is clicked.

Return type:

DLCProject

Note

Project names must contain an underscore for proper DLC configuration handling.

process_dlc_project(event=None, *args, **kwargs)

Train the DeepLabCut model without leaving the annotation UI.

rc2 (1.1.0rc2): training runs off the GUI thread, the DUSTrack window stays open under a “Training in progress” overlay (progress bar + scrolling stdout tail), and on success the new DLC prediction layers are added to the live session via add_annotation_layers() – no relaunch. The overlay transitions to a “Complete” / “Failed” state with a Done button so the user can review the final stdout before the predictions swap in (or read the error before retrying); failure paths fold into the overlay rather than popping a separate QMessageBox.

1.2.0a2: a Training options modal runs before everything else (Qt path only). It surfaces DLCProject.train_iteration()’s arg surface in the UI: refine_mode (scratch / in-project / external), source iteration/snapshot picker for in-project, Browse… for external .pt, training epochs (DLC3) / iterations (DLC2), and a create-labeled-video toggle. Cancel returns to the UI without kicking off training. On accept, the underlying call routes through DLCProject.train_iteration() (the explicit-args sibling of DLCProject.process()) – so the DLC2 silent-drop bug on refine=<str> is gone, and external snapshots work on both DLC versions (DLC3 via train_network(snapshot_path=...), DLC2 via a pose_cfg init_weights edit).

A unified pre-flight runs before the overlay starts, scanning every manual annotation layer in the session (file-pattern detection – .json alongside the video, <video_stem>_annotations*.json, minus dlccorr / buffer / dlc*; agnostic to which layer is active, the overlay, or a placeholder, and to whether the user renamed iteration-N to something else). For each manual layer we report (a) in-memory diffs vs the on-disk JSON and (b) frames missing one or more bodyparts. If any layer has either kind of issue, a single modal lists the per-layer breakdown and offers two actions: Save and clean (write per-layer recovery sidecars for the dropped frames, drop the incomplete frames from every affected layer, then save every affected layer, then train) and Cancel (return to the UI). Layers without issues are not touched; DLC trace layers and dlccorr are not in scope. Recovery sidecars are written as <fstem>.dustrack-dropped-incomplete-<YYYYMMDDTHHMMSS>; the composite extension avoids .json so the annotation-discovery glob does not re-ingest them on subsequent training runs.

DLC’s stdout/stderr are also teed to the launching terminal (sys.__stdout__) so callers who launched from a shell can watch progress there as before.

On non-Qt backends (or if the QMainWindow can’t be located), the method falls back to the pre-rc2 behavior: close the figure, run training synchronously on the calling thread, and return a fresh DUSTrack via DLCProject.annotate(). The pre-flight modal is skipped on this path (no GUI to host it).

Parameters:

• event – Mouse/keyboard event (unused, for button compatibility).

• *args – Forwarded to DLCProject.process() on the non-Qt fallback path; ignored on the Qt path (the Training options modal owns the kwarg surface there).

• **kwargs – Same – non-Qt fallback only. create_video defaults to False on the non-Qt path (vs. True for direct DLCProject.process() calls) so the annotate -> train -> review -> annotate loop doesn’t write a labeled mp4 each pass; the Qt path’s create-labeled-video checkbox owns this on its own.

Returns:

self on the Qt path (training is asynchronous; the same DUSTrack will refresh in place when the user clicks Done). Also self if the user cancels either the Training options modal or the pre-flight modal – the UI is left intact. On the fallback path, the freshly-launched DUSTrack from DLCProject.annotate().

Return type:

DUSTrack

Raises:

• ImportError – If deeplabcut isn’t installed.

• ValueError – If no DLCProject has been created yet.

process_with_lk(event=None, *args, **kwargs)

Apply Lucas-Kanade optical flow post-processing to reduce tracking jitter.

rc2 (1.1.0rc2): on a Qt backend, the LK-RSTC pass runs off the GUI thread under a progress overlay that mirrors the Train DLC flow (tqdm bars drive the progress widget; phase label reflects “Submitting” vs “Processing”; Done button lets the user confirm before the smoothed layer swaps in). On non-Qt backends the call runs synchronously and returns the new VideoAnnotation.

Uses the Lucas-Kanade RSTC (Reverse Sigmoid Tracking Correction) algorithm to smooth trajectories. The processed annotation is saved and added as a new layer, with the original set as overlay for comparison.

rc2 layer-naming harmonisation: the new layer is adopted via _adopt_layer(), which derives its name from the output filepath via VideoFileManager.canonical_layer_name() – identical to what a close + reopen would show. Replaces the earlier behaviour where the in-session layer briefly carried the "noname" fallback until reload. When the source layer is a DLC trace, the smoothed output is also plot-type-normalised to "line" so it looks identical to other dlc_* layers.

The source layer is save()-ed to disk right before LK kicks off (mirroring the pre-train save in process_dlc_project()), so the on-disk state matches what LK sees. In the typical workflow the source is the dlccorr layer (active after apply_manual_corrections()) and the save persists any in-memory manual edits. Sources without a .json filename (e.g. raw DLC traces loaded from .h5) are read-only inputs; the save is skipped for them with a one-line note.

Parameters:

• event – Mouse/keyboard event (unused, for button compatibility).

• *args – Additional arguments passed to lk_moving_average_filter.

• **kwargs – Additional keyword arguments (e.g., window_size) passed to filter.

Returns:

The smoothed annotation layer on the sync path. None on the Qt async path – the new layer is added to self.annotations when the Done button is clicked.

Return type:

VideoAnnotation

See also

dustrack.lk_filter.lk_moving_average_filter: The filtering algorithm.

detect_blips_workflow(event=None)

Detect blip outliers on the active layer + remove them.

Two-stage flow that mirrors process_with_lk()’s Qt-vs-mpl dispatch:

1. On Qt: pop the BlipOptionsDialog modal (knob tuning + in-modal Detect button + results pane + drop-frame checkbox). Cancel returns; on Remove blips, save a without-blip copy of the source layer via dustrack.blip.remove_blips() and adopt it.

2. On mpl-fallback: run detect + remove synchronously with module defaults; print a one-line summary.

Either way, the without-blip layer adopts via _adopt_layer() with the source layer pinned as overlay (mirrors Reduce jitter’s adoption shape so the user sees the source DLC trace + cleaned version side-by-side immediately).

The LK-interpolation alternative (per-blip RSTC re-track, producing a sparse corrections layer) shipped first but turned out to be less useful in the pia02 workflow than just dropping the contaminating frames – the model trains on a cleaner subset rather than on synthesized positions. The LK function dustrack.blip.interpolate_blips() stays available for headless callers who explicitly want it.

remove_current_layer(event=None)

Remove the active annotation layer from the DUSTrack session.

Session-only: the underlying JSON / HDF5 file on disk is not touched – so on next launch the layer reappears if its file is still next to the video. Pair with a manual file delete (or Save annotation as... to a different name + delete the original) when the intent is “undo manual corrections”.

Refuses with a notice if only one removable layer remains in the session (excluding the implicit "buffer" layer, which is never user-visible but always present). Otherwise confirms via ConfirmOverlay; the confirm body is severity- aware via _is_dense_layer_name():

• Dense / derived (dlc_*, dlccorr, *lkmovavg*): regenerable, default button = Remove.

• Sparse / authored (manual labels): irreversible, default button = Cancel.

copy_existing_annotations_from_overlay(event=None)

Copy overlay annotation points to the current annotation layer for selected frames.

Useful when DLC predictions are more accurate than manual labels. Typically used with manual annotations in the primary annotation layer, and the model predictions in the overlay layer. Data is copied only for frames that exist in the primary annotation layer. Perform this action within a specified frame range by selecting an interval.

Parameters:

event – Mouse/keyboard event (unused, for button compatibility).

Raises:

ValueError – If no annotation overlay is currently selected.

Note

Only affects the current label. Other labels remain unchanged.

MANUAL_CORRECTIONS_SUFFIX = '_manual_corrections'

apply_manual_corrections(event=None)

Splice the active layer’s manual entries into the overlay to produce/refresh the dlccorr layer.

Workflow context (step 4 of the DUSTrack pipeline): after iterating DLC training (steps 2-3), you flip into a manual-correction mode. The active annotation layer holds sparse hand-edits (a few frames where DLC was wrong); the annotation overlay points at the DLC trace you want to correct. Clicking this button produces a layer named CORRECTIONS_LAYER_NAME ("dlccorr") that is the overlay’s data with your manual entries spliced in wherever they exist. The file lives next to the video as <video>_annotations_dlccorr.json and is automatically excluded from DLC training input (the _dlccorr filter in _extract_frames()) – this is a terminal output.

Preflight save. If the active (patch) layer has any unsaved in-memory edits, they are written to disk before the splice runs. The user has explicitly clicked Apply, so “intent to commit” is signalled; saving the source as a side effect keeps the on-disk state coherent with dlccorr (which is always saved by this method).

Source-layer rename. On a successful splice the patch layer is renamed in place from <old_name> to <old_name>_manual_corrections (the file is moved on disk too, old file deleted). The rename marks “this is the layer whose manual entries the on-disk dlccorr was spliced from” so the relationship survives reload. No-op if the patch is already named *_manual_corrections (idempotent re-apply).

Post-apply state. The corrections layer becomes the active annotation layer and the (now-renamed) manual layer becomes the overlay so you can see where your hand was. To iterate, switch the active layer back to your manual layer, set the overlay back to the DLC trace, add more points, click again. Each click regenerates the corrections layer from the current (overlay, active) pair, so adding annotations directly to the corrections layer is not recommended – they’ll be discarded on the next apply.

Idempotency. If a dlccorr layer is already present in the session, its in-memory data is replaced wholesale (with a _revision bump so the trace cache invalidates) and the file overwritten. Otherwise a fresh VideoAnnotation is built, saved, and adopted under the canonical machinery.

Parameters:

event – Mouse/keyboard event (unused, for button compat).

Raises:

ValueError – If no annotation overlay is currently set, or if the active annotation layer is already the corrections layer (would create a circular splice).

save_annotation_as(event=None)

Save the active annotation layer to a user-chosen path.

Opens a Qt save-file dialog seeded with the video’s folder and a suggested filename of <video_stem>_annotations_<layer>.json. Falls back to self.ann.save() (which writes to the layer’s existing .fname) when no Qt window is available – e.g. headless / Agg backend.

swap_active_and_overlay(event=None)

Swap the active annotation layer with the overlay layer.

No-op if no overlay is currently selected.

update()

Update the display with current frame and maintain frozen axis limits if set.

Returns:

Result from parent class update() method.

swap_to(index: int) → bool

Switch the active video to self._bundles[index]. See _bundle_swap.swap_to().

swap_prev(event=None) → bool

Move to the previous bundle (no-op at index 0).

Connected to the sidebar’s ◀ button and the Alt+Left keybinding. Returns the underlying swap_to() result so keybinding-handler callers can short-circuit if desired.

swap_next(event=None) → bool

Move to the next bundle (no-op at last index).

Connected to the sidebar’s ▶ button and the Alt+Right keybinding.

add_video(path_or_paths, *, layer_name=None, set_active=False, **dustrack_kwargs) → list[int]

Append one or more videos to this tracker’s bundle list. See _bundle_swap.add_video().

remove_video(index: int) → bool

Drop a bundle from the tracker’s bundle list. See _bundle_swap.remove_video().

replace_active_with(path_or_paths, *, layer_name=None, **dustrack_kwargs) → list[int]

Swap the active bundle for newly-picked video(s). See _bundle_swap.replace_active_with().

classmethod from_annotations(annotations: list[dustrack.annotations.VideoAnnotation], *args, **kwargs) → DUSTrack

add_annotation_layers(annotation_names: list[str] | dict[str, pathlib.Path] | list[dustrack.annotations.VideoAnnotation], n_labels: int = 1) → None

Load data from annotation files if they exist, otherwise initialize annotation layers.

remove_annotation_layer(name: str) → None

Remove an annotation layer from the active session.

Tears down the layer’s plot artists (scatter + per-label trace lines on x/y trace axes) via VideoAnnotation.clear_display(), drops the layer from annotations via AssetContainer.remove(), then resyncs the annotation_layer / annotation_overlay statevariables through _refresh_annotation_state_lists() so the dropdown rotations + current selections stay valid.

Pre-flight: - name must be an existing layer name. - Refuses if it would leave the container empty (callers needing

a “reset to single empty layer” semantic should use VideoAnnotation.reload() on the surviving layer instead).

Active-layer handoff: if name is currently the primary layer (annotation_layer.current_state), the previous layer in the rotation becomes the new primary (or the first one if the removed layer was at index 0). If name is currently the overlay, the overlay clears to None.

Note: "buffer" is not excluded here – the dnav layer treats every named layer the same. Consumers (DUSTrack) enforce buffer-exclusion at the UI layer.

set_key_bindings() → None

Set the keyboard actions.

Groups mirror the 5-step workflow in docs/source/resources/ keyboard_shortcuts.png: layer selection -> label selection -> frame navigation -> edit -> refine. self._section_order pins the cheatsheet’s section order to the workflow order (section 3 first, then 1, 2, 4, 5a, 5b, 5c) regardless of when each binding was originally registered. Bindings not on the PNG (save, refresh, reset view, pan, keep-overlapping, toggle-num-keys mode) fall through to the “Other” section.

refresh(event: Any | None = None) → None

Force-refresh the UI from the current annotation .data.

Drops the trace-display cache on every annotation layer and the frame-marker cache, then calls update() so the next draw re-reads every value from the backing dicts.

Recovery path for the rare case where .data was mutated directly (e.g. from an IPython prompt: v.ann.data["0"][42] = [x, y]) bypassing the public add() / remove() / add_at_frame() API and therefore skipping the _revision bump that VideoAnnotation.update_display_trace() and update_frame_marker() cache on. In normal in-code mutation paths the public API bumps _revision and the caches invalidate automatically; calling refresh() is always safe but rarely necessary.

add_events() → None

Add an event to specify time intervals for interpolating with lucas-kanade.

property ann: VideoAnnotation

Return current annotation layer.

set_image_background_color(color: Any) → None

Set the background color of the image region.

Tier 1 routes to self._ax_image.set_facecolor (matplotlib); Tier 2 routes to the Qt image pane’s set_background_color. The single-method surface lets consumers (DUSTrack’s _apply_dark_theme) issue one call regardless of which tier is active.

update_annotation_visibility(draw: bool = False) → None

Update the visibility of all annotation layers, for example, when the layer is changed.

update_frame_marker(draw: bool = False) → None

Update the current frame location in the trace plots.

The trace ylim / FOI tick positions are a pure function of the active label + per-annotation contents + frames_of_interest, none of which change when only _current_idx moves. Cache them keyed on a cheap tuple of (label, annotation revisions, frames_of_interest) so per-frame navigation only does the frame-marker set_data calls.

copy_annotations_from_overlay() → None

Copy annotations from the overlay layer into the current layer in the current frame.

copy_current_annotation_from_overlay() → None

Copy annotations from the overlay layer into the current layer.

copy_frames_of_interest_from_overlay() → None

copy annotations at frames of interest from buffer into the current layer. If there is no buffer, then copy from the overlay layer.

copy_frames_in_interval_from_overlay() → None

For the current label only.

add_annotation(event: Any) → None

Add annotation at frame. If it exists, it’ll get overwritten.

remove_annotation(event: Any | None = None) → None

remove annotation at the current frame if it exists

previous_frame_with_current_label(event: Any | None = None) → None

Go to the previous frame with the current label in the current annotation layer.

next_frame_with_current_label(event: Any | None = None) → None

Go to the next frame with the current label in the current annotation layer.

previous_frame_with_any_label(event: Any | None = None) → None

Go to the previous frame with any label in the current annotation layer.

next_frame_with_any_label(event: Any | None = None) → None

Go to the next frame with any label in the current annotation layer.

previous_frame_of_interest(event: Any | None = None) → None

Go to the previous frame of interest.

next_frame_of_interest(event: Any | None = None) → None

previous_annotation_layer() → None

Go to the previous annotation layer.

next_annotation_layer() → None

Go to the next annotation layer

previous_annotation_overlay() → None

Go to the previous annotation overlay layer.

next_annotation_overlay() → None

Go to the next annotation overlay layer.

previous_annotation_label() → None

Set current annotation label to the previous one.

next_annotation_label() → None

Set current annotation label to the next one.

update_annotation_label_states() → None

Update the states of the annotation label state variable.

cycle_number_keys_behavior() → None

Number keys can be used to either select labels, or place a specific label. Toggle between these two behaviors.

increment_label_range() → None

Increment the label range by 10.

decrement_label_range() → None

Increment the label range by 10.

increment_if_unannotated(event: Any | None = None) → None

Advance the frame if the current frame doesn’t have any annotations. Useful to pause at annotated frames when adding a new label.

decrement_if_unannotated(event: Any | None = None) → None

Go to the previous frame if the current frame doesn’t have any annotations. Useful to pause at annotated frames when adding a new label.

save() → None

Save current annotation layer json file.

.h5 layers (DLC predicted traces, labeled_data blocks) are not in-place editable through this key — the JSON-only contract on VideoAnnotation.save() would raise mid-keypress and look like a silent no-op. Short-circuit with a clear printed message that points the user to manual layers or to Save annotation as... for explicit copy-out.

select_label_with_mouse(event: Any) → None

Select a label by clicking on it with the left mousebutton.

place_label_with_mouse(event: Any) → None

Place the selected label with the right mousebutton.

go_to_frame(event: Any) → None

toggle_frame_of_interest(event: Any) → None

Mark/unmark the current frame as a frame of interest

keep_overlapping_continuous_frames() → None

keep_overlapping_frames() → None

predict_labels_with_lucas_kanade(labels: str | list[str] = 'all', start_frame: int | None = None, mode: str = 'full') → Any

Compute the location of labels at the current frame using Lucas-Kanade algorithm.

get_selected_interval() → tuple[int, int]

remove_labels_in_interval(all_labels: bool = False) → None

decimate_annotations_in_interval() → None

Prune incomplete frames, then drop every other complete frame in the selected interval.

Prep step for training: ensures the surviving in-interval frames are (a) fully annotated across the required-label set and (b) half as dense (even-stride sampling). Frame-level – every label is removed at each dropped frame.

Required-label set follows the same project-aware / project-unaware split the Train preflight uses (_preflight.scan_incomplete_frames()):

• DLC project loaded – required = project bodyparts (mapped through _dlc_bodyparts_to_layer_labels()). Stray non-bodypart labels on the layer are ignored from the required check, but any frame they sit on is still part of all_frames – if it’s missing a required label it still counts as incomplete, and pruning cleans the stray with it.

• No DLC project – required = labels with at least one annotation. Empty labels are treated as UI placeholders (user created a slot and abandoned it). This is the best inference available without external truth and matches the project-unaware Train preflight rule.

Starter form of the “general-model workflow” decimation feature; the DINOv3-feature farthest-point-sampling variant is deferred. Incomplete frames in the interval are always pruned. The every-other halving is a no-op when fewer than 2 complete frames remain in the interval after pruning.

interpolate_with_lk(all_labels: bool = False) → None

Interpolate with lk-rstc between selected interval. Use data in the overlay layer as start and end points. Add interpolated points to the current layer.

predict_range_with_dlc() → None

Run DLC inference over the selected interval (z z e).

The fast half of the refine loop: rather than a whole-video analyze_videos pass, this asks the current model what it predicts right here, in the interval just marked with z. Results land in the PREDICT_LAYER_NAME overlay so they render against the primary layer instead of overwriting it – the point is to see what changed, which is lost if the prediction clobbers the trace being compared.

Synchronous, like its a / x siblings in this group.

interpolate_with_lk_norstc(all_labels: bool = False) → None

Infer the motion of the current label using lucas-kanade algorithm in the selected interval.

check_labels_with_lk(mode: str = 'minimal') → None

Interpolate between all labeled frames. This only makes sense for sparse-labeled annotations. Use this when doing first-time annotations (as opposed to refinement).

I am testing if refining the start labels using this strategy, and augmenting the training data will improve deeplabcut tracking!

Parameters:

mode (str, optional) – “all” - LK-interpolation for all labels across all labeled frames “current” - LK-interpolation for current label across all labeled frames “minimal” - LK-interpolation for current label between labeled frames near the current frame (two previous to two next) Defaults to “minimal”.

render(start_frame: int, end_frame: int, out_vid_name: str | None = None) → None

Render the video with annotations.

Deep learning

Interface class for using ResNets via DeepLabCut.

class dustrack.dlcinterface.DLCProject(path, videos=[], name='test_01', experimenter='x', annotation_suffix='', internal_to_dlc_labels: dict = None, link_videos: bool | None = None)

Bases: object

Interface to deeplabcut training and inference Current workflow:

1. Create a project with some videos. Videos will be copied.

   d = DLCProject(r’C:/data_opr02/004_02/ml_models/dlc’, name=’opr02_s004_muscles’, experimenter=’praneeth’, videos=[<video_list>])

2. Launch the initial annnotator for video 0, repeat if there are more videos

   d.annotate(0)

3. Extract frames, train network, evaluate network, analyze videos, and create labeled video

   d.process()

4. Refine the labels

   d.annotate(0, ‘praneeth_2’) # the second argument determines the suffix for the annotations file. CAUTION: Make sure that the files are read by extract_frames in the correct order! Pay attention to the output of this method.

5. Re-train network with refined labels

   d.process()

Repeat steps 4 and 5 until satisfied with the results.

__init__(path, videos=[], name='test_01', experimenter='x', annotation_suffix='', internal_to_dlc_labels: dict = None, link_videos: bool | None = None)

Initialize or load a DeepLabCut project.

If a config.yaml exists at the path, loads the existing project. Otherwise, creates a new project with the provided videos.

Parameters:

• path (str) – Directory containing or for the project.

• videos (list) – List of video file paths to include.

• name (str) – Project name (must contain underscore for proper config handling).

• experimenter (str) – Experimenter identifier.

• annotation_suffix (str) – Suffix for annotation files (e.g., ‘manual’, ‘refined’).

• internal_to_dlc_labels (dict, optional) – Custom label name mapping.

• link_videos – How to place each source video inside <project>/videos/. None (default) hard-links via os.link() on the same volume and falls back to a regular copy with a stderr warning on cross-volume. True requires hard-linking and raises OSError on failure. False always copies (the pre-1.3.0a2 behavior).

Note

Videos are hard-linked into the project folder by default (1.3.0a2+), saving disk space when the source and the project share a volume. Pre-1.3.0a2 behavior (deep copy) is recoverable via link_videos=False. Project names without underscores may cause config issues with network paths.

property paths: Mapping[str, Path]

Full paths to project folder and standard DLC subfolders.

Returns:

Mapping of folder names to Path objects with keys:

• ’project’: Main project directory

• ’models’: Trained model weights (dlc-models or dlc-models-pytorch)

• ’results’: Evaluation results

• ’labels’: Labeled frame data

• ’training_data’: Training datasets

• ’videos’: Video files

Return type:

dict

property config: dict

Current project configuration dictionary.

Returns:

Parsed contents of config.yaml.

Return type:

dict

property name: str

Project name from configuration.

property trackers: list

Names of tracked body parts as used internally by DLC.

Returns:

Body part names (e.g., [‘point0’, ‘point1’]).

Return type:

list

property label_names: list

Human-readable names for tracked points.

Returns meaningful names from dlc_trackermap.txt if available, otherwise returns the internal tracker names.

Returns:

Display names for body parts.

Return type:

list

property trackermap

Load meaningful label names from dlc_trackermap.txt.

This file maps internal names (point0, point1) to biological names (nose, left_ear, etc.) for better interpretability.

Returns:

Mapping from internal names to display names.

Return type:

dict

Example dlc_trackermap.txt content:

point0 - muscle_boundary point1 - fascia point2 - bone

edit_config(config_file=None, **kwargs)

Modify project configuration parameters.

Parameters:

• config_file (str, optional) – Path to config file. Defaults to main config.

• **kwargs – Configuration parameters to update (e.g., iteration=2, snapshotindex=5).

Returns:

Result of deeplabcut.auxiliaryfunctions.edit_config().

property video_list: list[pathlib.Path]

Full paths to videos in the project.

property video_names: list[str]

Video filenames without extensions.

property current_iteration: int

Model iteration number currently set in config.yaml.

property latest_iteration: int

Highest iteration number in dlc-models folder.

property latest_trained_iteration: int

Most recent iteration that has saved model snapshots.

property all_iterations: list

All iteration numbers found in dlc-models, sorted ascending.

property all_snapshots: Mapping[int, list[int]]

Training snapshots for each model iteration.

Returns:

Maps iteration number to list of training iteration numbers.

For DLC3, identifies .pt files; for DLC2, identifies .index files.

Return type:

dict

current_iteration_is_trained() → bool

Check if current iteration has any saved snapshots.

latest_iteration_is_trained() → bool

Check if latest iteration has any saved snapshots.

iteration_is_trained(iteration_num: int) → bool

Check if a specific iteration has been trained.

Parameters:

iteration_num (int) – Model iteration to check.

Returns:

True if snapshots exist for this iteration.

Return type:

bool

increment_iteration()

Advance to next iteration if current one is trained.

Returns:

For method chaining.

Return type:

self

add_videos(videos: list[pathlib.Path], link_videos: bool | None = None)

Add new videos to existing project and copy their annotations.

Parameters:

• videos – List of video file paths to add.

• link_videos – Placement mode for the new videos. None (default) hard-links with copy fallback on cross-volume. True requires hard-linking. False always copies. Same semantics as DLCProject.__init__().

Returns:

For method chaining.

Return type:

self

copy_annotations(video_name: Path | list)

Copy DUSTrack JSON files into project’s video folder.

Parameters:

video_name – Single video path or list of video paths.

Returns:

Path(s) to copied annotation file(s), or None if not found.

Return type:

str or list

Note

Looks for files matching {video_stem}_annotations_{suffix}.json

extract_frames(annotation_file_names=None, suffix_merged='merged', save_merged_json=False, check=False)

Extract labeled frames from videos and convert annotations to DLC format.

This method: 1. Finds all annotation JSON files for each video 2. Merges multiple annotation files if present 3. Extracts the annotated frames from videos 4. Converts annotations to DLC’s CSV/HDF5 format in labeled-data folder

Parameters:

• annotation_file_names (list, optional) – Specific annotation files to use. If None, automatically finds all matching files.

• suffix_merged (str) – Suffix for merged annotation file. Defaults to ‘merged’.

• save_merged_json (bool) – Whether to save the merged JSON. Defaults to False.

• check (bool) – Whether to run deeplabcut.check_labels(). Defaults to False.

Returns:

For method chaining.

Return type:

self

Note

Automatically excludes files with ‘_dlccorr’ suffix (correction files).

get_pose_cfg_file(iteration_num: int = None, type_: str = 'train') → Path

Get path to pose configuration file for an iteration.

Parameters:

• iteration_num (int, optional) – Iteration number. Defaults to current.

• type (str) – ‘train’ or ‘test’ subfolder. Defaults to ‘train’.

Returns:

Full path to pose_cfg.yaml (DLC2) or pytorch_config (DLC3).

Return type:

Path

get_best_snapshot(iteration_num: int = None) → int

Find training iteration with lowest test error.

For DLC3, uses the snapshot marked as ‘best’ unless DLC3_USE_LAST_SNAPSHOT is True in config, in which case returns the last snapshot. For DLC2, parses CombinedEvaluation-results.csv.

Parameters:

iteration_num (int, optional) – Model iteration. Defaults to current.

Returns:

Training iteration number of best snapshot.

Return type:

int

get_best_snapshot_test_error(iteration_num: int = None) → float

Get test error (RMSE in pixels) at best snapshot.

Parameters:

iteration_num (int, optional) – Model iteration. Defaults to latest trained.

Returns:

Test error in pixels, or -1.0 if evaluation file doesn’t exist.

Return type:

float

get_best_snapshot_idx(iteration_num: int = None) → int

Get snapshot index (not training iteration number) of best snapshot.

Parameters:

iteration_num (int, optional) – Model iteration. Defaults to current.

Returns:

Index in the all_snapshots list for this iteration.

Return type:

int

initialize_weights(source_iteration: int = None, source_snapshot: int = None, dest_iteration: int = None)

Initialize model weights from a previous iteration (transfer learning).

Used when refining a model with additional labels. Edits the pose_cfg file to set init_weights parameter.

Parameters:

• source_iteration (int, optional) – Iteration to copy from. Defaults to second-to-last iteration.

• source_snapshot (int, optional) – Training iteration within source_iteration. Defaults to best snapshot.

• dest_iteration (int, optional) – Iteration to initialize. Defaults to latest iteration.

Returns:

For method chaining.

Return type:

self

Note

Does nothing if there’s only one iteration (no source to copy from).

create_training_dataset(**kwargs)

Call deeplabcut.create_training_dataset.

train(**kwargs)

Train the neural network model.

Sets custom learning rate schedule and trains with more iterations than DLC defaults for better convergence.

Parameters:

**kwargs – Passed to deeplabcut.train_network(). - maxiters (int): Total training iterations. Default: 500000 (DLC2) or 1000 (DLC3 epochs). - max_snapshots_to_keep (int): Max saved checkpoints. Default: 20.

Returns:

For method chaining.

Return type:

self

Note

Custom learning rate schedule: [0.005@10k, 0.02@350k, 0.002@425k, 0.001@1M]

evaluate(**kwargs)

Evaluate all training snapshots on test set.

Temporarily sets snapshotindex to ‘all’ to evaluate every checkpoint, then restores original value.

Parameters:

**kwargs – Passed to deeplabcut.evaluate_network().

Returns:

For method chaining.

Return type:

self

analyze_videos(iteration_num=None, snapshotindex=None, create_video=True, **kwargs)

Run inference on videos and optionally create labeled output videos.

Parameters:

• iteration_num (int, optional) – Model iteration to use. Defaults to current.

• snapshotindex (int, optional) – Snapshot index to use. Defaults to best snapshot. Negative indices supported.

• create_video (bool) – Whether to create labeled video. Defaults to True.

• **kwargs – Additional arguments for deeplabcut.analyze_videos(). - videos: List of video paths or indices. If not provided, analyzes all videos.

Returns:

For method chaining.

Return type:

self

Note

Results saved to videos/iteration-{N}/ subfolder. If videos kwarg contains integers, they’re treated as indices into self.video_list.

process(iteration_num=None, maxiters=None, refine: bool | str = True, create_video=True, source_snapshot=None, **kwargs)

Automated workflow: extract frames, train, evaluate, and analyze.

This is the main method for handling the full DLC pipeline. It intelligently decides what steps to run based on the current project state: - If iteration already evaluated: just analyze videos - If frames need extraction: extract them - If not trained: train the model - If refining: initialize weights from previous iteration

Parameters:

• iteration_num (int or str, optional) – Iteration to process. Can be integer or ‘latest’. Defaults to ‘latest’.

• maxiters (int, optional) – Training iterations. Defaults: 500000 (DLC2) or 1000 epochs (DLC3).

• refine (bool) – Use transfer learning from previous iteration. Defaults to True.

• create_video (bool) – Create labeled output video. Defaults to True.

• source_snapshot (int, optional) – Specific snapshot for weight initialization.

• **kwargs – Additional arguments. - videos: List of videos to analyze (can be indices or paths).

Returns:

For method chaining.

Return type:

self

Example

>>> proj = DLCProject('path/to/project')
>>> proj.process()  # Full automated workflow

train_iteration(*, refine_mode: Literal['scratch', 'in_project', 'external'] = 'scratch', source_iteration: int = None, source_snapshot: int = None, external_snapshot_path: str = None, maxiters: int = None, create_video: bool = False, videos: list = None, analyze_batchsize: int = None)

Explicit-args training driver for UI-triggered flows.

Distinct from process() (auto-infer for CLI ergonomics). Caller decides everything: refine source, training duration, output options. Strict validation per refine_mode; no inference and no silent fallbacks.

The mechanics of advancing iterations (extract_frames → increment_iteration if latest is trained → create_training_dataset if needed) mirror process(); the only difference is how weights are initialised once the destination iteration is in place.

Parameters:

• refine_mode – How to initialise weights for the next training round. "scratch" starts from random init (no pose_cfg edit); "in_project" copies weights from a snapshot in this project (requires source_iteration, optional source_snapshot); "external" initialises from an arbitrary snapshot file (requires external_snapshot_path; supported on both DLC2 and DLC3 – DLC3 passes the path through train_network(snapshot_path=...), DLC2 edits pose_cfg’s init_weights via _initialize_weights_from_external_path()).

• source_iteration – in-project iteration to copy weights from. Only valid with refine_mode="in_project"; must point at a trained iteration.

• source_snapshot – specific snapshot within source_iteration. Only valid with refine_mode="in_project"; defaults to the best snapshot when None.

• external_snapshot_path – path to an external snapshot file (.pt on DLC3, .index on DLC2). Only valid with refine_mode="external"; the file must exist at call time.

• maxiters – training epochs (DLC3) or iterations (DLC2). Defaults to the same values process() uses (50 / 500000) so the two methods stay consistent until the UI exposes the field.

• create_video – write a labeled output video after analyze. Defaults to False (the UI ergonomics default; process() defaults to True for CLI parity).

• videos – list of videos (indices or paths) to analyze. Forwarded to analyze_videos. None analyses every video in the project.

• analyze_batchsize – batchsize for analyze_videos. Forwarded on if set; None lets analyze_videos pick its own default (post-2026-05-20: rc14 knee at 4).

Returns:

For method chaining.

Return type:

self

Raises:

• ValueError – on refine_mode / argument mismatch (see _validate_train_iteration_args()).

• TypeError – if source_iteration / source_snapshot aren’t int when given.

• FileNotFoundError – if external_snapshot_path is set but the file doesn’t exist.

annotate(video_index: int = 0, new_annotation_suffix=None, **dustrack_kwargs)

Launch interactive annotation GUI for a video.

Opens DUSTrack interface with existing annotation layers loaded, including any DLC predictions as line plot overlays.

Parameters:

• video_index (int) – Index of video in video_list. Defaults to 0. Negative indices supported.

• new_annotation_suffix (str, optional) – Suffix for new annotation layer. Defaults to ‘iteration-{N}’ where N is the next iteration number.

• **dustrack_kwargs – Forwarded to the DUSTrack constructor. Notable pass-through options: fast_render=True (datanavigator 1.5.0+ Tier 2 Qt-native video pane, ~3x speedup on the interosseous_pn24-x benchmark), dark_mode=True, clahe_clip, clahe_grid, gamma, brightness.

Returns:

Interactive annotation interface.

Return type:

DUSTrack

Note

Creates a ‘buffer’ layer for temporary annotations. Latest DLC predictions are automatically set as overlay.

get_trajectories(videos=None, iteration=None)

Load tracking results as DLCData objects.

Parameters:

• videos (list or str, optional) – Videos to load. Defaults to all videos.

• iteration (int, optional) – Model iteration. Defaults to current.

Returns:

Maps video stem to DLCData object.

Return type:

dict

Raises:

ValueError – If a requested video is not in the project.

open()

Open project folder in Windows Explorer.

Optical flow

Module for optical flow based postprocessing using the LK-RSTC algorithm.

Post-processing module for video annotations using optical flow algorithms, typically applied on the output of deep learing models when the results have frame to frame jitter.

The primary algorithm is Lucas-Kanade optical flow with Reverse Sigmoid Tracking Correction (RSTC) applied in a “transposed” moving average window.

Key Features:

• Lucas-Kanade optical flow tracking for point trajectories

• Reverse Sigmoid Tracking Correction (RSTC) guarantees the locations of the start and end points

• Moving average filter over time windows

• Parallel processing support for faster computation

• Video frame buffering to minimize disk I/O

Typical Usage:

>>> from dustrack.lk_filter import lk_moving_average_filter
>>> from dustrack import VideoAnnotation
>>>
>>> # Load annotations
>>> ann = VideoAnnotation('video_annotations.json', 'video.mp4')
>>>
>>> # Apply post-processing with 0.5 second window
>>> ann_smooth = lk_moving_average_filter(ann, window_size=0.5)

In practice, access the lk_moving_average_filter method via VideoAnnotation:

>>> from dustrack import VideoAnnotation
>>> ann = VideoAnnotation('video_dlc_predictions.h5', 'video.mp4')
>>> ann_rstc = ann.postprocess()

Algorithm Overview:

The RSTC algorithm tracks points both forward and backward in time, then combines the two trajectories using sigmoid weights that emphasize the forward path at the start and the backward path at the end. This reduces drift accumulation compared to pure forward tracking.

The moving average applies RSTC over overlapping time windows and averages the results, further reducing noise while maintaining temporal coherence.

Performance:

Processing speed depends on: - Video resolution and frame rate - Number of tracked points - Window size (larger = more computation) - Parallel processing (use_parallel=True recommended)

Typical performance: ~5-10 fps for 1080p video with 10 points and 0.5s window.

Reference for the LK-RSTC Algorithm:

Magana-Salgado, U., Namburi, P., Feigin-Almon, M., Pallares-Lopez, R., & Anthony, B. (2023). A comparison of point-tracking algorithms in ultrasound videos from the upper limb. BioMedical Engineering OnLine, 22(1), 52.

dustrack.lk_filter.gray(video_frame: ndarray) → ndarray

Convert a color video frame to grayscale.

The Lucas-Kanade algorithm operates on grayscale images. Input frames come from dnav VideoReader (PyAV+TOC). Since 1.5.0a2, dnav auto-detects monochrome-encoded sources (h265 pix_fmt=gray) and decodes them directly as (H, W) gray ndarrays, skipping the YUV->RGB color matrix. On those inputs this helper is a no-op short-circuit. On the legacy color-source path the frame arrives as RGB and is converted via BT.601 COLOR_RGB2GRAY (Y = 0.299 R + 0.587 G + 0.114 B).

Parameters:

video_frame (np.ndarray) – Input frame – (H, W, 3) RGB or (H, W) gray.

Returns:

Grayscale frame (H, W).

Return type:

np.ndarray

Pre-2026-05-21 this used COLOR_BGR2GRAY on the same RGB input – functionally still grayscale, but it swapped the R and B coefficients, computing Y = 0.114 R + 0.587 G + 0.299 B. Self-consistent within lk_moving_average_filter (both template and search frames went through the same wrong conversion, so LK gradient + iteration math worked fine), but it diverged from dustrack.lk_opticalflow which always used COLOR_RGB2GRAY. Unified to RGB2GRAY in the 1.2.0a2 perf pass.

dustrack.lk_filter.lucas_kanade_2(frame_list: list, init_points: ndarray, **lk_config) → ndarray

Track points through a sequence of frames using Lucas-Kanade optical flow.

This implementation uses OpenCV’s pyramidal Lucas-Kanade method to track points from the initial position through all subsequent frames. Tracking is performed in a single forward pass.

Parameters:

• frame_list (list) – List of grayscale video frames (each frame is np.ndarray).

• init_points (np.ndarray) – Initial point positions, shape (n_points, 2) where each row is [x, y] in pixel coordinates.

• **lk_config – Configuration parameters for cv.calcOpticalFlowPyrLK(): - winSize (tuple): Window size for search. Default: (45, 45) - maxLevel (int): Pyramid levels. Default: 2 - criteria (tuple): Termination criteria. Default: (EPS|COUNT, 10, 0.03)

Returns:

Tracked point trajectories, shape (n_frames, n_points, 2).

Missing/lost points are filled with NaN.

Return type:

np.ndarray

Note

Larger winSize provides more robustness but slower computation. Higher maxLevel helps track larger motions but may lose precision.

Example

>>> frames = [gray(f) for f in video_frames]
>>> initial_pts = np.array([[100, 200], [150, 250]])
>>> trajectories = lucas_kanade_2(frames, initial_pts)
>>> # trajectories.shape = (len(frames), 2, 2)

dustrack.lk_filter.compute_sigmoid_weights(n_frames: int, epsilon: float = 0.01) → tuple[numpy.ndarray, numpy.ndarray]

Compute forward and reverse sigmoid weights for RSTC blending.

The weights control how forward and backward tracking paths are combined: - At the start: forward weight ≈ 1, reverse weight ≈ 0 - At the end: forward weight ≈ 0, reverse weight ≈ 1 - In the middle: both weights ≈ 0.5

Parameters:

• n_frames (int) – Number of frames.

• epsilon (float, optional) – Small value to control sigmoid scaling. Defaults to 0.01.

Returns:

Two arrays of shape (n_frames,):

• sigmoid_forward: Decreasing weights from ~1 to ~0

• sigmoid_reverse: Increasing weights from ~0 to ~1

Forward and reverse weights sum to 1.0 at each frame.

Return type:

tuple[np.ndarray, np.ndarray]

Mathematical Form:

sigmoid(x) = 1 / (1 + exp(b * (x - c))) where b controls steepness and c is the center point.

dustrack.lk_filter.lucas_kanade_rstc_2(frame_list: list, start_points: ndarray, end_points: ndarray, sigmoid_forward: ndarray = None, sigmoid_reverse: ndarray = None, **lk_config) → ndarray

Apply Reverse Sigmoid Tracking Correction (RSTC) for robust point tracking.

RSTC combines forward tracking (from start_points) and backward tracking (from end_points) using sigmoid weights. This reduces error accumulation compared to pure forward tracking, especially over long sequences.

Algorithm:

1. Track forward from start_points through all frames

2. Track backward from end_points through all frames (in reverse)

3. Blend the two paths using sigmoid weights: result = forward_path * sigmoid_forward + reverse_path * sigmoid_reverse

Parameters:

• frame_list (list) – List of grayscale video frames.

• start_points (np.ndarray) – Initial positions at first frame, shape (n_points, 2).

• end_points (np.ndarray) – Final positions at last frame, shape (n_points, 2). Should match start_points for consistency.

• sigmoid_forward (np.ndarray, optional) – Forward weights. If None, computed automatically.

• sigmoid_reverse (np.ndarray, optional) – Reverse weights. If None, computed automatically.

• **lk_config – Lucas-Kanade configuration (passed to lucas_kanade_2).

Returns:

Blended tracking paths, shape (n_frames, n_points, 2).

Return type:

np.ndarray

Note

Precomputing sigmoid weights and reusing them across windows significantly improves performance when processing multiple sequences of the same length.

Example

>>> frames = [gray(f) for f in video_frames[10:20]]
>>> start = np.array([[100, 200]])
>>> end = np.array([[105, 205]])  # Approximate end position
>>> trajectory = lucas_kanade_rstc_2(frames, start, end)

dustrack.lk_filter.process_window(video_frame_buffer, start_points, end_points, sigmoid_forward, sigmoid_reverse)

Process a single time window using RSTC.

This is a helper function designed for parallel execution by ThreadPoolExecutor. It wraps lucas_kanade_rstc_2 with precomputed sigmoid weights.

Parameters:

• video_frame_buffer – Deque or list of grayscale frames for this window.

• start_points (np.ndarray) – Point positions at window start.

• end_points (np.ndarray) – Point positions at window end.

• sigmoid_forward (np.ndarray) – Precomputed forward weights.

• sigmoid_reverse (np.ndarray) – Precomputed reverse weights.

Returns:

Tracked trajectories for this window, shape (window_frames, n_points, 2).

Return type:

np.ndarray

dustrack.lk_filter.lk_moving_average_filter(tracked_points: str | VideoAnnotation, video_name: str = None, window_size: float = 0.5, use_parallel: bool = True, save_raw: bool = True) → VideoAnnotation

Apply Lucas-Kanade moving average filter to smooth tracking annotations.

This is the main post-processing function. It applies RSTC over overlapping time windows and averages the results to produce smooth, jitter-free trajectories.

Workflow:

1. Load video and annotations

2. Slide a time window through the video

3. For each window: - Apply RSTC to get smoothed trajectory - Store results indexed by window position

4. Average all trajectories that cover each frame

5. Save smoothed annotations

Parameters:

• tracked_points (Union[str, VideoAnnotation]) – Either: - Path to annotation JSON file - VideoAnnotation object with tracking data

• video_name (str, optional) – Video path. Defaults to None; required when tracked_points is a file path (asserted at call time). Ignored when tracked_points is already a VideoAnnotation.

• window_size (float, optional) – Time window in seconds for moving average. Larger windows provide more smoothing but increase computation. Typical range: 0.1 to 1.0 seconds. Defaults to 0.5.

• use_parallel (bool, optional) – Use ThreadPoolExecutor for parallel processing. Significantly faster for longer videos. Defaults to True.

• save_raw (bool, optional) – If True (default), allocate the full (n_window_frames, n_total_frames, n_labels, 2) per-window RSTC array and dill-pickle it to {stem}_lkmovavg_{window_size}.pkl alongside the averaged JSON. Per-window data is consumed downstream by pn-projects/wobble and gaitmusic (their .rawlk property → lk_gradients velocity estimation), so keep True for those flows. If False, stream a running (n_total_frames, n_labels, 2) sum and per-cell count instead, divide at the end, and skip the .pkl write. Cuts peak memory from W*N*L*16 bytes to N*L*12 bytes (~10x on typical W=15, L=4 cases). On the example-video bench wall time is essentially unchanged (post-processing is <0.3% of total) but on real-world long videos (N=36715+) the .pkl round-trip alone is a couple of seconds. FP summation order changes between modes so save_raw=False output is numerically close (np.allclose with atol~1e-10) rather than bit-exact vs save_raw=True.

Returns:

New annotation object with smoothed trajectories.

Saved to {original_stem}_lkmovavg_{window_size}.json

Return type:

VideoAnnotation

Output Files:

• {stem}_lkmovavg_{window_size}.pkl: Per-window RSTC results (only when save_raw=True; consumed by .rawlk downstream).

• {stem}_lkmovavg_{window_size}.json: Averaged smoothed annotations (always written).

Performance Tips:

• Use use_parallel=True for videos longer than a few seconds

• Smaller window_size is faster but provides less smoothing

• Results are cached: the .json is the primary cache, and when save_raw=True the .pkl is reused on subsequent calls regardless of the save_raw flag (a pre-existing .pkl is always read if the .json is missing). Delete either to recompute the corresponding stage.

Example

>>> # Post-process manual annotations
>>> ann = VideoAnnotation('video_annotations.json', 'video.mp4')
>>> ann_smooth = lk_moving_average_filter(ann, window_size=0.5)
>>>
>>> # Post-process DLC predictions with larger window
>>> ann_dlc = VideoAnnotation('video_dlc_predictions.h5', 'video.mp4')
>>> ann_smooth = lk_moving_average_filter(ann_dlc, window_size=1.0)
>>>
>>> # Non-parallel processing for short clips
>>> ann_smooth = lk_moving_average_filter(ann, window_size=0.3, use_parallel=False)
>>>
>>> # GUI-style: skip the .pkl sidecar (memory + I/O savings)
>>> ann_smooth = lk_moving_average_filter(ann, save_raw=False)

In practice, the postprocess shortcut on a VideoAnnotation is equivalent: >>> from dustrack import VideoAnnotation >>> ann = VideoAnnotation(‘video_dlc_predictions.h5’, ‘video.mp4’) >>> ann_rstc = ann.postprocess()

Raises:

• AssertionError – If video_name is not provided when tracked_points is a file path.

• AssertionError – If tracked_points is not a VideoAnnotation or valid file path.

Note

The algorithm uses a deque-based frame buffer to minimize memory usage and avoid repeated video decoding. Frames are read once and discarded after the window moves past them.

Batch processing

Helpers for warming a folder of ultrasound videos before an annotation session. Pre-build the PyAV+TOC sidecars (build_toc) so dustrack.open(...) returns instantly on every file, and re-encode yuv420p captures as h265 monochrome (convert_to_mono) to fix the chroma-noise penalty that yuv420p clips carry into DLC inference.

The batch surface is also wired into the GUI: dustrack.open() (with no path) pops a welcome modal whose top-right Batch process… button opens a click-driven interface for the same operations; the Tools menu on the main window has the same entry for use mid-session.

Mono encoding — CRF quality reference

Ultrasound capture pipelines typically store frames as h264 yuv420p, which folds chroma-plane noise back into the gray channel during RGB→gray conversion and costs ~0.7 px median DLC keypoint error. convert_to_mono re-encodes as h265 4:0:0 monochrome (-c:v libx265 -pix_fmt gray) so the chroma planes are dropped entirely — both for storage savings (~6% smaller at default settings) and for DLC accuracy.

The CRF default is 22 (chosen for parity with capture-side quality). Higher CRF values trade pixel accuracy for storage savings. The following table summarises the tradeoff measured on a 30 s pia02 ultrasound subclip (2010 frames × 2 keypoints, interosseous_pn24 ResNet-50 model, pixel error relative to a lossless mono reference):

CRF	file size (relative)	median error (px)	p95 (px)
22 (default)	100%	0.37	1.04
24	72%	0.47	1.29
26	51%	0.58	1.60
28	35%	0.73	1.96

All values stay sub-pixel at the median; the p95 statistic is the better gauge of how often DLC trace post-processing (LK refinement) will need to clean up a wobble. CRF 28 sits at the edge where p95 starts to approach the LK refinement window, so 22-26 is the recommended operating range. Storage-constrained workflows can pass crf=24 or crf=26 to convert_to_mono (or to immersionlab.telemed.crop_video(mono=True, crf=...) for the one-pass crop+mono variant).

Batch utilities for ultrasound video corpora.

Two workflows live here:

• convert_to_mono() — re-encode ultrasound clips as h265 monochrome (drops chroma noise; unlocks dnav’s pix_fmt='gray' auto-detect for ~6x sequential-decode speedup). See the original module header below.

• build_toc() — pre-build the PyAV+TOC sidecar (<video>.dnav-toc) for every video in a folder. First DUSTrack open of a video pays the per-file TOC build cost (a full sequential demux to record per-packet offsets + per-frame timestamps); pre- building means dustrack.open(...) returns essentially instantly on warm folders. Delegates to datanavigator.precompute_toc_folder() so other portfolio consumers can hit the same code path. For DLC projects, point this at <project>/videos with recursive=False.

Original convert_to_mono docstring follows:

Batch re-encode ultrasound videos as h265 monochrome (pix_fmt=gray).

Ultrasound is inherently grayscale, but the typical capture pipeline stores it as yuv420p h264 with U/V planes carrying chroma noise that gets mixed back into Y during BT.601 RGB->gray conversion downstream. Re-encoding as h265 4:0:0 monochrome:

• drops the chroma planes entirely (parity_decoder.py 2026-05-21: chroma noise contribution to gray was ~1.4/255 mean per frame, DLC inference parity median = 0.19 px)

• unlocks dnav’s pix_fmt='gray' auto-detect path (~6x sequential decode speedup on the LK / lk_filter paths)

• shrinks the file ~6% at CRF 22 vs the typical capture-side CRF

• speeds up TOC build ~20-40% (fewer + smaller packets per frame)

Codec choice: libx264 cannot produce true 4:0:0 in ffmpeg (silently falls back to yuvj420p-with-constant-chroma). libx265 writes real monochrome cleanly; cv2.VideoCapture and PyAV both handle h265 mono without ceremony, and the 2026-05-21 cv2-vs-PyAV parity sweep confirms they produce bit-exact identical decoded frames on this format.

Defaults (CRF 22, preset slow) match the immersionlab telemed convention adjusted upward for h265’s better compression efficiency. On a 706x558 typical ultrasound clip these land within 6% of the source file size.

Originals are never touched: output goes to <original_stem><suffix>.mp4 next to each source. Recovery from a bad batch = delete the *_mono.mp4 outputs.

Example

>>> import dustrack
>>> dustrack.convert_to_mono('S:/study/clip01.mp4')
>>> # or a directory:
>>> dustrack.convert_to_mono('S:/study/')
>>> # or a list:
>>> dustrack.convert_to_mono(['vid1.mp4', 'vid2.mp4'])

dustrack.batch.convert_to_mono(sources: str | Path | Iterable[str | Path], *, crf: int = 22, preset: str = 'slow', suffix: str = '_mono', skip_existing: bool = True, skip_already_mono: bool = True, ffmpeg_bin: str | None = None, ffprobe_bin: str | None = None, verbose: bool = True, show_progress: bool = False, progress_callback: Callable[[int, int, Path, str], None] | None = None, cancel_check: Callable[[], bool] | None = None) → list[pathlib.Path]

Re-encode each source as h265 monochrome alongside the original.

Parameters:

• sources – A video path, an iterable of video paths, or a directory (recursive walk of supported extensions). Originals are never modified – each output is written as <source_stem><suffix>.mp4 in the same folder.

• crf – libx265 quality. Lower = better quality, bigger file. Default 22 lands within 6% of typical ultrasound capture file size on 706x558 clips; DLC inference parity at CRF 22 is sub-pixel (median 0.19 px). Pass a lower value (e.g. 18) for tighter pixel parity at a 50%+ file-size cost.

• preset – libx265 preset. Slower = better compression at same CRF. Default "slow" matches the immersionlab telemed convention.

• suffix – Appended to the source stem to form the output filename. Default "_mono".

• skip_existing – When True (default), skip sources whose output file already exists. Pass False to force re-encode.

• skip_already_mono – When True (default), skip sources whose encoded pix_fmt is already monochrome (no point re-encoding a mono file). Detected via ffprobe.

• ffmpeg_bin – Override ffmpeg path; default looks on PATH and at C:/ffmpeg/bin/ffmpeg.exe.

• ffprobe_bin – Override ffprobe path; same lookup rules as ffmpeg.

• verbose – Print one status line per source.

• show_progress – If True, wrap the per-file loop in a tqdm bar (per-file status lines are routed through tqdm.write so the bar stays clean). Matches the show_progress kwarg on build_toc(). Off by default to preserve the historical print-only behaviour for shell users.

• progress_callback – Optional fn(idx, total, src_path, status) invoked after each source. status is one of "ok", "skip_missing", "skip_overwrite", "skip_existing", "skip_already_mono", or "failed". Used by the Qt batch-process modal to drive its own progress UI without relying on tqdm.

• cancel_check – Optional zero-arg callable polled at the top of each source. If it returns truthy, the loop exits early. Used by the Qt batch-process modal’s Cancel button.

Returns:

List of output paths actually written (skipped sources omitted).

dustrack.batch.build_toc(sources: str | Path | Iterable[str | Path], *, extensions: Iterable[str] = ('.mp4', '.mov', '.mkv', '.avi', '.m4v'), recursive: bool = True, force: bool = False, show_progress: bool = True, progress_callback: Callable[[int, int, Path, str], None] | None = None, cancel_check: Callable[[], bool] | None = None) → dict

Pre-build the PyAV+TOC sidecar for every video under sources.

Thin pass-through to datanavigator.precompute_toc_folder() so DUSTrack callers can stay inside the dustrack namespace. First open of a video pays the per-file TOC build cost; pre-building means dustrack.open(...) returns essentially instantly afterward.

Example:

import dustrack
dustrack.batch.build_toc("M:/us_videos_for_tracking2")

Parameters:

• sources – A directory, a video file, or an iterable mixing both. Directories are walked for extensions.

• extensions – File extensions to include (case-insensitive). Default matches dnav’s DEFAULT_VIDEO_EXTENSIONS.

• recursive – If True (default), recurse into subdirectories.

• force – If True, rebuild even when a valid cache exists. Useful for upgrading pre-1.3 sidecars to schema v2 with per-frame timestamps.

• show_progress – If True (default), wrap iteration in a tqdm bar. Suppressed automatically when progress_callback is set.

• progress_callback – Optional fn(idx, total, video_path, status) invoked after each video. status is the per-file result string from datanavigator.precompute_toc() ("hit", "built", "built (uncached)", "error: ..."). Forwarded to dnav; the path is converted to a Path for consumer convenience (dnav itself emits a string path).

• cancel_check – Optional zero-arg callable polled at the top of each file. If truthy, the loop exits early and the partial result dict is returned.

Returns:

{path: status} per datanavigator.precompute_toc(), plus an "error: missing" entry for any explicitly-named path that doesn’t exist.

Helpers

class dustrack.annotations.VideoAnnotation(fname: str | None = None, vname: str | None = None, name: str | None = None, n_labels: int = 1, **kwargs)

Bases: object

Manage one point annotation layer in a video.

Each annotation layer can contain up to 10 labels, which are string representations of digits 0-9. Each label is a dictionary mapping a frame number to a 2D location on the video frame.

Parameters:

• fname (str, optional) – File name of the annotations (.json) file. If it doesn’t exist, it will be created when the save method is used. If this is a video file, fname will default to <video_name>_annotations.json. This can also be a DeepLabCut .h5 file (either labeled data OR predicted trace). Defaults to None.

• vname (str, optional) – Name of the video being annotated. Defaults to None.

• name (str, optional) – Name of the annotation (something meaningful, e.g., <muscle_name>_<scorer> such as brachialis_praneeth). Defaults to None.

• n_labels (int, optional) – Number of labels for annotation. Defaults to 10.

• **kwargs – Additional optional parameters: - palette_name (str, default=’Set2’): Color scheme to use. Defaults to ‘Set2’ from seaborn. - ax_list (list, default=[]): If specified, the annotation display will be initialized on these axes. Alternatively, use VideoAnnotation.setup_display() to specify the axis list and colors. - preloaded_json (dict, optional): The result of VideoAnnotation._load_json (in case you prefer to pickle the JSON files).

to_dlc()

Convert from JSON file format into a DeepLabCut DataFrame format, and optionally save the file.

__init__(fname: str | None = None, vname: str | None = None, name: str | None = None, n_labels: int = 1, **kwargs) → None

property data: dict[str, dustrack.annotations._TrackedFrameDict]

Per-label frame→location dictionary.

Reads behave like a normal dict[str, dict[int, list[float]]]. Writes through ann.data[label][frame] = ... automatically bump _revision, keeping consumers’ caches consistent. See _TrackedFrameDict.

classmethod from_multiple_files(fname_list: list[str], vname: str, name: str, fname_merged: str, **kwargs) → VideoAnnotation

Merge annotations from multiple files. If multiple files contain an annotation label for the same frame, values from the last file will be kept.

load(n_annotations: int = 1, **h5_kwargs) → dict

Load annotations from a json file, dlc h5 file, or initialize an annotation dictionary if a file doesn’t exist.

DUSTrack-shaped: the DeepLabCut .h5 branch exists in the generic class because datanavigator and DUSTrack co-developed. The DLC paths will migrate to dustrack.VideoAnnotation in 1.3.0 alongside the pointtracking.py split; the JSON path stays here.

property n_frames: int

Number of frames in the video being annotated

property n_annotations: int

Number of points being annotated in the video.

property labels: list[str]

Labels of the annotations.

property palette: list[tuple]

Color palette for the annotations.

property frames: list[int]

Frame numbers in the video that have annotations.

property frames_overlapping: list[int]

list of frames in the video where all the labels are annotated.

property plot_type: str

Type of plot to use for the annotations.

get_frames(label: str) → list[int]

Return a list of frames that are annotated with the current label.

reload() → None

Drop in-memory state and reload from disk (or start fresh).

Inverse of save(). If fname is None or the file doesn’t exist, restores the empty {str(i): {} for i in range(n)} shape via the existing load() fallback (see the file-missing branch). Wholesale- replaces self.data so the property setter rewraps every per-label inner dict as a _TrackedFrameDict, and bumps _revision explicitly so per-frame caches keyed on (label_list, _revision) invalidate – the outer setter rewraps but does not itself bump (mirrors sort_data() and remove_empty_labels()).

save(fname: str | None = None) → None

Save the annotations json file. self.fname should be a valid file path.

Empty-but-declared labels are preserved (written as "label": {}). 1.4.0rc2 promoted labels from a derived property of “which keys have data” to first-class schema; the previous implicit self.remove_empty_labels() call here is gone. Callers that want a lean export (e.g. DUSTrack pre-flight before DLC training) still drive remove_empty_labels() explicitly.

to_json() → None

Alias for save method.

sort_labels() → None

Sort labels in the data dictionary.

sort_data() → None

Sort annotations by the frame numbers.

clip_trailing_empty_labels() → None

Remove trailing empty labels from the annotation dictionary.

remove_empty_labels() → None

Remove empty labels from the annotation dictionary.

get_values_cv(frame_num: int) → ndarray

Return annotations at frame_num in a format for openCV’s optical flow algorithms

add_at_frame(frame_num: int, values: ndarray) → None

Add annotations at a frame, given the annotation values.

get_at_frame(frame_num: int) → list[list[float]]

Retrieve annotations at a given frame number. If an annotation is not present, nan values will be used.

to_dlc(scorer: str = 'praneeth', output_path: str | None = None, file_prefix: str | None = None, img_prefix: str = 'img', img_suffix: str = '.png', label_prefix: str = 'point', save: bool = True, internal_to_dlc_labels: dict[str, str] | None = None) → DataFrame

Save annotations in deeplabcut format.

DUSTrack-shaped: emits a DeepLabCut-shaped DataFrame (and writes an .h5 if save=True); slated to migrate to dustrack.VideoAnnotation in 1.3.0 alongside the pointtracking.py split.

to_trace(label: str) → ndarray

Return a 2d numpy array of n_frames x 2.

Parameters:

label (str) – Annotation label.

Returns:

2d numpy array of number of frames x 2.

xy position values of uannotated frames will be filled with np.nan.

Return type:

np.ndarray

Schema-tolerant: a label absent from data is treated as all-frames-unannotated and returns the full NaN array. Lets cross-layer trace consumers (e.g. _DUSTrackBase.update_frame_marker(), which iterates every annotation layer with one shared label) survive a layer that legitimately doesn’t carry the active label – either a freshly created layer or a layer where the user hasn’t placed any points for that label yet. 1.4.0rc2: prior to the schema-tolerant relaxation this asserted, which crashed the corrections-layer flow when the patch had been saved with one of its labels empty (save then pruned that label, so it disappeared from labels and the assert fired).

to_traces() → Mapping[str, ndarray]

Return annotations as traces (numpy arrays of size n_frames x 2).

Returns:

Dictionary mapping labels to 2d numpy arrays.

Return type:

Mapping[str, np.ndarray]

to_signal(label: str) → Data

Return an annotation as pysampled.Data at the frame rate of the video.

Parameters:

label (str) – Annotation label.

Returns:

Signal sampled at the frame rate of the video being annotated.

Return type:

pysampled.Data

to_signals() → Mapping[str, Data]

Return annotations as a dictionary of sampled Data signals sampled at the frame rate of the video being annotated.

Returns:

Dictionary mapping labels to pysampled.Data.

Return type:

Mapping[str, pysampled.Data]

to_pysampled() → Data

Return annotations as a pysampled.Data object

Returns:

pysampled.Data

add_label(label: str | None = None, color: tuple[float, float, float] | None = None) → None

add(location: list[float], label: str, frame_number: int) → None

Add a point annotation (location) of a given label at a frame number.

remove(label: str, frame_number: int) → None

Remove a point annotation of a given label at a frame number.

setup_display_scatter(ax_list_scatter=None) → None

Setup scatter plot display.

Each entry in ax_list_scatter is either: - a matplotlib Axes (Tier 1) – a PathCollection is

created via ax.scatter and stashed in plot_handles;

• a Qt QGraphicsItemGroup (Tier 2) – a datanavigator._qt._QtScatterArtist is built on the group and stashed instead. Both expose the same mpl-PathCollection-shaped API consumed downstream.

setup_display_trace(ax_list_trace_x: None | Axes | list[matplotlib.axes._axes.Axes] = None, ax_list_trace_y: None | Axes | list[matplotlib.axes._axes.Axes] = None) → None

Setup trace plot display.

setup_display(ax_list_scatter: None | Axes | list[matplotlib.axes._axes.Axes] = None, ax_list_trace_x: None | Axes | list[matplotlib.axes._axes.Axes] = None, ax_list_trace_y: None | Axes | list[matplotlib.axes._axes.Axes] = None) → None

Setup display for scatter and trace plots.

clear_display() → None

Clear the display.

re_setup_display() → None

re-establish display elements when adding a label

invalidate_caches() → None

Drop per-annotation render caches so the next update_display_trace re-runs the full ydata sweep.

Recovery hatch for direct .data mutations from the command line (which skip the _revision bump the trace cache keys on). Normal in-code mutations should go through add() / remove() / add_at_frame() instead.

update_display_scatter(frame_number: int, draw: bool = False) → None

Update scatter plot display.

update_display_trace(label: str | None = None, draw: bool = False) → None

Update trace plot display.

Trace contents are a pure function of (label_list, self.data); none of that changes when only the parent’s _current_idx moves. Cache on (label_list, self._revision) so per-frame navigation skips the to_trace / set_ydata sweep entirely.

update_display(frame_number: int, label: str | None = None, draw: bool = False) → None

Update scatter and trace plot display.

hide(draw: bool = True) → None

Hide all elements (scatter, traces) in this annotation.

show(draw: bool = True) → None

Show all elements (scatter, traces) in this annotation.

show_trace(label: str, draw: bool = True) → None

Show trace for a specific label.

hide_trace(label: str, draw: bool = True) → None

Hide trace for a specific label.

show_one_trace(label: str, draw: bool = True) → None

Show only one trace for a specific label.

set_alpha(alpha: float = 0.4, label: str | None = None, draw: bool = True) → None

Set the transparency level of all (or one) the traces and labels in this annotation.

Parameters:

• alpha (float, optional) – alpha value between 0 and 1. Defaults to 0.4.

• label (_type_, optional) – Defaults to all labels.

• draw (bool, optional) – Update display if True. Defaults to True.

set_plot_type(type_: str = 'line', draw: bool = True) → None

Set the plot type for traces.

Records the choice on _plot_type and applies the visual style, so subsequent re_setup_display() / setup_display() calls (which read self.plot_type) don’t revert to the stale value. Symmetric with the plot_type property setter.

Parameters:

• type (str, optional) – “line” or “dot”. Defaults to “line”.

• draw (bool, optional) – Update display if True. Defaults to True.

clip_labels(start_frame: int, end_frame: int) → None

Remove annotations outside the clip range. Clip range includes start and end frame.

keep_overlapping_continuous_frames() → None

Keep data from consecutive frames that have all labels.

keep_overlapping_frames() → None

Keep data only from frames where every label is annotated.

Sibling of keep_overlapping_continuous_frames() without the consecutive-runs constraint: fully-labeled but isolated frames are preserved. Motivating use case is DLC training pre-flight, where partial frames degrade the trained model even though DLC tolerates per-bodypart NaN in its CSV.

get_area(labels: list[str] | str, lowpass: float | None = None) → Data | ndarray

Get the area in pixel squared.

export_video(out_file_name=None, start_frame=None, end_frame=None)

postprocess

Post-processes the annotation using LK-RSTC filter. See lk_moving_average_filter() for details.

class dustrack._file_management.VideoFileManager(d: DLCProject, video_index: int)

Bases: FileManager

File manager for organizing annotation and result files for one video.

Provides convenient access to all files associated with a video in a DLC project: - Manual annotation JSON files - DLC prediction HDF5 files - Labeled data files for training

Variables:

• project_name (str) – Name of the DLC project.

• video_stem (str) – Video filename without extension.

• video_fname (str) – Full path to video file.

__init__(d: DLCProject, video_index: int)

Initialize file manager for a specific video.

Parameters:

• d (DLCProject) – Parent DLC project.

• video_index (int) – Index of video in project’s video list.

property annotations: dict

Map annotation names to file paths.

Returns:

{annotation_name: file_path} for all JSON annotation files.

Return type:

dict

property annotation_files: list

List of full paths to annotation JSON files.

property annotation_names: list

List of annotation layer names (without paths or extensions).

static canonical_layer_name(fname) → str

Single source of truth for DUSTrack layer names derived from a filepath.

The DUSTrack workflow produces three categories of layer file:

• Manual / hand-edited annotations: <video>_annotations[_<name>].json. Returns the suffix after _annotations (or empty string if absent), which is what users picked when they saved.

• DLC prediction traces: live under videos/iteration-{N}/ and have DLC in the stem. Returns 'dlc_iteration-{N}_<last underscore-token of stem>'. This pattern also catches LK-RSTC post-processed outputs, which inherit the DLC source stem – so a jitter-reduced layer gets a deterministic dlc_iteration-{N}_<window> name rather than the "noname" fallback that VideoAnnotation.__init__ produces for paths without _annotations.

• Anything else: the file stem.

Called by annotations / dlc_traces at fresh-load time AND by DUSTrack._adopt_layer() for in-session adds, so the name a user sees in the layer panel is identical regardless of whether the layer was discovered on disk or produced live.

property dlc_traces: dict

Map DLC trace names to HDF5 file paths.

Returns:

{trace_name: file_path} for all DLC prediction files.

Trace names format: ‘dlc_iteration-{N}_{training_iter}’

Return type:

dict

property dlc_trace_files

List of full paths to DLC prediction HDF5 files.

property dlc_trace_names

List of DLC trace identifiers.

property labeled_data

Path to HDF5 file containing training labels in DLC format.

Returns:

Full path to CollectedData HDF5 file.

Return type:

str

Raises:

AssertionError – If file doesn’t exist or multiple files found.

get_new_json(new_suffix) → Path

Create path for a new annotation file with given suffix. Used to generate the the filename for the next refinement iteration.

Parameters:

new_suffix (str) – Suffix for new annotation layer.

Returns:

Full path to new JSON file.

Return type:

Path

Raises:

ValueError – If file with this suffix already exists.

get_all_annotation_layers(new_annotation_suffix: str = None)

Collect all annotation sources for loading into DUSTrack.

Parameters:

new_annotation_suffix (str, optional) – Suffix for a new layer to create.

Returns:

Maps layer names to file paths, including:

• Existing JSON annotation files

• New empty layer (if suffix provided)

• Labeled training data

• DLC prediction HDF5 files

Return type:

dict

dustrack._file_management._extract_frames(video_file_name: str, frame_idx: list, output_path: str, coords: list)

Legacy frame extraction using DLC’s VideoWriter (OpenCV-based).

Note

This function is kept for backwards compatibility but _extract_frames_decord is now used by default for better performance, and because of discrepancy in extracted frames (seeking issues) when using OpenCV vs decord.

Parameters:

• video_file_name (str) – Path to video file.

• frame_idx (list) – Frame numbers to extract (0-indexed).

• output_path (str) – Directory to save extracted frames.

• coords (list) – Crop coordinates [x, y, width, height].

Returns:

Paths to saved image files.

Return type:

list

dustrack._file_management._extract_frames_decord(video_file_name: str, frame_idx: list, output_path: str, coords: list)

Extract video frames using Decord library for better performance.

This is the default frame extraction method. It uses batch reading for better I/O efficiency compared to OpenCV sequential reading.

Parameters:

• video_file_name (str) – Path to video file.

• frame_idx (list) – Frame numbers to extract (0-indexed).

• output_path (str) – Directory to save extracted frames.

• coords (list) – Crop coordinates. Interpreted as: - [x1, y1, x2, y2] if values look like absolute corners - [x, y, width, height] otherwise

Returns:

Paths to saved image files.

Return type:

list

Note

Skips extraction if image file already exists. Handles invalid frame indices gracefully.

dustrack._file_management.get_annotation_file_name(video_file_name: Path, annotation_suffix: str = '') → str | None

Get full path to annotation file if it exists.

Parameters:

• video_file_name (Path) – Video file path.

• annotation_suffix (str) – Annotation suffix (e.g., ‘manual’, ‘refined’).

Returns:

Full path if file exists, None otherwise.

Return type:

str or None

dustrack._file_management.make_annotation_file_name(video_file_name: Path, annotation_suffix: str = '') → str

Construct annotation filename from video filename and suffix.

Parameters:

• video_file_name (Path) – Video file path.

• annotation_suffix (str) – Annotation suffix. Empty string means no suffix.

Returns:

Full path to annotation file (may not exist yet).

Return type:

str

Example

>>> make_annotation_file_name('video.mp4', 'manual')
'video_annotations_manual.json'
>>> make_annotation_file_name('video.mp4', '')
'video_annotations.json'

dustrack._file_management.merge_annotations_in_folder(path, annotation_suffix='merged')

Merge multiple annotation files for each video in a folder.

Useful for combining annotations from multiple annotators or sessions. Creates a single merged JSON file for each video.

Parameters:

• path (str) – Directory containing videos and annotation JSON files.

• annotation_suffix (str) – Suffix for merged output files. Defaults to ‘merged’.