​

Labellerr: Segment Anything Model 2 (SAM 2) Video Object Tracking Documentation

​

Introduction

​

Key Features

• AI-driven, frame-by-frame object tracking
• Compatible with multiple label types (player, ball, etc.)
• Combines manual annotation, point prompts, and automated tracking
• Efficiently handles occlusions and out-of-frame moments

​

Step-by-Step Guide: Using SAM 2 for Video Object Tracking

​

1. Preparing the Video and Labels

• Upload your video to the Labellerr platform.
• Play the video briefly to familiarize yourself with its content.
• Create labels for each object you want to track (e.g., player, ball).

​

2. Annotating Key Frames with SAM 2

​

a. Select Object and Annotation Tool

• Select the relevant label (e.g., ball).
• Click the Magic Brush and choose Segment Anything Model 2 (SAM 2) from the dropdown menu.

​

b. Add Point Prompts

• Click the interact icon to begin.
• Place point prompts on the object to help the model segment it accurately.
• Confirm the segmentation by clicking the tick icon.

​

c. Track Object Across Video

• Right-click on the object and select SAM 2 Tracking.
• SAM 2 automatically tracks and segments the object across the entire video timeline.
• Repeat for additional objects (e.g., select player and follow the same steps).

​

3. Reviewing and Adjusting Tracks

​

a. Timeline Visualization

• Tracked objects appear on a colored timeline.
• Colored dots indicate present, absent, or segmentation error.
• Gaps in the timeline indicate occlusion or the object moving out of frame.

​

b. Handling Occlusions & Errors

• If SAM 2 misses a frame:
  • Right-click and select Mark Out of View (if object is absent).
  • Right-click and select Mark in View (if object is present).

​

c. Seamless Results

• Play the labeled video to preview accurate object tracking across all frames.

​

4. Customizing Annotations (Optional)

• Use bounding boxes for simple shapes (e.g., players).
• Use polygons for small or irregular objects (e.g., balls).
• Add attributes (e.g., player activity, jersey color) and video-wide classifications for detailed datasets.

​

5. Firing Multiple Annotation Jobs

​

How It Works

1. Upload your videos and create object labels.
2. Select objects to annotate.
3. Click the SAM 2 button in the object header.
4. Choose the object for this job.
5. Click Run to automatically track and segment the object across all frames.

​

6. Running SAM 2 on a Single Timeline

​

How It Works

• Upload your video and create labels for all objects.
• Select each object to include in the timeline job.
• Right-click on the timeline, choose SAM 2 Tracking, and select Run SAM 2.
• Monitor progress in the Timeline section, where colored dots indicate object presence, absence, or segmentation errors.

​

Benefits

• Dramatically faster: Reduces hours of manual tracking to just a few clicks.
• High accuracy: AI leverages previous frame information to minimize drift and errors.
• Flexible: Suitable for any moving object in videos—sports, robotics, animals, retail, etc.
• Easy error correction: Quick manual fixes via right-click if predictions are off.