Using the Utility

Clipping and stitching GoPro videos from SEFIS video surveys

Two tips before getting started:

It is recommended to create a new branch for any changes or experiments. This allows you to work without affecting the main branch and helps minimize version conflicts in the future.
Do not modify the configurations-template.yml file provided in the repository. Instead, make a copy of this file to preserve the version-controlled template for future use. Newly created configurations will be ignored by Git version control.

There are multiple ways to use this utility depending on your familiarity with Git, R, Python, and/or command line. In order of increasing technical complexity, the options are:

No command line

The easiest way to launch the utility on Windows is to use the process_videos.bat file. You will be prompted to select the configurations YAML file to use, and the script will then run according to the settings specified in that file.

Alternatively, the process_videos_dragndrop.bat file will open a Windows Command Prompt terminal and prompt you to drag and drop the configurations YAML file into the terminal. Open a File Explorer window, navigate to the desired YAML file, and drag it into the terminal as instructed. The script will then run according to the settings specified in that file.

In either case, status updates, errors or warnings, and instructions will be printed in the terminal window that opens when the file is run, so be sure to check there if you are expecting output of any kind.

RStudio and R

For users comfortable using R and RStudio, the process_videos.R script can be run directly from RStudio. Open the project in RStudio, set your working directory to the project directory (where the scripts are located), and run the process_videos.R script. You will be prompted to select the configurations YAML file to use, and the script will then run according to the settings specified in that file.

A note for R users

The process_videos.R script is a launcher/wrapper script that runs the Python video processing utility process_videos.py. It is pre-configured to use the “Private Library” (Virtual Environment) created during setup. You don’t need to manually activate anything; just double-click the Rscript file and provide your configuration file when prompted.

Python and command line

Users comfortable with command line and Python can run the process_videos.py script directly from the command line.

Open a command line terminal and navigate to the project directory. Activate the Python virtual environment created at setup depending on your operating system and the method you used to create the virtual environment.

For pip environments:
```
pip environment in Windows Command Prompt
```
```
.venv\Scripts\activate
```
```
pip environment in Git Bash
```
```
source .venv/Scripts/activate
```
```
pip environment in Linux or Mac Terminal
```
```
source .venv/bin/activate
```
Or, for conda environments:
```
conda environment
```
```
conda activate video-processing-venv
```
Execute the process_videos.py script, passing the path to the desired configurations YAML file as an argument:
```
python process_videos.py path/to/config.yaml
```
No other arguments are needed. The script will run according to the settings specified in the YAML file.

When finished, deactivate the virtual environment:

pip environment

deactivate

conda environment

conda deactivate

Want audio?

The script clip_and_stitch.py does not include audio in the stitched output video. To include audio in the new video, use clip_and_stitch_waudio.py instead. Everything else remains the same, including the configuration settings.

Troubleshooting

The default setting for quality_crf, which sets the video encoding quality during processing, has been chosen to minimize the amount of visual video quality loss between the original “chaptered” videos and the final stitched video. Performance, however, will likely be machine- and hardware-dependent. To help the user quantitatively assess how well the output video aligns with that of the original, the utility prints to the screen and to the log file an estimated file size for the output video, the actual file size of the output video, the average bits per pixel (BPP) of the original videos that were used to create the output video, and the BPP of the output video. These metrics can be used as follows:

The expected output file size is estimated by multiplying the bit rate (\(b\); bits per second), by the duration (\(d\)) of the original video segment for each video “chapter” that contributes to the output video. For partial (i.e., clipped) videos, the duration of the retained clip is used. Converted to bytes, these values estimate the file size of each video chunk assuming no quality loss. Adding these together for all \(N\) video segments gives an estimate of the expected file size of the output video assuming no quality loss during processing: \[\text{expected size} = \sum_{i=1}^{N} \left( \frac{b_i d_i}{8} \right)\]

Compare this expected size (reported in GB) to the actual file size of the output video:
- An actual file size much smaller than the expected file size is indicative of a much lower video quality than the original videos. In this case, try decreasing the quality_crf setting in the configuration YAML file and re-run the utility. Note that lowering the quality_crf value will increase the file size of the output video and may also increase processing time.
- An actual file size much larger than the expected file size is indicative of a much higher video quality than the original videos and an unnecessarily large file size. In this case, try increasing the quality_crf setting in the configuration YAML file and re-run the utility. This will decrease the file size of the output video and may also decrease processing time.
Important

Remember that video quality is inversely proportional to quality_crf: lower values mean higher quality and larger file sizes, while higher values mean lower quality and smaller file sizes.
The bits per pixel (BPP) is a metric for quantifying the “entropy” or the amount of information in each video frame. High BBP indicates a complex scene with high information density, such as lots of fish, movement, low turbidity, or abundant light. Conversely, low BBP indicates a simpler scene, which the encoder is able to compress more heavily without losing quality. BPP is calculated by dividing the bit rate (\(b\)) by the product of the frame rate (\(f\)) and the total number of pixels in each frame (\(p\)):

\[\text{BPP} = \frac{1}{N}\sum_{i=1}^{N} \frac{b_i}{f p_i}\]

where \(p\) is the product of the video frame width and height in pixels. BPP is averaged over all \(N\) frames for a given video. In the case of the input video, it is averaged across all input segments. Compare the BPP of the output video to the average BPP of the original videos. As with file size, increase quality_crf if the output BPP is much higher than the original BPP, and decrease quality_crf if the output BPP is much lower than the original BPP.

NoteA note on the BPP “sweet spot”

Because the libx264 (CPU) encoder used in the utility is generally more efficient than the GoPro’s internal hardware, one will often find that an output file with a lower bitrate than the original actually contains the same amount of qualitatively visual information. Your target shouldn’t necessarily be an identical file size, but rather a file size and BPP that stays within 80-90% of the original in order to maintain “visually lossless” quality. The utility will display whether the file size and BPP are within 80% or 90% of the original, so use this as a guide when adjusting quality_crf.
The timestamps in the log table indicate where in the original and new videos to look to compare frames for assessing whether stitching worked as expected. Be on the lookout for jumps in the video scene or repeated scenes across these time marks, both of which may indicate a stitching error.