Behavior video standards

An encoding standard, and transcoding 230 TB of archival video to it

High-speed behavior cameras at the Allen Institute produce a lot of video. Much of the archive was stored as large AVIs with inconsistent encoding and colour-range metadata, which made it expensive to store and easy to misread in analysis.

With Bruno Cruz, I wrote the institute’s behavior video file standard, which sets how video is encoded for storage and adds preview clips and poster images for browsing. I also wrote its implementation, aind-video-utils, which keeps the encoding profiles in one place and provides probing, colour-range-correct frame extraction, and command-line tools for transcoding and QC.

Transcoding the archive

I then converted the existing archive in the institute’s public data bucket:

  • About 11,100 videos (230 TB, 19 billion frames, 14,400 hours) were transcoded to standard-compliant H.264 MP4.
  • The encoder settings, chosen by measuring quality and speed, match libx264’s veryslow preset on VMAF, PSNR and SSIM at 2.6× its speed. Output takes about a third of the original space.
  • The job ran on an EC2 Spot Fleet in about 4.5 days for about $1,350, and produced an automatic QC figure for every video.