search

pytorch / torchcodec

PyTorch video decoding
BSD 3-Clause "New" or "Revised" License
87 stars 10 forks source link

Added benchmark chart to README.md #315

Closed ahmadsharif1 closed 1 month ago

ahmadsharif1 commented 1 month ago

This PR:

  1. Adds some code to generate synthetic videos.
  2. Adds a script that the user can run to generate a png file with benchmark results comparing torchcodec with torchvision and torchaudio. It also compares it to torchcodec with 1 thread. The benchmark is run on 3 seeking patterns -- just getting the first frame, getting 10 consecutive frames and seeking uniformly and decoding a frame after seeking.
  3. Adds this figure to the readme.
scotts commented 1 month ago

This is great! Are we also okay with comparing to Decord?

NicolasHug commented 1 month ago

Should we also compare a call to .get_frames_at_indices (or pts)? I suspect torchcodec will shine compared to (at least) torchvision?

scotts commented 1 month ago

I'm happy to commit this version and work on follow-ups later. I think we'll probably spend a bunch of time tweaking the presentation of the graph, and iterating on the exact experiments we run.

ahmadsharif1 commented 1 month ago

I merged this as is.

I'll create an issue for follow-up work on benchmarking -- throughput benchmarks, sampler benchmarks, better video selection, etc.