search

livepeer / go-livepeer

Official Go implementation of the Livepeer protocol
http://livepeer.org
MIT License
546 stars 171 forks source link

Orchestrator processes only one segment at time #886

Open darkdarkdragon opened 5 years ago

darkdarkdragon commented 5 years ago

Describe the bug Orchestrator was made with assumption that segments coming in with realtime speed, in steady manner. That means, that orchestrator always have segment length seconds to process segment before next one arrives. But there are legitimate cases - see here (and probably it can happen in real networks, with data buffer somewhere along the way and then arrived to broadcaster in big batch) - when this is not the case, and we don't handle these cases.

This will not allow us to achieve our goal of 99.999% reliability, especially given that we're also aiming at enterprise user's use cases.

j0sh commented 5 years ago

We have to also take into consideration the assumptions we're making for the operating environment.

  1. Is the transcoder capable of processing segments in faster-than-realtime? Or if not,
  2. Do you have more than one transcoder?

If neither of these are true, then the transcoding environment is not appropriate for live streaming. There's only so much we can (or should) do to fix that. That includes ensuring the ingest setup is reliable.

Now, there are cases where we may indeed want to support "batch" transcoding, eg VOD. But that calls for a different type of workflow, and right now I feel pretty strongly that we should optimize for those cases separately from the live workflow.