shaka-project / shaka-streamer

A simple config-file based approach to preparing streaming media, based on FFmpeg and Shaka Packager.
https://shaka-project.github.io/shaka-streamer/
Apache License 2.0
198 stars 62 forks source link

feat(cloud): Adding an upload proxy node #103

Open mariocynicys opened 2 years ago

mariocynicys commented 2 years ago

This PR adds an upload proxy node. A state aware node that handles the work of uploading content to a remote location on behalf of Shaka Packager. This PR adds no new dependencies.

Issues:

joeyparrish commented 2 years ago

My -H "Authorization=Bearer $(gcloud auth print-access-token)" hack failed after about one hour of running. I started getting HTTP 429 responses from Google Cloud, which makes me think it wasn't an issue with the token itself, but something more fundamental we will need to resolve.

Google's docs say:

If you run into any issues such as increased latency or error rates, pause your ramp-up or reduce the request rate temporarily in order to give Cloud Storage more time to scale your bucket. You should use exponential backoff to retry your requests when:

Receiving errors with 408 and 429 response codes. Receiving errors with 5xx response codes.