search

svt / encore

Transcode media files in an epic manner
European Union Public License 1.2
273 stars 25 forks source link

how to add processing nodes ? #12

Closed deedos closed 3 years ago

deedos commented 3 years ago

Hi, I could not find at the docs how to add encore "relay" nodes to descentralize processing. Is there a way to do it ? thanks a lot

janderssonse commented 3 years ago

Hi, there are much improvements for more descriptions in the documentation, (additions are welcomed). . Basically, there is not much work - just startup Encore instances (nodes) and point them to you main Redis setup (sentinel?/single?cluster?)-setup and Redis will handle the queue distribution work from by making sure each encore instance/node that is free and requests jobs gets "the next one in the queue"- so just add encore instances/nodes as you wish to scale. It is briefly described under "Scaling, distribution and queues" in the https://svt.github.io/encore-doc/#concepts

(Additional: Each encore instance/node can also by itself be fine tuned to how many jobs it will concurrently run on each instance/node with the concurrency settings , which default is set to max 3 (2 really, it is 0-indexed)). The default should be fine though so you should not need to change this.).

deedos commented 3 years ago

Thanks a lot. You'll definitely see doc additions by me !Thanks for putting together such a nice project. PS: Planning/Studying to replace the discontinued codem (madebyhero, scheduler and transcode) for in house transcoding at the public broadcasting here in Brazil.

Em ter., 30 de nov. de 2021 às 03:27, Josef Andersson < @.***> escreveu:

Hi, there are much improvements for more descriptions in the documentation, (additions are welcomed). . Basically, there is not much work - just startup Encore instances (nodes) and point them to you main Redis setup (sentinel?/single?cluster?)-setup and Redis will handle the queue distribution work from by making sure each encore instance/node that is free and requests job gets it - so just add encore instances/nodes as you wish to scale. It is briefly described under "Scaling, distribution and queues" in the https://svt.github.io/encore-doc/#concepts

(Additional: Each encore instance/node can also by itself be fine tuned to how many jobs it will concurrently run on each instance/node with the concurrency settings https://svt.github.io/encore-doc/#application-properties, which default is set to max 3 (2 really, it is 0-indexed)). The default should be fine though so you should not net to change this.).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/svt/encore/issues/12#issuecomment-982326870, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABNFJHYLOAXIDYGY255QDLUORVF3ANCNFSM5I73XWGA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Daniel Roviriego (21) 972361654