search

madebyhiro / codem-transcode

Offline video transcoder written in node.js
Other
153 stars 68 forks source link

Encoder locking up on job request, stalling scheduler #35

Closed scottf-tvw closed 10 years ago

scottf-tvw commented 10 years ago

jobs/$id call locking up and does not respond, I am not getting any further output from the stdout watching the encoder. I have about 4300 jobs in cue from the scheduler and 2 encoders. one encoder continues to lock up on curl requests on specific job id requests. the regular /job request works fine.. it never times out.. I would like to move the db to mysql but I could never get the packages to jive on ubuntu 14 and get it working. since I am in a crunch on this, I would be willing to use pay services to get this resolved quickly..

scottf-tvw commented 10 years ago

I found out that this only happens if the transcoder system runs out of memory, which it seems to do very quickly using codem transcoder. Node.js is eating up all the memory.

scottf-tvw commented 10 years ago

So an update, the version of node that installs with apt on ubuntu, gc is not working well with codem-transcoder. I moved to 0.8.2X and it seem to be working fine now..

scottf-tvw commented 10 years ago

still doing it with a large volume of jobs in the cue on node 0.8.2X. at a loss here on how to keeps these encoders going and responding.. node just keeps creeping up in memory usage when running codem-transcoder until it stops responding to api calls for specific job id's, eventually it chokes the entire system.

tieleman commented 10 years ago

Hi Scott, sorry to hear about your problems. In general, we recommend using the latest stable branch of Node (currently 0.10.x). I'm not sure what version of Ubuntu you're using, but there are 0.10 packages available for both 12.04 LTS (via the NodeJS repository) and 14.04 LTS (via the standard repositories).

I've not heard of memory leaks in codem-transcode before, but of course it might be possible there are. Can you check with 0.10 for me and if the problem is still there on 0.10 please let me know. We've had transcoders running for weeks/months without any memory issues.

scottf-tvw commented 10 years ago

I am using Ubuntu 14.04 LTS. I decided to switch from sqlite3 to mysql and it's running better and I removed all but 20 at a time in the cue. so far the encoder is responsive and the memory usage for node is hovering at about 1100000. it is creeping up though from 90000 when it started running over a period of about 2 hours. the encoder is at 10 slots full pretty much 24/7. I will light up another fresh copy and use the latest node using mysql with codem-transcoder and see how it performs.