search

jacksonlcrews / darkice

Automatically exported from code.google.com/p/darkice
0 stars 0 forks source link

Not using all CPU resources when needed, just 1 CPU max #47

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. use many encoders
2. look at cpu loading
3. notice that it does not go above 100% on 8 core machine

What is the expected output? What do you see instead?
Above 100% usage when needed

What version of the product are you using? On what operating system?
Latest HEAD

Please provide any additional information below.
The MultiThreadedConnector.cpp does not allow for parallel running of encoders. 
The "MultiThreadedConnector::sinkThread" run in a pipeline, not parallel, which 
is easily fixed.

Every encoder (consumer) does a :
pthread_mutex_lock(&mutexProduce);

And only releases it when it is done with encoding.
This means only 1 encoder can run at same time.

Also the consumers (MultiThreadedConnector::sinkThread)
send a broadcast report that the PRODUCER HAS DATA ...
This makes no sense to me, also the PRODUCER waits on this signal from 
consumers to indicate that the consumers are finished with work...
This makes no sense semantically to me at least. You would use 2 condition 
variables for this:
1) report to all consumers via broadcast that data is available. condVarProducer
2) report to producer that one consumer has finished, the producer can keep 
score on when everyone is done

I am not including a fix right now, but will be fixing this soon.

Original issue reported on code.google.com by oetelaar.automatisering on 13 Mar 2011 at 9:25

GoogleCodeExporter commented 9 years ago 
Any progress on this?

Original comment by dhruvah...@gmail.com on 29 Nov 2011 at 12:50

GoogleCodeExporter commented 9 years ago 
I'd really like to see a patch implementing this idea Edwin!

Original comment by rafael2k...@gmail.com on 16 Jan 2012 at 9:21

GoogleCodeExporter commented 9 years ago

Original comment by rafael2k...@gmail.com on 16 Jan 2012 at 9:29

GoogleCodeExporter commented 9 years ago 
I will have to look up how I fixed this, but I will post a patch soon. OK?

Original comment by oetelaar.automatisering on 16 Jan 2012 at 9:29

GoogleCodeExporter commented 9 years ago 
Great.
Thanks Edwin.

Original comment by rafael2k...@gmail.com on 16 Jan 2012 at 9:31

GoogleCodeExporter commented 9 years ago 
Here is some preliminary patch, I have not tested it at all.
Can someone please test if it applies cleanly and run some tests please. Do not 
integrate without real testing.

Original comment by oetelaar.automatisering on 2 Feb 2012 at 8:50

Attachments:

GoogleCodeExporter commented 9 years ago 
About my last patch, *NOT GOOD* do not use it.
Forget it. I am fixing it now

Original comment by oetelaar.automatisering on 30 Mar 2012 at 11:39

GoogleCodeExporter commented 9 years ago 
Now I had to fix it.
So I did, please test on your system too.

Changes :
- changed byte counter of source from int to unsigned long int
- changed MultiThreadedConnector to run parallel tasks instead of sequential
- changed some locks/mutex stuff to allow for parallel 
- done some testing on it to prove it works
- fix null-pointer ref in CastSink.h while closing a FileSink at program end

Attaching screenshots of before and after analysis, and a patch!

Original comment by oetelaar.automatisering on 30 Mar 2012 at 7:04

Attachments: