search

jlesage / docker-handbrake

Docker container for HandBrake
MIT License
822 stars 95 forks source link

Help needed getting hardware acceleration started #40

Open ChrisWinGerm opened 5 years ago

ChrisWinGerm commented 5 years ago

Sorry for asking. I'm new in Synology Docker experience. So likely it's an absolute silly question.

First of all: The experience with your HandBrake solution is great so far (after an longer time with problems caused only by myself until I got the conclusion between your written down start command and the transfer to the synology docker GUI). Thank you so much!

So I wanted to do the next step now. I own a DS218+ with Intel processor listed for hardware acceleration.

My stupid question is: where do I have to write down the command "--device /dev/dri" in the Docker GUI?

As there was no clue for me, I tried this command via ssh form my pc:

sudo docker run -d --name=handbrake --device /dev/dri -e USER_ID=1026 -e-GROUP_ID=1000 -p 5803:5800 -v /volume1/homes/christian/HandBrake/config:/config:rw -v /volume1/homes/christian/HandBrake/storage:/storage:ro -v /volume1/homes/christian/HandBrake/watch:/watch:rw -v /volume1/homes/christian/HandBrake/output:/output:rw jlesage/handbrake Took me a longer time to realize that I have to make sudo to make it. But after all a new container was started. Poorly when I open the HandBrake-GUI nevertheless there is no "H.264 (Intel QSV)".

Your advice is to check the container's log for the issue. Unfortunately due to lack of knowledge I'm not able to see what's the reason.

Could you give me a hand?

Mevertheless my prefered way would be using the GUI because I'm not that firm with command line.

handbrake_Prozesslog_Hardwarebeschleunigung.html.pdf

RealCommerzpunk commented 5 years ago

Hello, maybe i can help you or give you some information at least. Btw. i am struggeling with similar problems. I have hardware acceleration running with almost the same configuration. My Syno is a 718+ modell.

Look at my questions here: https://github.com/jlesage/docker-handbrake/issues/32

This is my command to run the container from a planed Synology Task:

!/bin/bash

sudo docker run --name handbrake2 --net=host --restart always --volume /volume1/docker/handbrake/config:/config:rw --volume /volume1/NeueFilme:/output:rw --volume /volume1:/storage:ro --publish 5800:5800 --env USER_ID=0 --env GROUP_ID=101 --device /dev/dri jlesage/handbrake:dev-latest

I am sorry, for my knowledge there is no way to pass the "--device /dev/dri" command in the Synology GUI, thats why i use the command - but got problems with that too.

I am from germany too, active in the kodinerds forum: https://www.kodinerds.net/index.php/User/35876-Commerzpunk/

Maybe you wanna go there to discuss in german? There are a few guys with knowledge in docker and some with handbrake too.

jlesage commented 5 years ago

Could you also provide the following log file: /volume1/homes/christian/HandBrake/config/ghb/Activity.log.XXXX.

ChrisWinGerm commented 5 years ago

Thanks RealCommerzpunk.

Here you are jilesage. Thank you very much. Activity.log.970.pdf

jlesage commented 5 years ago

Can you try to re-create the container, but instead of using USER_ID=1026, try with USER_ID=0.

ChrisWinGerm commented 5 years ago

Thank you very much. Works for me to get option "H.264 (Intel QSV)".

But the same issue occurs that RealCommerzpunk described: I have not even one video in the watch folder actual but the cpu is running like on the run.

handbrake_on handbrake_stopped

.

jlesage commented 5 years ago

So you only changed the value of USER_ID and see the CPU usage bump? Can you try to re-create the container, but this time without the --device /dev/dri parameter?

ChrisWinGerm commented 5 years ago

I did as you told me

sudo docker run -d --name=handbrake2 -e USER_ID=0 -e-GROUP_ID=1000 -p 5803:5800 -v /volume1/homes/christian/HandBrake/config:/config:rw -v /volume1/homes/christian/HandBrake/storage:/storage:ro -v /volume1/homes/christian/HandBrake/watch:/watch:rw -v /volume1/homes/christian/HandBrake/output:/output:rw jlesage/handbrake

and - wow - CPU is up to ~38%

So the same behavior as with "device"

jlesage commented 5 years ago

Ok so the issue seems to be related to the fact that the container is running as root.

RealCommerzpunk commented 5 years ago

Alright, lets go on here because i think this issue is somewhere arround "user" and "hardware acceleration". My issue still isnt solved.

I tried a lot last night, put literally everything upside down and got a whole new setup.

What i did NOT:

What ever i tried

Its not working all together, one issue is always left.

Finally i am at the very first point again, acceleration is working, i can transcode - but only one file and a high CPU usage. So in the end: A whole new setup and the same issues.

Ideas?

RealCommerzpunk commented 5 years ago

Ok so the issue seems to be related to the fact that the container is running as root.

No, its not. My container runs with "1024" and "101" which is my admin and the administrators group: And its not in "privileged" mode.

admin@nas-27:~$ id uid=1024(admin) gid=100(users) groups=100(users),101(administrators)

jlesage commented 5 years ago

When not running as root, do you see high CPU usage at idle?

RealCommerzpunk commented 5 years ago

I am pretty sure yes, but can tell you for sure in about an hour when my latest recording is transcoded. ;-)

jlesage commented 5 years ago

I found an issue with the auto video converter where a loop is created when running as root. This certainly explain the high cpu usage. I'm working on a fix.

RealCommerzpunk commented 5 years ago

I am very glad to hear this! Good luck.

Here is my update: My container runs with IUD "1024" and GID "101" which is my admin and the administrators group: And it was "privileged" mode. (which is the same as "run as root" as i understand it).

When i run it not in privileged mode, i dont have the QSV options, so HW acceleration is not availible.

Lets see what you come up with!

ChrisWinGerm commented 5 years ago

Sound's great, jilesage. Thank you!

jlesage commented 5 years ago

And it was "privileged" mode. (which is the same as "run as root" as i understand it).

Running as root and privileged mode are 2 different things. Running as root means that the processes in the container run as user ID 0. This doesn't affect privileges of the container itself nor its capabilities.
Activating the privileged mode gives all capabilities to the container, provide full access to all devices on the host, etc.

jlesage commented 5 years ago

There is a new image available. It should fix the CPU usage problem.

ChrisWinGerm commented 5 years ago

Thank you very much. I'll test it in the evening when back from work. I will report.

RealCommerzpunk commented 5 years ago

Thank you so much.

latest or dev-latest tag to get this fix?

jlesage commented 5 years ago

latest tag should be used.

ChrisWinGerm commented 5 years ago

Updated. Looking very good so far. After start CPU is rising up to 17 % for a moment and immediately going down (no video in watch). And...."H.264 (Intel QSV)" is selectable in HandBrake.

handbrake_hw_acc_neu

And further on. Finaly now on 0,6 % on idle.

Will try transcoding now.

RealCommerzpunk commented 5 years ago

I did the update too > latest running in bridged mode,priviledged, UID 0

good:

not good:

See these 2 logs, its the exact same file (but copied, so 2 seperated input files). Starting a queue with the exact same settings and fired. No. 1 did work and created a output, No. 2 did not even create a output video file. The last line is "libhb: work result = 3" Googling a little bit about it showed some information about permission issues. Maybe thats a hint?

1SampleVideo_1280x720_20mb 2018-11-13 17-20-04.log

2SampleVideo_1280x720_20mb 2018-11-13 17-21-12.log

ChrisWinGerm commented 5 years ago

Mmm. Seems I do something wrong. Nothing happens.

This is the way I did:

Started Container.

Created a new preset and set it on default in the HandBrake GUI.

Stopped the container.

Wrote the name of the new preset into Automated_Conversion.

Started the container.

Nothing happens except for a moment CPU goes up i red zone and than down to I guess idle.

container handbrake_voreinstellung Activity.log.982.pdf handbrake_Acceleration.html.pdf

ChrisWinGerm commented 5 years ago

So got it running at least. Took the Preset in the terminal:

sudo docker run -d --name=handbrake1 --device /dev/dri -e USER_ID=0 -e GROUP_ID=1000 -e AUTOMATED_CONVERSION_PRESET=Hardware_Acceleration -p 5800:5800 -v /volume1/homes/christian/HandBrake/config:/config:rw -v /volume1/homes/christian/HandBrake/storage:/storage:ro -v /volume1/homes/christian/HandBrake/watch:/watch:rw -v /volume1/homes/christian/HandBrake/output:/output:rw jlesage/handbrake

Running now. Curious to see the result tomorrow. GN8@all

jlesage commented 5 years ago

@RealCommerzpunk, I think the issue you have about not being able to encode 2 files is related to the QSV H265 encoder. Do you have the same problem with the H264 version? Also, do you have the issue with the automatic video converter? For the CPU issue, can you check which process is taking the CPU? By the way, there is no need to run the container with privileged mode.

jlesage commented 5 years ago

@ChrisWinGerm, are you all setup now?

ChrisWinGerm commented 5 years ago

Starting and running good so far. 👍(Had no queue like RealCommerzpunk at the moment. So I can't report to that issue). Next will be to make some comparison of different presets in Handbrake and their effects on quality, duration and workload.

Thank you very much for your dedicated work! Is there a piggybank to reveal?

RealCommerzpunk commented 5 years ago

@jlesage You are right. Using the WebGUI it only fails to encode more then one file in a queue when using H265 QSV. H264 QSV works and few others i tried.

But now i am stunned. Using the watch folders, its even processing more then one file! Using the same preset which uses h265 qsv of course!

One more thing: When ever a conversion fails this way, the h265 qsv codec seems to be blocked. Only after restarting the container its possible to encode one file again. This happens to the watch folders too. So if the codec is blocked, even watch folders dont work, failing too. After a fresh start of the container several files can be transcoded via watch folders.

Does that help you to catch this nasty bug? Anything i can help? Logs, testing scenarios?

jlesage commented 5 years ago

Can you try again, but by adding the HANDBRAKE_DEBUG=1 environment variable when creating the container? Reproduce the issue and please provide log/hb/handbrake.debug.log and ghb/Activity.log.XXX. These files are in the folder you mapped to /config.

RealCommerzpunk commented 5 years ago

Sorry, that took a while. @jlesage You told me there is no need to run the container with privileged mode. For me there is a Need, because if i dont, there is no QSV Option availible.

Today i finally did the logs you asked for, see attached.

handbrake.debug.log Activity.log queue.log

Activity.log and Queue.log have be renamed be me to upload them. Before the Suffix was "991".

Here is a screen what it Looks like after finishing a Job and using too much CPU:

handbrake

jlesage commented 5 years ago

You told me there is no need to run the container with privileged mode. For me there is a Need, because if i dont, there is no QSV Option availible.

Then there is probably a permission issue on the host with the device. Make sure you have the latest image and look at the container log during the startup: it should indicate why QSV can't be used.

For the QSV H265 issue, I suspect it's an issue/bug with HandBrake (i.e. not sure it's an issue with the container). I can try to report the issue to the HandBrake team and see what they think about this. I would use your log if you don't mind, since I can't reproduce the issue (I don't have a CPU supporting H265 QSV).

jlesage commented 5 years ago

Also, could you confirm that QSV H265 issue is reproducible with the latest nightly/development build (v1.14.9-dev-20181124-2b2bf33f47)?

jlesage commented 5 years ago

@RealCommerzpunk, can you try the tag v1.14.10-beta1 to see if it fixes the H265 QSV issue?

RealCommerzpunk commented 5 years ago

Hello, i made a all fresh container with the v1.14.10-beta1 image. All new even a new empty config folder. Running as UID 0, priviledged mode, bridge network.

Took the default preset, just changed it to MKV and H265 QSV, grabbed my 2 testfiles, put them in a queue.

Same issue as before, first one succeeds, second fails. Activity.log.txt queue.txt

Logs attached.

jlesage commented 5 years ago

Thanks for the report. Note that I also pushed the v1.14.10-beta3 image. This version should also fail, as confirmed by another user, but if you can also confirm this it would be great.

RealCommerzpunk commented 5 years ago

Yes, it its exactly the same with beta3 imaga. Same procedure as before.

jlesage commented 5 years ago

Could you now try with v1.14.10-beta5?

RealCommerzpunk commented 5 years ago

Hey! That did it! Beta 5 did run my 2 testfiles from the Queue in the row! First time since month! Cool!

Any logs you Need?

jlesage commented 5 years ago

Thanks for the verification! Logs not needed. An official build will be available today :)

muellerk22 commented 3 years ago

For those not wanting to use the docker CLI, I got this working following this guide: https://wildestpixel.co.uk/enabling-quick-sync-video-on-handbrake-for-docker-on-synology-2

I too am a big fan of the container and thank you @jlesage .

cypekis11 commented 2 years ago

@muellerk22 Could you repost the link? I did manage to unlock the h.264 QSV and h.265 QSV encoding, however only h.264 works and the other crashes at the beginning...

IMG-20220511-WA0001

facetto commented 2 years ago

I did manage to unlock the h.264 QSV and h.265 QSV encoding, however only h.264 works and the other crashes at the beginning...

IMG-20220511-WA0001

Hi @cypekis11 , have you been able to solve the issue? Looks like I am stuck exactly like your log shows above. I am pretty sure it´s only a small step from there, however so far I didnt didnt a solution.

Thanks for a hint, BR, Christoph

facetto commented 2 years ago

Finally I was able to activate QSV with H.265 as well. It was the lowpower=0 setting which for some reason I didnt do right although I dont understand what now I didnt differently...

BR, Christoph

jaromirrivera commented 2 years ago

@facetto Where did you set that lowpower=0 option?

facetto commented 2 years ago

In the handbrake GUI export a preset, open it in an editor, search for the variable "VideooptionExtra" or so and add "lowpower=0". search the web for what exactly you have to enter. safe the preset file and import it in the handbrake GUI.

BR, Christoph