truenas / charts

TrueNAS SCALE Apps Catalogs & Charts
BSD 3-Clause "New" or "Revised" License
305 stars 293 forks source link

Plex GPU Acceleration P2000 #2252

Closed sync-by-unito[bot] closed 2 months ago

sync-by-unito[bot] commented 8 months ago

Using official Plex container, GPU is recognised in Plex (lists the exact device in Plex nowadays. When playing a video, status changes to using hw decoding, however it only displays a blank screen and time counter does not move. Looking in the container nvidia-smi is working, detects the video card with correct ram etc, but is receiving nothing from Plex. I assume with the very limited options to configure that this should all work out of the box and it isn’t. So registering a bug here.

┆Attachments: image-20240322-175236.png | image-20240503-051946.png | image-20240503-052018.png | Screenshot 2024-03-02 at 15.26.14.png | Screenshot 2024-03-02 at 15.46.39.png | Screenshot 2024-03-07 at 16.38.43.png | Screenshot 2024-03-25 at 14.06.57.png | Screenshot 2024-03-25 at 14.11.56.png | Screenshot 2024-04-18 at 17.07.58.png

sync-by-unito[bot] commented 8 months ago

➤ Bug Clerk commented:

Thank you for submitting this TrueNAS Bug Report! So that we can quickly investigate your issue, please attach a Debug file and any other information related to this issue through our secure and private upload service below. Debug files can be generated in the UI by navigating to System > Advanced > Save Debug.

https://ixsystems.atlassian.net/servicedesk/customer/portal/15/group/37/create/153

sync-by-unito[bot] commented 8 months ago

➤ Daniel Pizappi commented:

Hi Marshalleq, can you please submit a debug from the affected system via https://ixsystems.atlassian.net/servicedesk/customer/portal/15/group/37/create/153 ( https://ixsystems.atlassian.net/servicedesk/customer/portal/15/group/37/create/153|smart-link )? Thanks.

sync-by-unito[bot] commented 8 months ago

➤ Marshalleq commented:

Done, also just noticing there is an option to completely isolate the GPU - which on unraid and also from what I read on TrueNAS is only needed when passing through to VM’s - but just raising it in case it’s different here.

sync-by-unito[bot] commented 8 months ago

➤ Marshalleq commented:

A few minutes prior to the debug generation, I ran up a couple of tries transcoding with hardware and got a new error message this time. Attached is screenshot plus evidence the card is detected in Plex gun.

!Screenshot 2024-03-02 at 15.26.14.png|width=485,height=250!

!Screenshot 2024-03-02 at 15.46.39.png|width=1363,height=1164!

sync-by-unito[bot] commented 8 months ago

➤ Bonnie Follweiler commented:

Thank you Marshalleq.

I have moved this ticket into our queue to review.

An engineering representative will update with any further questions or details in the near future.

sync-by-unito[bot] commented 8 months ago

➤ Stavros Kois commented:

Hello, can you please try some other app (eg Jellyfin)?

I can also verify as you did too, that it is passed to the container correctly and also it has NVIDIA_DRIVER_CAPABILITIES set to all, so its not really restricted. Also I didn’t find any issues with the nvidia plugin. Which is why I am asking you to try on another app, so we can see if its isolated to plex only. Maybe also try the non-plexpass variant?

Thanks!

sync-by-unito[bot] commented 8 months ago

➤ Marshalleq commented:

OK, good idea, I will try a few of those tests and come back to you.

sync-by-unito[bot] commented 8 months ago

➤ Marshalleq commented:

Just confirming Jellyfin appears to work, in that if I enable hardware acceleration in Jellyfin, play a video, set it to a lower resolution to enable transcoding, it still plays successfully and I can see in the nvidia-smi tool that Jellyfin is listed, utilisation goes up. When stopping playback of the video, the Nvidia-smi tool removes Jellyfin from its list of active processes. See Screenshot.

!Screenshot 2024-03-07 at 16.38.43.png|width=634,height=300!

sync-by-unito[bot] commented 8 months ago

➤ Stavros Kois commented:

Thanks for confirming, that means system side everything is good. The issue seems to be with Plex container, and I’m not sure what we could do about that, as long as it gets recognized inside the app and nvidia-smi inside the container also showing it, means that gpu is passed through.

Did you also try the non-plexpass variant? Not sure if the database schema is compatible tho.

sync-by-unito[bot] commented 8 months ago

➤ Marshalleq commented:

As I understand it, GPU acceleration is a Plex pass only feature. I guess I’ll move this one to a custom docker container too, it’s my last remaining official app as I’ve found Kubernetes to be extremely unreliable so far. I think I saw some survey recently where requesting docker was at or near the top of the list, this kind of situation I’d say is why.

Anyway, I’ll try custom image and let you know outcome as perhaps it helps you with something.

Thanks.

sync-by-unito[bot] commented 8 months ago

➤ Marshalleq commented:

Well, I’ve spent a bunch of time this afternoon trying a few things - the non plex pass container gives the same result. Using docker based images have some other weird permissions problem - nearest I can find is that possibly TrueNAS has a patched version of docker that requires a special v /run to be passed to it though I haven’t figured out exactly how to get past that, as it’s definitely not a permissions issue. Also, this GPU worked fine on unraid and is one of the most recommended plex GPU’s to get. If you think it’s not a system error, then perhaps it’s literally a bug with plex itself. Though I’m going to have a hell of a time proving it by myself with a TrueNAS container. I officially hate kubernetes lol. It just doesn’t seem to work well for this kind of containerisation. If I stick with TrueNAS I might just have to put all my apps in a VM, which would be a huge shame and waste of resources. I’m really trying to use TrueNAS rather than unraid.

sync-by-unito[bot] commented 7 months ago

➤ Stavros Kois commented:

TrueNAS uses containerd and from the debug I can see that the correct Runtime is selected for nvidia.

Can you try the new upgrade and see if there is any difference? (you might need to refresh your catalog first if you try today) Thanks!

sync-by-unito[bot] commented 7 months ago

➤ Marshalleq commented:

Oh right yes of course. Confusing when I am getting my images from a site called docker.io ( http://docker.io ) to remember I’m not using docker!

sync-by-unito[bot] commented 7 months ago

➤ Marshalleq commented:

OK done. Unfortunately it comes up with the same playback error. I do notice that the GPU is no longer selectable in the drop down list, Auto exists and I confirmed it is enabled for the app, but the drop down doesn’t drop down so I can’t select it if I wanted to. Auto should work anyway, but it doesn’t. Sorry, this didn’t work.

sync-by-unito[bot] commented 7 months ago

➤ Stavros Kois commented:

Yea “docker” is a lot of things, its an engine, is a registry, its a tool. The image comes from the docker registry, but consumed by the containerd engine.

Regarding the “its no longer selectable”. Do you mean in the Apps UI or in the Plex settings? Did you do any changes (besides the app upgrade) that caused it to change the display options?

sync-by-unito[bot] commented 7 months ago

➤ Marshalleq commented:

Stavros Kois no longer selectable in the plex app. It was still selectable in the truenas app config as I checked. I didn’t change anything else. Though each time I test it I have to enable the option in plex to offload transcoding to hardware obviously. Then I have to disable it afterwards or nothing transcodes.

sync-by-unito[bot] commented 7 months ago

➤ Stavros Kois commented:

Can you please open on a new tab on your browser the logs in Plex? It should be near the end of the Plex UI While you have this open, start a movie/something and see if there are any logs coming in. Maybe we are lucky and get something from there.

Thanks

!image-20240322-175236.png|width=279,height=242!

sync-by-unito[bot] commented 7 months ago

➤ Marshalleq commented:

Well this has proved interesting. I also enabled debug BTW.

It appears to be having a problem writing to both the transcode directory and the config directory, but only while transcoding via GPU, not while transcoding CPU - I double checked that. I also double checked that permissions are set to apps:apps recursively etc with read and write.

Here’s are a couple of screenshots.

!Screenshot 2024-03-25 at 14.11.56.png|width=1482,height=861!

!Screenshot 2024-03-25 at 14.06.57.png|width=1463,height=864!

In this console, while transcoding via CPU no errors are displayed either. Very strange. I do note that there is a new Nvidia driver in RC1 of the next TrueNAS version and am wondering if that’s worth a try.

sync-by-unito[bot] commented 7 months ago

➤ Stavros Kois commented:

Can you go to Plex Settings → Transcoder > Trascoder temporary directory and check what the value is there? It should be /transcode Then check on the TrueNAS side, for Plex app, the storage section named Transcode, see where it is mounted and check the permissions for that directory too

sync-by-unito[bot] commented 6 months ago

➤ Stavros Kois commented:

Closing this one for one. Let me know if you still having issues, we can take a look again. Be sure to answer my last question if you do have issues.

Thanks!

sync-by-unito[bot] commented 6 months ago

➤ Marshalleq commented:

Hi this wasn’t resolved, I have just been busy. I did check what you asked but must have forgotten to post it. So, therefore I have checked it again and I can see one thing that doesn’t look right.

It seems the ix systems plex container does not let me choose the container side of the transcode path (see Type Host Path in image). The Host path is set correctly on the host side. Therefore I assume that there is a specific path for transcode the ix container needs to see. I have it currently set to /transcode in plex, I tried just making it blank, but that results in the same error.

I could also just create a custom path mapping to do it, but don’t seem to be able to remove the ‘official’ transcode mapping. There must be something obvious I’m missing. Is there a specific transcode path the ix container needs to see?

Edit - Sorry reading your post again, I see it is meant to be /transcode, which it is. I have also checked permissions, the app is 568 and the dataset is apps:apps full control.

Thanks.

!Screenshot 2024-04-18 at 17.07.58.png|width=441,height=335,alt="Screenshot 2024-04-18 at 17.07.58.png"!

sync-by-unito[bot] commented 6 months ago

➤ Stavros Kois commented:

Hey, I’m not really sure what is happening here.

We have so far verified:

Plex seem to have trouble creating a directory under the config directory on transcode requests. Which shouldn’t happen assuming correct permissions.

I think it might be a good idea to hit the Plex forums, hoping that someone from Plex can give some meaning on those logs or at least provide another way to troubleshoot.

Thanks

sync-by-unito[bot] commented 6 months ago

➤ The Gipper commented:

Just wanted to throw out there that I’m also having the same issue, have been since moving from TrueNAS CORE to SCALE.

Will provide details/logs in the morning.

sync-by-unito[bot] commented 6 months ago

➤ The Gipper commented:

FYI after moving to Dragonfish and updating my plex container to the latest version, I no longer have this issue. I’ve also been able to manually select my P2000 in the Plex transcoding settings.

!image-20240503-051946.png|width=820,height=400,alt="image-20240503-051946.png"!

!image-20240503-052018.png|width=434,height=161,alt="image-20240503-052018.png"!

sync-by-unito[bot] commented 6 months ago

➤ Stavros Kois commented:

Thanks The Gipper for letting me know. Does this also work for you Marshalleq ?

sync-by-unito[bot] commented 6 months ago

➤ Marshalleq commented:

The Gipper 👍Stavros Kois Hmm will have to try it tomorrow. I did test on the beta but haven’t tested on the final release. Here’s hoping!

sync-by-unito[bot] commented 6 months ago

➤ Marshalleq commented:

Just confirming the issue still exists for me.

sync-by-unito[bot] commented 6 months ago

➤ The Gipper commented:

Marshalleq one thing I did that may be helpful for you is to completely recreate the Plex container. There are new settings available from charts (such as persistent log storage location) that I needed, so recreating may help you out there. Also in the Plex settings, are you able to select the GPU instead of picking “Auto”?

sync-by-unito[bot] commented 6 months ago

➤ Marshalleq commented:

The Gipper ok thanks I will try that. Yes I am able to select it. For a while I thought I couldn’t but it turned out it was just rendering off screen for some reason.

sync-by-unito[bot] commented 6 months ago

➤ Stavros Kois commented:

Hey Marshalleq did you have a chance to try that?

(btw you dont really have to recreate it just for the new UI options, if you upgrade you should see those) But due to how plexpass works (it downloads the application during the container startup), there might be a case where recreating might help.

sync-by-unito[bot] commented 6 months ago

➤ Marshalleq commented:

Sorry, I’m a bit slow at the moment due to ongoing concussion symptoms. Anyway, I tried this upgrade and no it didn’t help, so it would seem that my next course of action is to delete the whole thing and start again. Which on TrueNAS is a bit painful, but I will do it. I am wondering if I can pass through the GPU to jailmaker as honestly, I’ve never found any app running in kubernetes on TrueNAS to actually run that well. Since migrating to TrueNAS there seem to be issues with Live TV as well which I think is just something to do with disk performance issues, but I’m guessing. Anyway, one thing at a time. I’ll recreate the container/pod. Thanks for your help.

sync-by-unito[bot] commented 2 months ago

➤ Stavros Kois commented:

Due to changing to docker backend in the next scale version (electric eel) and the huge differences on the setup there. I’ll close this for now, and you can re-open if you encounter issues again after upgrading to electric eel when its released.

Thanks