pulsejet / memories

Fast, modern and advanced photo management suite. Runs as a Nextcloud app.
https://memories.gallery
GNU Affero General Public License v3.0
3.18k stars 86 forks source link

Several install issues with hardware transcode, go-vod and docker #873

Open WGandy opened 1 year ago

WGandy commented 1 year ago

I'm trying to follow the instructions and I found these issues:

1) I needed to make the start.sh file executable or I got a permission denied error. So I added this middle line to the Dockerfile.

COPY start.sh /
RUN chmod +x /start.sh
CMD /start.sh

2) php-fpm doesn't seem to exist in the new container or is not in path? I get this error when taking docker compose up. | /start.sh: line 7: php-fpm: command not found

3) The group render2 appears to already exist in the container??
| groupadd: group 'render2' already exists

4) And lastly, since I want to use go-vod for transcoding, isn't that the container that needs the va drivers anyway? Or do I need to install ffmpeg and video drivers in the nextcloud container in order for go-vod to function?

I tried to get this working several months ago and eventually gave up. I figured I'd give it another try now but I'm having a hard time understanding the current instructions. (I have both an intel cpu with graphics and an nvidia card in the server and want to compare to see which one gives me better results.)

pulsejet commented 1 year ago

I needed to make the start.sh file executable or I got a permission denied error.

Just make the script executable on your host.

php-fpm doesn't seem to exist in the new container or is not in path? I get this error when taking docker compose up.

Use the nextcloud:stable-fpm image. These instructions are old though.

The group render2 appears to already exist in the container??

That's okay. Add a || true to suppress the error

And lastly, since I want to use go-vod for transcoding, isn't that the container that needs the va drivers anyway? Or do I need to install ffmpeg and video drivers in the nextcloud container in order for go-vod to function?

You're mixing up two sets of instructions. You install the VA drivers in the external go-vod container OR the nextcloud container (NOT both). Read the docs carefully, the VA-API section mentions at the top that the instructions aren't applicable to external transcoders.

I strongly recommend going the external transcoder route, it's much less error prone. All you need to do is have the exact same mounts on your Nextcloud docker container and the go-vod container; then just build the Dockerfile from this repo and put them in the same network.

WGandy commented 1 year ago

I needed to make the start.sh file executable or I got a permission denied error.

Just make the script executable on your host.

Interesting.

php-fpm doesn't seem to exist in the new container or is not in path? I get this error when taking docker compose up.

Use the nextcloud:stable-fpm image. These instructions are old though.

Which instructions are old? (The ones you just gave me or the ones that are on the memories hardware transcoding page?)

And lastly, since I want to use go-vod for transcoding, isn't that the container that needs the va drivers anyway? Or do I need to install ffmpeg and video drivers in the nextcloud container in order for go-vod to function?

You're mixing up two sets of instructions..... the VA-API section....

Exactly, I'm mixing up the instructions. One of my reasons for posting this issue was to suggest that perhaps there should be a more explicit separation between "sections" in the instructions. VA-API could exist in both containers depending on what you are trying to setup, no?

All you need to do is have the exact same mounts on your Nextcloud docker container and the go-vod container; then just build the Dockerfile from this repo and put them in the same network.

Can you clarify something about this... Is go-vod transcoding compatible with S3 object storage as primary storage? (I saw a note somewhere suggesting to not use external storage with memories but S3 as primary is a different animal) What about when server side encryption is enabled? It would appear to me that if nextcloud and go-vod are both accessing the same mount and the nextcloud container is encrypting, then encryption would be out of the question for go-vod. ??

Thanks for the help (and the package in general). I'm excited to get it working.

pulsejet commented 1 year ago

Which instructions are old?

The ones you were following on the page. Check these docs for the latest instructions (they were updated yesterday)

VA-API could exist in both containers depending on what you are trying to setup, no?

Sure, but it's pointless. You only need the VA-API drivers in the container that runs ffmpeg. If you're using an external transcoder, then that means your Nextcloud container doesn't need drivers.

Is go-vod transcoding compatible with S3 object storage as primary storage?

No. Currently trancsoding needs the whole file to be available before it can start, which is terribly slow for external storage. That's why transcoding is explicitly disabled for external storage (this is mentioned in the docs here)

I saw a note somewhere suggesting to not use external storage with memories

This is the only limitation really. If you don't use transcoding then external storage works fine.

What about when server side encryption is enabled?

This one is supposed to work, but I've zero experience with it. I believe there are people using it out there, but not sure if that's with transcoding or not.

WGandy commented 1 year ago

Is go-vod transcoding compatible with S3 object storage as primary storage?

No. Currently trancsoding needs the whole file to be available before it can start, which is terribly slow for external storage. That's why transcoding is explicitly disabled for external storage (this is mentioned in the docs here)

I saw a note somewhere suggesting to not use external storage with memories

This is the only limitation really. If you don't use transcoding then external storage works fine.

I think we may have a terminology issue. As I understand it, there are two ways to use object storage with Nextcloud. One method is through the "external storage" framework that has existed in Nextcloud for quite a while. The other is more recent and it allows for object storage to be used AS THE PRIMARY storage. This is how I have one of my Nextcloud instances configured. My object storage has a node that is running on the same server so reads are relatively fast. (writes require a quorum with another node so can be delayed a little bit). Are you saying that transcoding is disabled for both methods or just for the Nextcloud "external storage" system?

pulsejet commented 1 year ago

The other is more recent and it allows for object storage to be used AS THE PRIMARY storage.

No idea, I'm not familiar with this. The simplest way to find out would be to try transcoding without hw acceleration; it's basically the same except it uses the CPU.