search

db0 / fedi-safety

A script that goes through a lemmy images in storage and tries to prevent illegal or unethical content
GNU Affero General Public License v3.0
63 stars 6 forks source link

Redesign dockerfile #9

Closed Security-Chief-Odo closed 9 months ago

Security-Chief-Odo commented 9 months ago

Builds the python image first with requirements and initializes the CLIP model. Entrypoint then only needs to load them from the containerfile and proceed; creates a faster starting container.

poVoq commented 9 months ago

It shouldn't only load the pictrs version. The old docker-file intentionally left that open to the person running the container.

Also: is there a way to dynamically load the clip model and store it in a volume so that you don't need to download 2.3GB container-files on each update?

Security-Chief-Odo commented 9 months ago

Yeah I was trying to make that latter part happen but in the context of what I need, that download on startup takes a longer time. I wanted an image ready to go; in this case its started and listening within 5s, instead of 200s.

poVoq commented 9 months ago

Indeed that is a bit confusing on container start. But one could put it in the instructions to manually download the clip model and store it somewhere for access via the container, and the container could just check if it is up to date?

Security-Chief-Odo commented 9 months ago

Indeed that is a bit confusing on container start. But one could put it in the instructions to manually download the clip model and store it somewhere for access via the container, and the container could just check if it is up to date?

Yeah I'll work on that method instead, makes sense.

db0 commented 9 months ago

why not download the model on the first run, and store it in a permanent volume?

poVoq commented 9 months ago

The LibreTranslate container does that, but on slow connections this means the container doesn't start for 30 minutes or so, with little feedback why it isn't starting, causing many people to think it is broken.

db0 commented 9 months ago

We can easily add a log about that. It would be easier than to ask everyone to download the model manually.