Cisco-Talos / cvdupdate

ClamAV Private Database Mirror Updater Tool
Apache License 2.0
93 stars 35 forks source link

Add docker container build instructions #47

Closed ismvru closed 1 year ago

ismvru commented 2 years ago

Added Dockerfile, instructions how to use docker container in README.md.

Docker container is based on python:3-slim docker image with installed cron, running in foreground.

micahsnyder commented 1 year ago

@ismvru I'm so sorry for neglecting this PR for nearly a year. I feel terrible about it. I will re-review and try to help get it merged asap.

ismvru commented 1 year ago

Hello! About a year has passed, I'm not sure that now I can remember whether there were such problems initially. Let me test again, and I can tell you what the problem is and how to solve it.

micahsnyder commented 1 year ago

@ismvru thank you. So sorry about that delay. Are you ready for me to try again?

ismvru commented 1 year ago

@micahsnyder Hello! It's okay, I myself have been terribly busy in the last few months.

You can try again, I have already made the necessary changes (and I am ready to make more if necessary, though tomorrow).

micahsnyder commented 1 year ago

Yup I saw the commits and thought I should ask. Thanks for understanding. I'll give it another try.

micahsnyder commented 1 year ago

It's looking pretty good now. I ran this:

cvdupdate on  PR-47 [?] via 🐍 v3.8.10
❯ docker run -it -v (pwd)/db:/cvdupdate/database -v (pwd)/logs:/cvdupdate/logs -e USER_ID=(id -u) cvdupdate-pr-47
ClamAV Private Database Mirror Updater Cron /dist/scripts/docker-entrypoint.sh
Creating user with ID 1001
useradd: warning: the home directory /cvdupdate already exists.
useradd: Not copying any file from skel directory into it.
Adding crontab entry
no crontab for root
2023-05-25 20:08:16 cvdupdate-1.1.1 INFO Using system configured nameservers
2023-05-25 20:08:17 cvdupdate-1.1.1 INFO Downloaded main-62.cdiff
2023-05-25 20:08:32 cvdupdate-1.1.1 INFO Downloaded main.cvd. Version: 62
2023-05-25 20:08:33 cvdupdate-1.1.1 INFO Downloaded daily-26918.cdiff
2023-05-25 20:08:39 cvdupdate-1.1.1 INFO Downloaded daily.cvd. Version: 26918
2023-05-25 20:08:39 cvdupdate-1.1.1 INFO Downloaded bytecode-334.cdiff
2023-05-25 20:08:40 cvdupdate-1.1.1 INFO Downloaded bytecode.cvd. Version: 334

And then checked the directory and it seems okay. I wonder if group ownership should be the same # as the UID. But having it be owned by "docker" group doesn't seem bad:

❯ ls -la db
total 228972
drwxr-xr-x  2 micah micah       4096 May 25 13:08 ./
drwxr-xr-x 10 micah micah       4096 May 24 14:32 ../
-rw-r--r--  1 micah docker         0 May 25 13:08 bytecode-334.cdiff
-rw-r--r--  1 micah docker    291965 May 25 13:08 bytecode.cvd
-rw-r--r--  1 micah docker      3089 May 25 13:08 daily-26918.cdiff
-rw-r--r--  1 micah docker  61379463 May 25 13:08 daily.cvd
-rw-r--r--  1 micah docker        42 May 25 13:08 dns.txt
-rw-r--r--  1 micah docker   2282177 May 25 13:08 main-62.cdiff
-rw-r--r--  1 micah docker 170479789 May 25 13:08 main.cvd

I also tried with a root owned directory (and omitting the USER_ID), and that worked well:

❯ docker run -it -v /srv/cvdupdate:/cvdupdate/database -v /var/log/cvdupdate:/cvdupdate/logs cvdupdate-pr-47
ClamAV Private Database Mirror Updater Cron /dist/scripts/docker-entrypoint.sh
Adding crontab entry
no crontab for root
2023-05-25 20:22:45 cvdupdate-1.1.1 INFO Using system configured nameservers
2023-05-25 20:22:45 cvdupdate-1.1.1 INFO Downloaded main-62.cdiff
2023-05-25 20:23:00 cvdupdate-1.1.1 INFO Downloaded main.cvd. Version: 62
2023-05-25 20:23:00 cvdupdate-1.1.1 INFO Downloaded daily-26918.cdiff
2023-05-25 20:23:06 cvdupdate-1.1.1 INFO Downloaded daily.cvd. Version: 26918
2023-05-25 20:23:06 cvdupdate-1.1.1 INFO Downloaded bytecode-334.cdiff
2023-05-25 20:23:07 cvdupdate-1.1.1 INFO Downloaded bytecode.cvd. Version: 334

Ownership is as you would expect:

❯ ls -la /srv/cvdupdate/
total 228972
drwxr-xr-x 2 root root      4096 May 25 13:23 ./
drwxr-xr-x 3 root root      4096 May 24 14:33 ../
-rw-r--r-- 1 root root         0 May 25 13:23 bytecode-334.cdiff
-rw-r--r-- 1 root root    291965 May 25 13:23 bytecode.cvd
-rw-r--r-- 1 root root      3089 May 25 13:23 daily-26918.cdiff
-rw-r--r-- 1 root root  61379463 May 25 13:23 daily.cvd
-rw-r--r-- 1 root root        42 May 25 13:23 dns.txt
-rw-r--r-- 1 root root   2282177 May 25 13:22 main-62.cdiff
-rw-r--r-- 1 root root 170479789 May 25 13:23 main.cvd
ismvru commented 1 year ago

@micahsnyder Hello. I can add group id processing, it's not a problem.

There is also an idea to add several variables, for example, specifying a mirror from where to download updates.

Maybe it's worth redoing the part with cron so that it would accept expressions like "every 2 days" (although I'm very lazy to do this, because I did not find ready-made libraries for translating such expressions into the cron syntax.)...

micahsnyder commented 1 year ago

specifying a mirror from where to download updates.

This one seems less likely since cvdupdate exists primarily to help host mirrors.

Maybe additional features can be added after this PR.

Is there anything else you want to do here? I think I'm happy with this as-is for a start. But I also don't have strong opinions about having a Docker image for cvdupdate in the first place 😅.

ismvru commented 1 year ago

@micahsnyder, Yes, a mirror for a mirror is some kind of recursion :)

Now I don’t want to do anything, maybe some other ideas will appear later, then I’ll do another PR.