search

Cisco-Talos / cvdupdate

ClamAV Private Database Mirror Updater Tool
Apache License 2.0
93 stars 35 forks source link

Store databases in S3 and use S3 static website #40

Closed arielsalvo closed 2 years ago

arielsalvo commented 2 years ago

I'm using ClamAV on ECS Fargate containers in different AWS accounts. I think startup time would be improved and bandwidth reduced if I could have a mirror of the databases.

Are CVD files patched when downloading a CDIFF file? When are CDIFFs needed? Are they removed at some point? Are CVD files redownloaded at any time?

I would like to make use of an S3 static website (that I don't need to monitor) to share the databases among all the accounts. I was thinking of using a lambda function to run on a cron to keep the mirror up to date.

It would be great if cvdupdate could upload the files as they are downloaded but, if files don't need to be patched, I could seed the bucket with an offline run and then update with CDIFFs after a run.

micahsnyder commented 2 years ago

Hi @arielsalvo just saw your question. Sorry about the delay.

Are CVD files patched when downloading a CDIFF file?

The cvdupdate program just downloads CVD's and CDIFF's so they can be served to ClamAV freshclam clients. It does not apply the CDIFF's to the CVD's.

When are CDIFFs needed? Are they removed at some point?

The CDIFF's are needed by freshclam for incremental updates. Freshclam will download a whole CVD the first time, and then only download the CDIFF files after that point, applying them as patches to stay up-to-date. cvdupdate default configuration only keeps the last 30 CDIFF's for a given database before it starts pruning the old ones when it downloads new ones.

Are CVD files redownloaded at any time?

No. The whole point of the cvdupdate program is that it will only download a CVD if there is a new CVD.

It would be great if cvdupdate could upload the files as they are downloaded but, if files don't need to be patched, I could seed the bucket with an offline run and then update with CDIFFs after a run.

cvdupdate expects the database files to be in the ~/.cvdupdate/database directory. If it doesn't find them there, it will re-download everything, even if the JSON files in the ~/.cvdupdatedirectory claim to have them. I don't have experience using AWS lambda functions, but I'm fairly certain it won't work withcvdupdate`.