The virus database is older than 7 days!

ellisium commented 1 year ago

Hello,

not sure if it's an issue or I messed the point on how to update it? refereing this issue https://github.com/awslabs/cdk-serverless-clamscan/issues/143

issue starting from 2023-01-17 Lambda DownloadDef doesn't report errors logs:

{
    "level": "INFO",
    "location": "decorate:440",
    "message": {
        "version": "0",
        "id": "b30047e4-83f1-6cd8-309d-b60db16d6b01",
        "detail-type": "Scheduled Event",
        "source": "aws.events",
        "account": "XXXXXX",
        "time": "2023-01-22T17:22:22Z",
        "region": "eu-west-1",
        "resources": [
            "XXX"
        ],
        "detail": {}
    },
    "timestamp": "2023-01-22 17:23:02,604+0000",
    "service": "freshclam-update",
    "cold_start": true,
    "function_name": "XXX",
    "function_memory_size": "1024",
    "function_arn": "arn:aws:lambda:eu-west-1:927280550839:function:seyna-platform-main-fileM-rClamScanDownloadDefs097-cXi4giQVVTDY",
    "function_request_id": "",
    "xray_trace_id": ""
}

I saw "Update the docker images of the Lambda functions with the latest version of ClamAV by re-running cdk deploy." We deploy everyday but maybe I miss a way to force this update?

dontirun commented 1 year ago

I'm not sure what the cause is here. My theory would be that there might be some stale definition files on EFS which are not getting updated for some reason

emmapatterson commented 1 year ago

Hi there, I am seeing this same warning even after a fresh deploy as the documentation suggests should update the database.

Thanks

dontirun commented 1 year ago

The fresh deploy won't solve this problem, that's when ClamAV is out of date (not the database).

I don't know why this is happening as I haven't been able to reproduce it in my own testing 😕.

@emmapatterson how frequently are files getting scanned? Is there any downtime? I'm curious if the freshclam process is getting in a locked up state and isn't getting a chance to reset if there are executions constantly running

emmapatterson commented 1 year ago

The fresh deploy won't solve this problem, that's when ClamAV is out of date (not the database).

I don't know why this is happening as I haven't been able to reproduce it in my own testing 😕.

@emmapatterson how frequently are files getting scanned? Is there any downtime? I'm curious if the freshclam process is getting in a locked up state and isn't getting a chance to reset if there are executions constantly running

Thanks for the reply, there is definitely downtime, the scans are irregularly throughout the day when our users choose to upload.

We saw this same issue when we had built our own lambda from a similar docker image as yourself. This issue was part of the reason we moved over to using this construct. 🤞🏼 we can find a solution

alanfreer commented 12 months ago

Hi. This problem is happening in our deployment as well. There are no errors in the logs for the lambda which should be updating the virus definitions. All the files in the virus definitions S3 bucket have been updated today apart from daily.cld which is a week old.

Are there any diagnostic techniques you could suggest please?

I tried adding --verbose to the freshclam command in assets/lambda/code/scan/lambda.py but I couldn't see the output in the logs.

Thanks

nkhine commented 4 months ago

Hello @dontirun,

I am also getting the same messages coming through, the update lambda shows no errors

{
    "level": "INFO",
    "location": "decorate:448",
    "message": {
        "version": "0",
        "id": "68272140-ba2d-390e-b7cd-f2017b22be68",
        "detail-type": "Scheduled Event",
        "source": "aws.events",
        "account": "***",
        "time": "2024-06-20T15:45:28Z",
        "region": "eu-west-1",
        "resources": [
            "arn:aws:events:eu-west-1:***:rule/BaseStage-TransferStack-ServerlessClamscanVirusDefs-pBjhZWASExuy"
        ],
        "detail": {}
    },
    "timestamp": "2024-06-20 15:45:36,502+0000",
    "service": "freshclam-update",
    "cold_start": true,
    "function_name": "BaseStage-TransferStack-ServerlessClamscanDownload-CbhdAfqY7Hvt",
    "function_memory_size": "1024",
    "function_arn": "arn:aws:lambda:eu-west-1:***:function:BaseStage-TransferStack-ServerlessClamscanDownload-CbhdAfqY7Hvt",
    "function_request_id": "3feb9c91-a674-431e-ae69-6f0bf167ce18",
    "xray_trace_id": "1-66744e9f-4a9aa75a082f695a3cbc857b"
}

This is the Event I get from the clamscan.cleanRule:

{
  "input_bucket": "***",
  "input_key": "home/nkhine/test.csv",
  "message": "LibClamAV Warning: **************************************************\nLibClamAV Warning: ***  The virus database is older than 7 days!  ***\nLibClamAV Warning: ***   Please update it as soon as possible.    ***\nLibClamAV Warning: **************************************************\nScanning /mnt/lambda/6484ae0e-1496-419a-9bba-dd6446aad266/home/nkhine/test.csv\n/mnt/lambda/6484ae0e-1496-419a-9bba-dd6446aad266/home/nkhine/test.csv: OK\n\n----------- SCAN SUMMARY -----------\nKnown viruses: 8692590\nEngine version: 0.103.11\nScanned directories: 3\nScanned files: 1\nInfected files: 0\nData scanned: 0.00 MB\nData read: 0.00 MB (ratio 0.00:1)\nTime: 44.063 sec (0 m 44 s)\nStart Date: 2024:06:20 21:47:04\nEnd Date:   2024:06:20 21:47:48\n",
  "source": "serverless-clamscan",
  "status": "CLEAN",
  "RowId": 0,
  "UpdatedRecords": false
}

Any advice is much appreciated

eXistenZNL commented 3 months ago

Hi,

we are using this module for a client of ours and they are also running into this notification. We have added some debugging and have noticed that whenever ClamAV says that the definitions are too old, the files that are available to ClamAV are indeed very old (more than one year old).

Somehow it seems that older files are sometimes mounted in the container, however it is very very hard for us to debug this issue since the layers of wrapping around the codebase (container, serverless function, trigger, etc.)

Please help us figure out what is going on here / fix the issue, now the solution cannot be trusted in a production environment!