deepfence / YaraHunter

🔍🔍 Malware scanner for cloud-native, as part of CI/CD and at Runtime 🔍🔍
https://deepfence.io/
Apache License 2.0
1.23k stars 155 forks source link

Json output can no longer be stored in file #72

Closed DeLaBatth closed 9 months ago

DeLaBatth commented 9 months ago

Hi!

Since release v2.0.0 (see commit) Json output can no longer be stored in output file. However, the opposite is specified in the documentation: Readme.md and docs/docs/yarahunter/configure/output.md :

docker run -it --rm --name=deepfence-yarahunter \
     -v /var/run/docker.sock:/var/run/docker.sock \
     -v /tmp:/home/deepfence/output \
     deepfenceio/yara-hunter:latest --image-name metal3d/xmrig:latest \
     --json-filename=xmrig-scan.json

With yara-hunter v2.0.0: I have this error when I specified flag --json-filename:

docker run -it --rm --name=yara-hunter -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/home/deepfence/output deepfenceio/yara-hunter:latest --image-name docker.io/library/debian:buster-20230725-slim --json-filename=scan.json
flag provided but not defined: -json-filename
Usage of /home/deepfence/usr/YaraHunter:
-config-path string
[...]

Message flag provided but not defined: -json-filename is caused by removing of flag --json-filename in this commit: https://github.com/deepfence/YaraHunter/commit/48571c79c26c44c96ffd20e45d11bf8e739c6096.

Question is:

If "Yes", I can do MR to update the documentation. If not, I can also do MR to fix this behavior and generate a json output file thanks to flag --output=json

I will be happy to contribute, and I just want to be sure to respect the vision of previous contributors, about this topic.

The current flag --output=json can be used to generate json output file, like this:

docker run -it --rm --name=yara-hunter \
-v /var/run/docker.sock:/var/run/docker.sock \
deepfenceio/yara-hunter:latest \
--image-name docker.io/library/debian:buster-20230725-slim \
--output=json > scan.json

But the problem is that there are also logs info in this file:

>head -20 scan.json
INFO[2023-09-29T15:52:27Z] updater.go:80 rule file exists: /home/deepfence/usr/threatintel-yara-rules-2023-09-29_01-26-52.tar.gz
INFO[2023-09-29T15:52:27Z] yara.go:71 including yara rule file /home/deepfence/usr/malware.yar
WARN[2023-09-29T15:52:27Z] yara.go:91 YARA compiler warning in %s ruleset: %s:%d %sfile/home/deepfence/usr/malware.yar18526expression always false - requesting 5 of 3.
INFO[2023-09-29T15:52:27Z] runner.go:73 Scanning image %s for IOC...
docker.io/library/debian:buster-20230725-slim
INFO[2023-09-29T15:52:27Z] process_image.go:70 image scan &{imageName:docker.io/library/debian:buster-20230725-slim imageId: tempDir:/tmp/Deepfence/YaraHunter/df_dockeriolibrarydebianbuster20230725slim imageManifest:{Config: RepoTags:[] Layers:[] LayerIds:[]} numIOCs:0}
INFO[2023-09-29T15:52:27Z] autodetect.go:181 connected successfully to endpoint: unix:///var/run/docker.sock
INFO[2023-09-29T15:52:35Z] autodetect.go:218 container runtime detected: docker
INFO[2023-09-29T15:52:37Z] process_image.go:631 Image docker.io/library/debian:buster-20230725-slim saved in /tmp/Deepfence/YaraHunter/df_dockeriolibrarydebianbuster20230725slim
INFO[2023-09-29T15:52:38Z] util.go:110 Deleting temporary dir /tmp/Deepfence/YaraHunter/df_dockeriolibrarydebianbuster20230725slim
INFO[2023-09-29T15:52:39Z] runner.go:125 result severity counts: {Total:28 High:16 Medium:3 Low:9}
summary:
  total=28 high=16 medium=3 low=9

{
  "Timestamp": "2023-09-29T15:52:39.049732262Z",
  "Image Name": "docker.io/library/debian:buster-20230725-slim",
  "Image ID": "e1b92a6f8d1298f27568bccc58adf390f553d579ccd5a1a65e6754787252a3c5",
  "Container ID": "",
  "IOC": [
shyam-dev commented 9 months ago

Hello @DeLaBatth -- Yes, we should continue to provide the option to write the json to a file. We welcome contributions.

gnmahanth commented 9 months ago

In the docker run command we were using -it which was messing up stdout and stderr, using -i instead of -it fixed the issue also we were printing summary in case of json output in wrong place

both issues are fixed and docs are update in PR https://github.com/deepfence/YaraHunter/pull/73

DeLaBatth commented 9 months ago

PR works great, thanks a lot @gnmahanth, but we need to update following doc files also:

docs/docs/yarahunter/configure/cli.md
docs/docs/yarahunter/configure/output.md
docs/docs/yarahunter/index.md
docs/docs/yarahunter/quickstart.md

I have added a comment.

Thanks @shyam-dev for your answer.