search

kylefarris / clamscan

A robust ClamAV virus scanning library supporting scanning files, directories, and streams with local sockets, local/remote TCP, and local clamscan/clamdscan binaries (with failover).
MIT License
230 stars 68 forks source link

Path issues - Ubuntu Server 20.04 + bad support for Windows #126

Closed sefinek24 closed 1 month ago

sefinek24 commented 1 month ago

1

The scanDir function seems to be malfunctioning. It generates an incorrect /usr/bin/clamdscan command, which looks like this (for example):

   /usr/bin/clamdscan --no-summary --fdpass --remove --multiscan
--log=/home/sefinek/logs/www/patrons.sefinek.net.log /mnt/samba/disk2/stella/uploaded_logs/8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676/Polska/XXX/1720989200642_unzipped/Logs/Prepare\ Stella 
/mnt/samba/disk2/stella/uploaded_logs/8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676/Polska/XXX/1720989200642_unzipped/Logs/Prepare\ Stella/2.0.1.0.log 
/mnt/samba/disk2/stella/uploaded_logs/8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676/Polska/XXX/1720989200642_unzipped/DeviceInfo.sfn

As you can see, where there is a space, the clamscan module unexpectedly adds \, causing the path to be incorrect. I think I fixed it with this commit in my fork: https://github.com/sefinek24/clamscam-linwin/commit/8bef262898da0febabc3cc36b761e95756663e4a, but I don't remember.

2

On my Windows workstation, I use a TCP connection with clamdscan. Unfortunately, there is another issue here. The clamscan module uses the find command to locate all files in folders. For some reason, this didn't work on Windows; the module couldn't find files in the folder. In my fork, I used this solution: https://github.com/sefinek24/clamscam-linwin/commit/2b17542c0e3201353704d26274468dccbc4b7c77

[StellaLogs]: Error during file scan: NodeClamError: There was an issue scanning the path specified!
    at E:\Projects\www\api.sefinek.net\node_modules\clamscan\index.js:2017:33
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  data: {
    path: 'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676\\JZWXCJ9XHBS8X3HSMEX9A4SG649H1D4TW5V1MGGVRATYYB1MZVWG\\1720987386402_unzipped',
    err: Error: Command failed: find E:\Projects\other\stella-data\uploaded_logs\8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676\JZWXCJ9XHBS8X3HSMEX9A4SG649H1D4TW5V1MGGVRATYYB1MZVWG\1720987386402_unzipped
    FIND: Parameter format not correct

        at genericNodeError (node:internal/errors:984:15)
        at wrappedFn (node:internal/errors:538:14)
        at ChildProcess.exithandler (node:child_process:422:12)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1105:16)
        at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
      code: 2,
      killed: false,
      signal: null,
      cmd: 'find E:\\Projects\\other\\stella-data\\uploaded_logs\\8.0.1.19+889a9d2ee7d2470a179f81211ad77688f6772676\\JZWXCJ9XHBS8X3HSMEX9A4SG649H1D4TW5V1MGGVRATYYB1MZVWG\\1720987386402_unzipped',
      stdout: '',
      stderr: 'FIND: Parameter format not correct\r\n'
    }
  },
  date: 2024-07-14T20:03:06.468Z
}

Thanks

kylefarris commented 1 month ago

I believe I have resolved your first issue in local branch. I'm working on writing the tests for the your second issue. I'm using pure JS to get the file listing now instead of using commands (file or cmd /c)--should be the better solution moving forward. That strategy was a holdover from a time when this module mostly just proxied calls to a unix shell.

Good catch and a great/thorough bug report. Thanks!

Fix should be out shortly.

sefinek24 commented 1 month ago

No problem, and thanks @kylefarris! Let me know when you manage to fix it, and I'll test it on my end too. Take your time.

kylefarris commented 1 month ago

Hey! So, I think I've created a fix for your issue. Please check out v2.3.0 and see if it works for ya!

sefinek24 commented 1 month ago

Tested 😢 Somethint went wrong

Maybe it would be worth using this https://github.com/sefinek24/clamscan-linwin/commit/2b17542c0e3201353704d26274468dccbc4b7c77 solution for Windows? Or simply use Get-ChildItem -Path from the PowerShell?

[StellaLogs]: ------------------------------- Starting the log file upload process by ::ffff:127.0.0.1 -------------------------------
=================================== 🔎 SCANNING THE ZIP FILE ===================================
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK 
node-clam: File is OK!
=================================== 🔎 SCANNING UNZIPPED FILE ===================================
[FileDir]: Extraction completed
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
[StellaLogs]: Error during file scan: NodeClamError: There was an issue scanning the path specified!
    at E:\Projects\www\api.sefinek.net\node_modules\clamscan\index.js:2010:33 {
  data: {
    path: 'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721729622471_unzipped',
    err: Error: Command failed: find E:\Projects\other\stella-data\uploaded_logs\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\XXX\1721729622471_unzipped
    FIND: Parameter format not correct

        at genericNodeError (node:internal/errors:984:15)
        at wrappedFn (node:internal/errors:538:14)
        at ChildProcess.exithandler (node:child_process:422:12)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1105:16)
        at Socket.<anonymous> (node:internal/child_process:457:11)
        at Socket.emit (node:events:519:28)
        at Pipe.<anonymous> (node:net:338:12) {
      code: 2,
      killed: false,
      signal: null,
      cmd: 'find E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721729622471_unzipped',
      stdout: '',
      stderr: 'FIND: Parameter format not correct\r\n'
    }
  },
  date: 2024-07-23T10:13:42.507Z
}
kylefarris commented 1 month ago

Oh shoot... 🫠 I'm an idiot. Hang tight.

kylefarris commented 1 month ago

Apologies for that. The previous commit had the relevant lines removed during clean up. File discovery now happens entirely using Node APIs. I would have caught this if I was testing on Windows. I might be able to add that to the GitHub Actions for CI--not sure how to do that yet but I can look into it.

Try v2.3.1 and you should be good.

sefinek24 commented 1 month ago

Thanks! You're amazing - everything is working correctly now. I'm also sending you the logs:

[StellaLogs]: ------------------------------- Starting the log file upload process by ::ffff:127.0.0.1 -------------------------------
=================================== 🔎 SCANNING THE ZIP FILE ===================================
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
=================================== 🔎 SCANNING UNZIPPED FILE ===================================
[FileDir]: Extraction completed
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Scanning a list of 12 passed files. [
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Configs\\FPS Unlocker.json',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Configs\\ReShade.ini',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\DeviceInfo.sfn',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\dxdiag.sfn',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Genshin Stella Mod\\2.1.0.1.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Info 4842\\2.0.0.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.0.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.1.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.2.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Launcher\\8.1.3.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Prepare Stella\\2.0.1.0.log',
  'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721754446801_unzipped\\Logs\\Prepare Stella\\2.0.2.0.log'
]
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 192.168.0.111:7418
node-clam: using remote server: 192.168.0.111:7418
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: The input stream has dried up.
node-clam: Received final data from stream.
node-clam: The transform stream has ended.
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Scan Complete!
node-clam: Num Bad Files:  0
node-clam: Num Good Files:  12
node-clam: Num Viruses:  0
node-clam: Socket/Host connection closed.
node-clam: ClamAV socket has been closed! false
[StellaLogs]: 📃 The log file was successfully received and saved in E:\Projects\other\stella-data\uploaded_logs\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\XXX\1721754446497.zip
[StellaLogs]: deviceId: XXX; ip: ::ffff:127.0.0.1; country: unknown; city: unknown
[200 POST 920.970 ms] /api/v7/genshin-stella-mod/launcher/logs/upload Mozilla/5.0 (compatible; StellaTelemetry/1.0.1.0; +https://stella.sefinek.net) "-" -
[FileDir]: Removed directory: E:\Projects\other\stella-data\uploaded_logs\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\XXX\1721754446801_unzipped
kylefarris commented 1 month ago

Sweet! Glad we got it working :)

sefinek24 commented 1 month ago

I have checked it more thoroughly now, and it works very well. Thanks again. Have a nice holiday. I'm not sure if the holidays are still ongoing in your country, but all the best anyway <:

node-clam: Received output from ClamAV Socket.
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: Win.Test.EICAR_HDB-1 FOUND
node-clam: Scan Response:  stream: Win.Test.EICAR_HDB-1 FOUND
node-clam: File is INFECTED!
node-clam: ClamAV is done scanning.
node-clam: Raw Response:  stream: OK
node-clam: File is OK!
node-clam: Scan Complete!
node-clam: Num Bad Files:  1
node-clam: Num Good Files:  12
node-clam: Num Viruses:  1
=================================== 🐛 THE SECOND FILE IS INFECTED! ===================================
 {
  goodFiles: [
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Configs\\FPS Unlocker.json',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Configs\\ReShade.ini',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\DeviceInfo.sfn',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\dxdiag.sfn',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Genshin Stella Mod\\2.1.0.1.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Info 4842\\2.0.0.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.0.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.1.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.2.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\8.1.3.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Prepare Stella\\2.0.1.0.log',
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Prepare Stella\\2.0.2.0.log'
  ],
  badFiles: [
    'E:\\Projects\\other\\stella-data\\uploaded_logs\\8.1.3.0+34b4e68c2048dd9c6ee841260d74e682096e1c69\\XXX\\1721755149601_unzipped\\Logs\\Launcher\\eicar.com.log'
  ],
  viruses: [ 'Win.Test.EICAR_HDB-1' ],
  errors: {}
}