Closed hanoj-budime closed 1 year ago
Hi @kylefarris,
Please can you help me
Hi @SaltwaterC, @genio, @ngraef
Please can you help me it's urgent to me 😊
I'm observing the same situation (clamd v0.104 and clamd v0.105). What I can say is that sometimes it is rejected
on socket.error
and sometimes it is resolved
with NodeClamError
:
Reject example:
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 127.0.0.1:3310
node-clam: Received final data from stream.
node-clam: The input stream has dried up.
node-clam: _initSocket Socket/Host connection failed: Error: write EPIPE
at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
errno: 'EPIPE',
code: 'EPIPE',
syscall: 'write'
}
node-clam: scanStream Error emitted from ClamAV socket: Error: write EPIPE
at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
errno: 'EPIPE',
code: 'EPIPE',
syscall: 'write'
}
node-clam: _initSocket Socket/Host connection closed.
node-clam: scanStream ClamAV socket has been closed! true
and here is Resolve with NodeClamError example:
node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 127.0.0.1:3310
node-clam: Received final data from stream.
node-clam: The input stream has dried up.
node-clam: scanStream Received output from ClamAV Socket.
node-clam: scanStream ClamAV is done scanning.
node-clam: scanStream Raw Response: INSTREAM size limit exceeded. ERROR
node-clam: Error Response: INSTREAM size limit exceeded.
node-clam: File may be INFECTED!
CUSTOM DEBUG scanStream before resolve: NodeClamError: An error occurred while scanning the piped-through stream: INSTREAM size limit exceeded.
at NodeClam._processResult (/home/benzino/Dokumenty/clamav-rest-api/node_modules/clamscan/index.js:781:20)
at Socket.<anonymous> (/home/benzino/Dokumenty/clamav-rest-api/node_modules/clamscan/index.js:2283:45)
at Socket.emit (events.js:326:22)
at endReadableNT (_stream_readable.js:1241:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21) {
data: { error: 'INSTREAM size limit exceeded. \u0000' },
date: 2022-07-27T10:07:14.898Z
}
node-clam: _initSocket Socket/Host connection failed: Error: write EPIPE
at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
errno: 'EPIPE',
code: 'EPIPE',
syscall: 'write'
}
node-clam: scanStream Error emitted from ClamAV socket: Error: write EPIPE
at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
errno: 'EPIPE',
code: 'EPIPE',
syscall: 'write'
}
node-clam: _initSocket Socket/Host connection closed.
node-clam: scanStream ClamAV socket has been closed! true
I have investigated it a little bit more, and it looks like it can be a problem on clamscan
side (don't know where exactly).
Calmav is calling function handle_stream
to process the network payload (link). If the payload exceeds StreamMaxLength
the handle_stream
function:
conn_reply_error
which sends INSTREAM size limit exceeded.
to the socket (link)error
to 1
-1
After that the socket is closed (link)
Here is the debug information from calmd side when StreamMaxLength
is exceeded:
WARNING: Thu Jul 28 12:32:11 2022 -> INSTREAM: Size limit reached, (requested: 104857600, max: 26214400)
Thu Jul 28 12:32:11 2022 -> Moved partial command: 4
Thu Jul 28 12:32:11 2022 -> mode == MODE_STREAM
Thu Jul 28 12:32:11 2022 -> Got chunksize: 104857600
Thu Jul 28 12:32:11 2022 -> Shutting down socket after error (FD 10)
Thu Jul 28 12:32:11 2022 -> Number of file descriptors polled: 1 fds
It looks like clamscan
does not every time receive/process the information that the payload is too big and even if this information is properly received/processed clamscan
does not properly handle such situation (it does not reject but resolve with NodeClamError
)
@benzino77 If you have any spare time to look into how to resolve the issue of clamscan
not receiving/processing the info consistently and submit a PR, I'd be very grateful. At the moment, I'm just a bit overloaded with a million other projects. Also, thank you so much for the investigative work you've done so far--it's been very helpful and informative.
No problem. I will make PR with proposition.
I have investigated it a little bit more, and it looks like it can be a problem on
clamscan
side (don't know where exactly). Calmav is calling functionhandle_stream
to process the network payload (link). If the payload exceedsStreamMaxLength
thehandle_stream
function:
- calls
conn_reply_error
which sendsINSTREAM size limit exceeded.
to the socket (link)- set
error
to1
- return
-1
After that the socket is closed (link)
Here is the debug information from calmd side when
StreamMaxLength
is exceeded:WARNING: Thu Jul 28 12:32:11 2022 -> INSTREAM: Size limit reached, (requested: 104857600, max: 26214400) Thu Jul 28 12:32:11 2022 -> Moved partial command: 4 Thu Jul 28 12:32:11 2022 -> mode == MODE_STREAM Thu Jul 28 12:32:11 2022 -> Got chunksize: 104857600 Thu Jul 28 12:32:11 2022 -> Shutting down socket after error (FD 10) Thu Jul 28 12:32:11 2022 -> Number of file descriptors polled: 1 fds
It looks like
clamscan
does not every time receive/process the information that the payload is too big and even if this information is properly received/processedclamscan
does not properly handle such situation (it does not reject but resolve withNodeClamError
)
@benzino77 - Thanks for spending some time here - to help us! 😊
No problem. I will make PR with proposition.
Definitely, this approach going to help us. therefore we can notify such error in our services & then we'll throw based that to my HTTP client request
@benzino77 @kylefarris Once Again - Thank you guys ( For your Support ).
Looking forward to this PR 🏳
Now I start thinking that it can be clamd
problem. I've performed some tests on v0.103.6 clamd
and it looks like clamscan
is detecting socket error (write EPIPE), before any information is send back by clamd
(that stream size limit is exceeded) almost every time (much, much more often than for v0.104.x
or v0.105.x
).
I'm starting my holidays so I will get back to this when I'll be back.
Thanks for the update @benzino77. Enjoy your holidays!
For such errors - Clamav file size limit
Are you throwing with an exact error message?
Expecting
Somehow it should return the same error message to us![image](https://user-images.githubusercontent.com/53425981/181046230-640a1d08-643d-459a-b599-9760f7cd2965.png)
This WARNING message should return to ClamScan level so that we can understand those errors - we can throw custom based on your error message info.