Filenames with Umlaut can't be scanned

[tigerplush]

Describe the bug

I have a ClamD running with an open WebSocket on Port 3310. I have written a c# program that connects with the ClamD via the Websocket and sends a scan command (nSCAN file\n). Because I have to be able to handle german files the system was tested with files with Umlauts in their name ( DavSEC_Testdokument_verschlüsselt.pdf ).

Relevant code snippet:

Socket sender = new Socket(_address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
sender.Connect(_endpoint);

byte[] msg = Encoding.UTF8.GetBytes(command);
byte[] buffer = new byte[1024];
sender.Send(msg);
int receivedBytes = sender.Receive(buffer);
string returnMessage = Encoding.UTF8.GetString(buffer, 0, receivedBytes);
sender.Shutdown(SocketShutdown.Both);
sender.Close();

Encoding.Default is UTF8 with name utf-8 and code page 65001

Environment

Windows 10 Pro 21H2 Processor: Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz 2.59 GHz RAM: 32,0 GB 64bit architecture

ClamAV Version 0.104.1 daily.cld database is up-to-date (version: 26418, sigs: 1970616, f-level: 90, builder: raynman) main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr) bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)

How to reproduce the problem

Create pdf with an umlaut in it's name (e.g. C:\file_with_ümlaut.pdf) Open ClamD Connect via websocket send command nSCAN C:\file_with_ümlaut.pdf\n

returns: C:\file_with_ümlaut.pdf File path check failure: Result too large or too small. ERROR\n

ClamD console also shows: LibClamAV Warning: Can't open file C:\file_with_ümlaut.pdf: 2 Mon Jan 10 14:37:46 2022 -> ^File path check failure for: C:\file_with_ümlaut.pdf

[micahsnyder]

Using our clamdscan.exe client with an eicar file containing the umlaut, I reproduced the error message you reported. I think this is a Windows-specific error having to do with the win32 API's we're using to verify real paths.

However, despite the error, the file is still scanned and a scan result is sent back in addition.

My clamdscan.exe output:

❯ .\clamdscan.exe C:\Users\micasnyd\Downloads\file_with_ümlaut.eicar.com
LibClamAV Warning: Can't open file C:\Users\micasnyd\Downloads\file_with_├╝mlaut.eicar.com: 2
LibClamAV Warning: Can't open file C:\Users\micasnyd\Downloads\file_with_├╝mlaut.eicar.com: 2
C:\Users\micasnyd\Downloads\file_with_ümlaut.eicar.com: File path check failure: Unknown error. ERROR
C:\Users\micasnyd\Downloads\file_with_ümlaut.eicar.com: Win.Test.EICAR_HDB-1 FOUND

----------- SCAN SUMMARY -----------
Infected files: 1
Total errors: 1
Time: 0.013 sec (0 m 0 s)
Start Date: 2022:01:11 19:51:43
End Date:   2022:01:11 19:51:43

My clamd.exe output:

C:/Program Files/ClamAV
❯ .\clamd.exe
Limits: Global time limit set to 120000 milliseconds.
Limits: Global size limit set to 104857600 bytes.
Limits: File size limit set to 26214400 bytes.
Limits: Recursion level limit set to 17.
Limits: Files limit set to 10000.
Limits: MaxEmbeddedPE limit set to 10485760 bytes.
Limits: MaxHTMLNormalize limit set to 10485760 bytes.
Limits: MaxHTMLNoTags limit set to 2097152 bytes.
Limits: MaxScriptNormalize limit set to 5242880 bytes.
Limits: MaxZipTypeRcg limit set to 1048576 bytes.
Limits: MaxPartitions limit set to 50.
Limits: MaxIconsPE limit set to 100.
Limits: MaxRecHWP3 limit set to 16.
Limits: PCREMatchLimit limit set to 100000.
Limits: PCRERecMatchLimit limit set to 2000.
Limits: PCREMaxFileSize limit set to 26214400.
Archive support enabled.
AlertExceedsMax heuristic detection disabled.
Heuristic alerts enabled.
Portable Executable support enabled.
ELF support enabled.
Mail files support enabled.
OLE2 support enabled.
PDF support enabled.
SWF support enabled.
HTML support enabled.
XMLDOCS support enabled.
HWP3 support enabled.
Self checking every 600 seconds.
Listening daemon: PID: 5520
MaxQueue set to: 100
LibClamAV Warning: Can't open file C:\Users\micasnyd\Downloads\file_with_├╝mlaut.eicar.com: 2
WARNING: File path check failure for: C:\Users\micasnyd\Downloads\file_with_ümlaut.eicar.com
Quarantine of the file may fail if file path contains symlinks.
C:\Users\micasnyd\Downloads\file_with_ümlaut.eicar.com: Win.Test.EICAR_HDB-1 FOUND

Are you seeing something similar in the response received by your C# client?

[ChrisZ16]

The bug is still there with 0.105.0-rc.win.x64, using clamdscan as well as clamscan. Error messages are the same as posted above by micahsnyder.

It started with 0.104.0. With 0.103.3 and 0.103.4 files were scanned without error.

Is there a chance to fix that in the final release of 0.105.0? The frequent errors/warnings are annoying and we never know, if the file is ok or not ...

[grzech1983]

I can confirm that this issue exists also for polish national characters (ą, ć, ę, ł, ń, ó, ś, ź, ż).

EDIT: This also affects folder names. If folder name contains national characters then whole folder and of course subfolders can't be scanned.

[ChrisZ16]

Still not working with 0.105.0-rc2 ...

LibClamAV Warning: Can't open file D:\temp\z\md├╝5sums.exe: 2
WARNING: File path check failure for: D:\temp\z\mdü5sums.exe

[ChrisZ16]

Works again in 0.105.0. Thank you for fixing!