search

starmessage / CrcCheckCopy

Command line utility to compare large folder hierarchies using CRC stamps for every file. Ideal for remote comparisons where the files are not at the same location.
Other
24 stars 1 forks source link

CRC checking speed seems to be limited to 1Gbit/sec #4

Closed starmessage closed 1 year ago

starmessage commented 1 year ago

I found your great tool but one fist use and I realized it's limitation. I have 10gbit connection to NAS and want to do checksum of folder structure of total 40TB. But the tool seems to be self limiting to 1gbit/s.

Can this limit be lifted? Total Commander checksum tool will utilize all 10gbit/s available to it.

Traffic is routed through correct LAN adapters so it looks like a limitation of your software.

starmessage commented 1 year ago

There is no limit within the application. Maybe it is CPU-limited by the CRC calculation. One pending issue is to use multiple threads for the CRC calculation.

starmessage commented 1 year ago

v2.6 released. Reorganized the code and the crc calculation algorithm to create a really fast CRC32 checksum. As a result you can see speed improvements of up to x20 on calculating the CRC of files from a local SSD drive, e.g. 2GB/sec.