search

sjdirect / abot

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.
Apache License 2.0
2.25k stars 559 forks source link

Abot is not FIPS compliance #156

Closed sjdirect closed 5 years ago

sjdirect commented 7 years ago

This file AbotCrawler\Abot\Core\CompactCrawledUrlRepository.cs uses md5 encryption which incompatible with FIPS, changing it to sha1 encryption fixes the problem.

sjdirect commented 7 years ago

The encryption used in this class is not used for security. its used to turn a string url into a number to save space and make the lookup faster.

usakgb commented 7 years ago

I fully understand what it's being used for and that it has nothing to do with security in this case, but if you try to run it on a PC, with FIPS security enabled, it'll fail. Any encryption that is not FIPS compliant will fail/throw an exception.

sjdirect commented 5 years ago

Another user suggested just using GetHashCode instead of MD5 which seems to work pretty well. May send out a build soon with this fix which should solve your issue.

sjdirect commented 5 years ago

This was fixed and is available in nuget version 1.6.0.20