Closed spalt08 closed 1 year ago
There’s two netflixes there that should probably be deduped.
This is interesting. How are you determining usage? Build processes likely alter the code before it lands.
@ljharb Good point. We don't have a reliable method for deduping hostnames other than checking if a favicon is the same.
As for detection, we analyze production webpack bundles by slicing it into bundled modules. Then each export inside each bundled module is compared with a pre-made signature database. After having a list of package/version candidates per export we can "guess" the original package with a certain probability, which depends on several factors, such as webpack version.
For websites that have sourcemaps the accuracy goes up to 90%, since we can use sha1(sourcemapContent)
for the comparison.
As for usage, we are going to re-scan once a week or so. So there might be a usage history graph in the future.
That's not an issue, just wanted to share an open source project I've been working on.
Source: https://gradejs.com/package/qs
So far, I've only parsed ~10K most popular websites, but the number will be increased in the nearest future. Also, the detection works only for webpack bundles with ~70-90% accuracy.