lupomontero / psl

JavaScript domain name parser based on the Public Suffix List
https://www.npmjs.org/package/psl
MIT License
388 stars 80 forks source link

Performance issues - iterating over 9k rules for every domain #301

Open gugu opened 2 years ago

gugu commented 2 years ago

I've made some benchmarks and performance of psl is around 2k ops/sec:

psl#isValid x 2,310 ops/sec ±2.00% (92 runs sampled)
psl#parse x 2,428 ops/sec ±0.24% (97 runs sampled)
psl#parse invalid domain x 2,415 ops/sec ±1.56% (94 runs sampled)

by using maps it is possible to increase speed to 300k ops/sec:

psl#isValid x 349,961 ops/sec ±0.48% (95 runs sampled)
psl#parse x 346,816 ops/sec ±0.18% (99 runs sampled)
psl#parse invalid domain x 375,047 ops/sec ±0.16% (96 runs sampled)
gugu commented 2 years ago

@lupomontero sorry for mentioning you. Can you please review this PR? I think 300x speed boost can helpful change to this module

billneff79 commented 1 year ago

We frequently are parsing upwards of 100k domains. We love the accuracy of psl, but the performance of psl is a real hindrance. It would be great if the PR that addresses this could get reviewed/merged

Stanback commented 1 year ago

Would love to see this get merged, it's insanely more performant

lupomontero commented 4 days ago

Hi there, many thanks for the proposal and apologies for the delay in dealing with this. I'm finally revamping this module and this is a very much needed improvement. I'll be reviewing this carefully tomorrow.