dis-moi / extension

:speech_balloon: Never browse alone again
https://www.dismoi.io/
Other
42 stars 11 forks source link

Improve matching engine performance #277

Open bmenant opened 5 years ago

bmenant commented 5 years ago

The general idea is to add a new fast shortcut to the matching context engine, with a fallback to the current mechanism.

From the backend point of view, we will split the matching context per domain.tld and expose a bloom filter endpoint to shortcut usual matching context logic. As a result, the browser will not run an ever growing list of regexes on each URL update. If the murmur passes the bloom filter, it will go on with the classic matching context process (we can see this as an incremental matching engine).

This requires contributors to restrict their matching contexts to a list of domain.tld. Therefore, global matching contexts such as samsung.*galaxy.*s8 should not be allowed anymore, or restricted to special use (advanced or authorized only contributor).

From a privacy point of view, everything is kept on the client side, greatly minoring personal data leak and tracking issue. However, we use bloom-filtering to largely decrease the browser load.

notes

lutangar commented 5 years ago

https://hackernoon.com/probabilistic-data-structures-bloom-filter-5374112a7832

christpet commented 3 years ago

@lutangar @JalilArfaoui does this issue still apply? If yes it could be a good issue for someone like Tonio (don't see his Github handle here)