kheina-com / Blue-Blocker

Blocks all Twitter Blue verified users on twitter.com
Mozilla Public License 2.0
337 stars 28 forks source link

convert queue to using indexed db #170

Closed kheina closed 4 months ago

kheina commented 1 year ago

this would allow a practically unlimited queue size while maintaining efficiency

kheina commented 1 year ago

we would actually want to have this be implemented like a stack, so that newer entries, which the user will likely see on screen, will have priority over older entries the user may have forgotten.

one issue with this is that it seems that idb indexes give priority to entries that have a lower sort value, so a custom key value would likely need to be used and have it be decreasing in value

something like a static number with the current timestamp subtracted from it would probably work, assuming there are no conflicts, a big if considering things are added to the queue in batches

kheina commented 1 year ago

though, I will say, the current system, while slow, handles importing the 200k+ user master blue list fairly well

kheina commented 1 year ago

something like 2000000000000 - (new Date().valueOf() + Math.random()) might work

kheina commented 1 year ago

2000000000000 - (new Date().valueOf() + Math.random() * 1000) would probably work better to introduce more randomness within a single batch. looks like using this a single batch would have at least 8 digits of randomness, upwards of 10 if the timestamp changes within a single exec, but I don't think it does

kheina commented 1 year ago

also a good idea to probably have a fallback to the local storage implementation in case the idb fails to start. additionally, the extension will need to migrate any entries in local storage into the db