Open hsanjuan opened 11 months ago
HTTP polling has been introduced at #22
We want to leverage this and switch ipfs.io
and dweb.link
to use RAINBOW_DENYLISTS=https://badbits.dwebops.pub/badbits.deny
.
Did some initial triage today:
Hey, nopfs watches denylists and reads any new lines appended to them. Adding updates in append-only fashion allows to do this without having to re-read the whole file.
I don't think #38 is a must. If the list upstream is append only, you:
Range
header. If no updates happened you will be reading a 0-range and it's essentially a no-op method (equivalent to checking for a If-Modified-Since tag), otherwise you obtain only the part of the content that you append to the local copy, and nopfs processes accordingly.I don't know if you saw, but the badbits list is published in append-only format here: https://denyli.st/badbits.deny.txt and that is what I used for my defunct gateway.
I have a github action that reads https://badbits.dwebops.pub/badbits.deny and finds any new lines and turns appends them to https://denyli.st/badbits.deny... so far so good, it's been going for months.
So you can use RAINBOW_DENYLISTS=https://denyli.st/badbits.deny.txt
already. In the meantime I would update badbits to be append only and not have to rely on a 3rd party.
The following are my thoughts on how to provide denylists so that they can be subscribed-to.
Server
denylist/<name>
. Must be a signed message.Client