search

WorldBrain / Memex

Browser extension to curate, annotate, and discuss the most valuable content and ideas on the web. As individuals, teams and communities.
https://worldbrain.io
4.43k stars 339 forks source link

bookmark importer does not rate limit #1232

Open Davidiusdadi opened 2 years ago

Davidiusdadi commented 2 years ago

If reporting a bug:

  1. Can you describe the problem and bug in more detail? I have a lot of https://news.ycombinator.com/item links in my bookmarks and during importing get 503 errors. I bet it is because you guys are not throtteling requests by domain.

  2. How can we replicate the issue? bookmark a couple dozed HN links and try to import

  3. Expected behavior (i.e. solution) not more than X requests to any specific domains should be made.

  4. Error stack (from extension crash page) nope

  5. Other comments

You should make sure that your users can not DOS the sites that they love! Just limiting the max number of parallel requests is not enough.

blackforestboi commented 2 years ago

Thanks David for the suggestion. Will take it up.

A workaround for now is to re-do the import later and check the box "include failed urls"

Davidiusdadi commented 2 years ago

true - Unfortunately the import link disappears after the first import into a space. And when i try to generally do an import the importer does not detect that files where already imported (i did an html import )

blackforestboi commented 2 years ago

So the link is imported, but the content not?