1thing-org / hatecrimetracker

Hackathon project to create anti-Asian hate crime tracker
10 stars 18 forks source link

Notif server #4

Closed ethannchen closed 1 year ago

ethannchen commented 1 year ago

There's a lot of changes, so I closed the previous PR.

Changes:

  1. Split the PR into two: (1) Format the code (2) Add the push notification support.
  2. Use rate limiting (flask_limiter) on "/token" route to prevent spamming or DDOS attack
  3. Add expiration_time field to Token class to help delete outdated tokens from database
  4. Load tokens in batches (fireo: fetch and next_fetch);
    • current batch_size = 10 for testing, target 1000?
    • the Token.collection.fetch() used in previous code is an generator, will it cause memory issue?
  5. For each batch of tokens, group them by first two characters, and process these groups in parallel (concurrent.futures)
    • regarding the previous comment "The other way is to attach a incident Id or timestamp to each token. So for each token/user, we know which notification has been sent, so we just need to send any incidents newer than that timestamp next time." The incident_publisher publish incidents based on its publisher_status field (Twitter, LinkedIn, push notification), so it looks like there's no need to tag each token for the most recent time the push notification was sent?