rooteco / tweetscape

the supercharged twitter feed
https://prototype.tweetscape.co
GNU Affero General Public License v3.0
18 stars 2 forks source link

Fix missing avatars by upserting users #400

Closed nicholaschiang closed 2 years ago

nicholaschiang commented 2 years ago

We have a bunch of outdated profile_image_urls stored in Postgres that aren't being upserted every time we sync with Twitter:

image

To fix all those 404s, I should upsert the user rows that are returned (in the includes field) whenever we sync tweets (related to #397).

nicholaschiang commented 2 years ago

Another quick fix (kinda) is to restore our custom image optimization route as that will cache the optimized images locally and then serve those whenever possible (regardless of whether or not the image no longer exists on Twitter's servers).

nicholaschiang commented 2 years ago

Similarly, I should update tweets and every other object type that we receive every time we executeCreateQueue so that our tweet like and reply counts also always stay relatively up-to-date and in sync with Twitter.

nicholaschiang commented 2 years ago

I'm going to have to use pg-promise instead of Prisma for this as Prisma lacks upsertMany functionality (see https://github.com/prisma/prisma/issues/5066, https://github.com/prisma/prisma/issues/4134, https://github.com/prisma/prisma/issues/10362, etc) while pg-promise has lots of good helpers for constructing bulk queries.