Feature : add Twitter user Home Timeline crawler

[xi-jjun]

notion : https://www.notion.so/X-Timeline-Crwaling-c49f87832bbf461299b9576cba884efa

Requirements

• Fetch all users home timeline data from twitter, and save it periodically

[xi-jjun]

Twitter python sdk

• SDK : https://github.com/sns-sdks/python-twitter

[xi-jjun]

tweet table

add unique

ALTER TABLE tweet
ADD CONSTRAINT uk_tweet_id UNIQUE (tweet_id);

tweets_users table

add index

CREATE INDEX idx_tweet_id ON tweets_users (tweet_id);

add unique constraints

ALTER TABLE tweets_users
ADD CONSTRAINT uk_user_id_tweet_id UNIQUE (user_id, tweet_id);

[xi-jjun]

retrieve access token by refresh token

docs : https://developer.x.com/en/docs/authentication/oauth-2-0/user-access-token

[xi-jjun]

supabase select by joined table

docs : https://supabase.com/docs/guides/database/joins-and-nesting?queryGroups=language&language=python

[xi-jjun]

Twitter OAuth2 retrieve access token has issue

https://www.notion.so/X-access-token-retrieve-failure-9c5be256b6de49cca556f90d42ef09dc