The "Yuga Labs x Animoca Brands" link appears twice because one of them has an expanded_url of http://somethingisbrewing.xyz/ while the other has https://somethingisbrewing.xyz/ and it seems like Twitter's shortened url counts differences in link protocol as differences in links.
While it may be wise decision to leave this as is (and simply rely on Twitter's shortened url to determine if two links are the same), it might be better for the end-user UX to normalize the links (disregarding things like protocol and domain name capitalization) before inserting them into the links PostgreSQL table and checking for uniqueness.
Imported from @nicholaschiang's original Linear issue TS-45.
On the homepage for NFTs:
The "Yuga Labs x Animoca Brands" link appears twice because one of them has an
expanded_urlofhttp://somethingisbrewing.xyz/while the other hashttps://somethingisbrewing.xyz/and it seems like Twitter's shortenedurlcounts differences in link protocol as differences in links.While it may be wise decision to leave this as is (and simply rely on Twitter's shortened
urlto determine if two links are the same), it might be better for the end-user UX to normalize the links (disregarding things like protocol and domain name capitalization) before inserting them into thelinksPostgreSQL table and checking for uniqueness.