saturclay
commented
5 months ago I've also thought about this, mostly with the goal of reducing duplicate artist entries. My best guess for the most practical way to implement the changes below would be to run the code to standardize URLs 1) on the create artist page and 2) whenever you save a given entry. That way, a script could go through and just hit save on every artist to update their links, and then there wouldn't be a need to re-run the code on every single link in the db every time it checks for duplicates (hopefully I've phrased that in a way that makes sense).
Here is a non-exhaustive list of changes I'd like to see to fix things that don't get detected as duplicates. A lot of them are for removing query strings, but there are some other odd issues I've seen as well.
NND:
- Removing /video from the end of the URL: https://www.nicovideo.jp/user/88776253/video -> https://www.nicovideo.jp/user/88776253
- Removing ?* from the end of the URL: https://www.nicovideo.jp/user/117165247/video?ref=pc_userpage_menu -> https://www.nicovideo.jp/user/117165247
- This one can't be solved by regex alone, but wanted to mention it. http://www.nicovideo.jp/mylist/53787559 should be equivalent to https://www.nicovideo.jp/user/50263010/mylist/53787559. If you wanted to get rid of the user/50263010/ part, that would be simple with regex, but the issue is that the user/userid/mylist/listid seems to be the new one and is probably what we want to use going forward, so changing the old format ones to the new format would require some other kind of query.
Twitter:
- Removing ?* from the end of the URL
- Removing #!/ from the middle of the URL: https://twitter.com/#!/nico_agehaP -> https://twitter.com/nico_agehaP
- Removing @ from the middle of the URL: https://twitter.com/@nico_agehaP -> https://twitter.com/nico_agehaP
YouTube:
- Removing /featured* from end of the URL: https://www.youtube.com/channel/UChcViXBJOkeYOVXDxC_QFyQ/featured?app=desktop -> https://www.youtube.com/channel/UChcViXBJOkeYOVXDxC_QFyQ
- Removing /about from the end of the URL
SoundCloud:
- Removing ?* from the end of the URL
BiliBili:
- Removing #!/index from the end of the URL: https://space.bilibili.com/1289345#!/index = https://space.bilibili.com/1289345
Pixiv:
- Changing old format Pixiv URLs to the new format: https://www.pixiv.net/member.php?id=2731279 -> https://pixiv.net/users/2731279
- Removing www. from the URL: https://www.pixiv.net/users/2731279 -> https://pixiv.net/users/2731279
- Removing /en from the URL: https://pixiv.net/en/users/2731279 -> https://pixiv.net/users/2731279
- Removing /artworks from the end of the URL (basically, this is the same as when people have /video at the end of NND URLs): https://pixiv.net/users/2731279/artworks -> https://pixiv.net/users/2731279
I'll add more if I run into any.
Shiroizu
bitbybyte
Is your feature request related to a problem? Please describe. (required)
Yes, most URLs with tracking elements or other irrelevant elements do not function with artist auto-adding.
Describe the solution you'd like. (required)
Use regular expressions to remove extraneous URL elements
Example:
sp.niconico.jp (mobile NicoNico, does not function w/autoadd or link recognition)
Example URL: https://sp.nicovideo.jp/watch/sm2154380 Include Pattern: ^https://sp\.nicovideo\.jp/(.*)$ Output: https://nicovideo.jp/$1
Niconico tracking elements can probably be removed by removing any non-numeric characters after user/ or video/ but I don't remember the structure off the top of my head
Checklist (required)
Fill out the checklist.