API implementation needs an update to the new REST API

[seefood]

As mentioned at https://github.com/emericg/OpenSubtitlesDownload/issues/94#issuecomment-1486960038 - the opensubtitles site usage via the old API is ending in a few months and is already giving degraded results and subtitle files are stamped with warnings to change to the new API. VLC's VLsub and many other projects have also become useless lately.

Ergo, the new API needs to be implemented: https://opensubtitles.stoplight.io/docs/opensubtitles-api

Thanks!

[FriendlyTroll]

Hi,

I've forked this repo and starting implementing this, you can check the repo here https://github.com/FriendlyTroll/OpenSubtitlesDownload.

Note that this is rough first version, meaning there are a lot of changes necessary to use the new API, so some code is obsolete. Currently the script is kind of working, but as of now only returns all of the languages, does not seem to use hash and a lot of other bugs. I will be working on this and will post the updates here.

[emericg]

I was planning to do this... sometime... maybe... But I'm really glad you are tackling this issue @FriendlyTroll ! Keep us posted.

[FriendlyTroll]

Hi,

I've added some new changes to the script in my forked repo. This should render the script mostly working again, at least on my limited testing on Ubuntu 22.04. Gnome enviroment. Some comments:

• Special handling (forcing the UTF-8) is not needed anymore as the URL returned by the API is always UTF-8 encoded, so I removed this code
• Languages should now be specified as a single string with no whitespace in between and only 2-letter codes seem to work
• Unzipping doesn't seem to be needed as the DL link containes the plain file
• If I am not mistaken, downloading the subtitles without providing the username, password and token is not possible anymore, as the API requires these things to be sent when making requests, so make sure to create an account and create the API token in the consumer section
• On that note a python3-requests library is needed now
• Also I commented a bunch of stuff out as well (like the CLI invocation part), as I did not check this code yet what it does and is it needed anymore, I was mostly trying to make it work on my setup :-D , but I will get to those parts as well
• I have a paid opensubtitles.com/org account and I used that in the testing

Feel free to provide any feedback.

[FriendlyTroll]

Hello,

I've added back the CLI functionality. Notes:

• languages should now be passed only as 2-letter codes separated by commas and no whitespace between e.g. ./OpenSubtitlesDownload.py ~/movie.avi --cli -x -l en,ro,hr
• in the current implementation you can't download more than one language at once

[emericg]

I'm looking at your patches, seems to work fine! I'll update you when I know more.

[emericg]

OK so I've taken a deeper look at the patches, and while they do work with the new API, they also broke some of the other features of this script ^^ You also mixed the work on the feature with other things like cosmetics or removal of features, which make it harder to parse the changes related to the new API only.

But all in all I've cleaned that up, it mostly works like it used to with the old API, and it should be ready for an initial release, meant to keep the script working beyond January 1st. Further work will be done later, I'm thinking about removing wget (at least for the CLI like before, so windows and mac users have an easier time using the script) and using the script on multiple files is still broken I think. I replaced python request by urllib, because the goal of this script was always to keep it simple, no external dependencies if not absolutely necessary.

I think I'm supposed to create an API consumer on my account and set it directly in the script so the users can have an easier time.

Anyway I'm putting the current code here https://gist.github.com/emericg/d47eeed36c7923f6147e7b6c641f0cee I'd like to commit this before January 1st, and I can commit this entire thing with your name to give proper credit to your work on this @FriendlyTroll, let me know what you think.

[FriendlyTroll]

Hi,

thanks for checking it out.

Yes I did mix the work on API with some cleanup stuff because it seemed relevant, for example the RPC stuff is not needed anymore.

I see your point on using urllib instead of requests and agree with the reasoning but I don't agree with removing wget as that is included by default in most Linux distros while curl is not, at least not for now.

How about instead of using an external tool like wget or curl we just use urllibs built in functionality for downloading files? This would also resolve the problem for mac os and win users would it not?

For the API consumer key I guess its okay to use yours, I'm not sure if this will hit some limit on opensubtitles side, but I guess we'll see :smile:

[emericg]

How about instead of using an external tool like wget or curl we just use urllibs built in functionality for downloading files? This would also resolve the problem for mac os and win users would it not?

Yes that's exactly what I was talking about. The GUIs using wget aren't a big problem because they are for Linux only anyway, but for the CLI urlib will do a better job across OS. Also, we had problems with wget not working when using a VPN, while the "previous" integrated method was working. Not entirely sure why though.

For the API consumer key I guess its okay to use yours, I'm not sure if this will hit some limit on opensubtitles side, but I guess we'll see 😄

I think it's how it is supposed to be used, but the new API has weirder requirements regarding authorizations and limits. I guess we will adjust if any problems arise.

So anyway, would you prefer to submit a pull request and then I can commit my patch on top or should I just commit my patch directly with your email so your account gets credited?

[FriendlyTroll]

I just opened a pull request so just patch it on top of that as you like.

[emericg]

Thanks again for your work on this @FriendlyTroll.

I've pulled your code and updated the wiki and instructions. I'll do a release very soon, but I still have to fix the wget usage on windows and macOS.

I'll close this issue, but I'll be opening a discussion for the v6 release, and I expect that some new issues will be opened in the next few days for specific problems ^^