Opensubtitles.org blocked from Denmark

cphmichael commented 4 years ago

Opensubtitles has been blocked in Denmark.

In the Opensubtitles forum they write:

We received "nice" letter from Danish https://rettighedsalliancen.dk to block our site from Danish users. So on our end we detect if User IP is from DK, and if yes, then they are redirected there.

https://forum.opensubtitles.org/viewtopic.php?f=1&t=16990&p=42876

I googled it and it seems like the guys that make Infuse video player has found a workaround for it:

It seems Denmark has changed the way they are blocking OpenSubtitles. Instead of a direct block, they have added a redirect, which is confusing Infuse.
We believe this should be resolved in today's 6.2.4 update.

https://firecore.com/forum/topic/24184

Do you think it is possible that you can do something to Sub-Zero to solve that?

pannal commented 4 years ago

Well, are there any details on how they solved it? I don't think their project is open source.

If there's a redirect I can simply choose to not follow, that'd be pretty easy.

I fear that this can only be solved by using an anonymous HTTP proxy with HTTPS support, which SZ already has support for.

cphmichael commented 4 years ago

Infuse video player writes:

Insted of a direct block, they have added a redirect.

Opensubtitles writes:

On our end we detect if User IP is from DK, and if yes, then they are redirected.

So maybe 'not following the redirect' can solve the problem.

From Kitana I have "Searched for English subtitles" and attached the SZ log.

Maybe you can see something

Logs_191219_10-13-21.zip

pannal commented 4 years ago

Well, this is what I can see:

  File "/config/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/core.py", line 162, in list_subtitles_provider
    results = self[provider].list_subtitles(video, provider_languages)
  File "/config/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/core.py", line 99, in __getitem__
    provider.initialize()
  File "/config/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/providers/opensubtitles.py", line 190, in initialize
    self.log_in()
  File "/config/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/providers/opensubtitles.py", line 147, in log_in
    lambda: checked(
  File "/config/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/providers/mixins.py", line 46, in retry
    return f()
  File "/config/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/providers/opensubtitles.py", line 148, in <lambda>
    lambda: self.server.LogIn(self.username, self.password, 'eng',
  File "/config/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/providers/opensubtitles.py", line 392, in checked
    raise ServiceUnavailable("Unknown Error, empty response: %s: %r" % (status_code, response))
ServiceUnavailable: Unknown Error, empty response: None: None

For an initial assessment: Could you open the opensubtitles website in Chrome, while having the developer console (F12) open, with the checkboxes enabled for Preserve log and Disable cache, then putting the output of the website (enable the Doc tab in the developer console's Network tab) and the webserver on a pastebin? You'd need to click on the first element in the network log, copy and paste the full Headers tab and the Response tab's content to a pastebin.

That would help, afterwards I can assess whether we can continue with a more verbose test build of SZ.

cphmichael commented 4 years ago

I hope I have done it correct :-)

Is it okay that I attach the information in a .txt file:

chrome-f12.txt

ukdtom commented 4 years ago

They are detecting remote address from the header, and if danish, issue an 302 towards https://tinyurl.com/dk-court

From there, we get an 301 towards https://rettighedsalliancen.dk/wp-content/uploads/2018/03/Frederiksberg-District-Court-RA-v-TDC-5-December-2017-Popcorn-Time-EN.pdf

pannal commented 4 years ago

@ukdtom so they're issuing a 302 redirect without a proper HTML response right?

That essentially means we're doomed. This'll be only fixable by using a VPN or an HTTP Proxy.

ukdtom commented 4 years ago

Sadly so, AFAICT

cphmichael commented 4 years ago

Do you know about a HTTP Proxy that can solve the problem?

If yes, then we maybe need an option to enter the information into SZ :-)

pannal commented 4 years ago

If yes, then we maybe need an option to enter the information into SZ :-)

There already is.

cphmichael commented 4 years ago

Do you know a HTTP Proxy I can use to get access to opensubtitles?

pannal commented 4 years ago

No, but finding or renting one shouldn't be much of an issue. I'd search for an anonymous HTTP proxy that supports HTTPS as well. You can try one of those for example: https://free-proxy-list.net/anonymous-proxy.html

But this is out of scope for SZ itself, sadly.

cphmichael commented 4 years ago

I suppose that I enter the proxy here:

HTTP proxy to use for providers (supports credentials)

The list says IP: 5.9.218.105 Port: 1080

Do I enter this as "5.9.218.105:1080" ?

pannal commented 4 years ago

Yes. Make sure to use an HTTP proxy with HTTPS enabled.

Also test it in your browser before using it with SZ!

cphmichael commented 4 years ago

I have tried some of the Proxy in my browser. The homepages load VERY slow.

ukdtom commented 4 years ago

Well, does SZ works with it?

cphmichael commented 4 years ago

It did not work in SZ. I think it is because the Proxy's response VERY slow. In the browser it could take 30 sec. before opensubtitles homepage was loaded. I tried to google Paid Proxy, and can see that it cost about 5 USD pr. month. For now I have un-ticked opensubtitles and hope Addic7ed and SubScene can do it.

cphmichael commented 4 years ago

@pannal I can see there is a homepage called "opensubtitles.com" that I can access from Denmark. Do you know if that is the same as the .org service?

ukdtom commented 4 years ago

It's not the same And try: 95.217.57.55:3128

ukdtom commented 4 years ago

Ping my fellow Dane...... Any news here?

cphmichael commented 4 years ago

I have unticked Opensubtitles in SZ.

Most TV-shows come with an embedded .srt subtitles and movies with .pgs

For the few items without embedded subtitles I use the Plex subtitles download instead.

Plex can still download from Opensubtitles :-)

cphmichael commented 4 years ago

Have you tried opensubtitles.com?

Maybe that could be an replacement for the .org if @pannal want to implement it in SZ.

pannal commented 4 years ago

Plex can still download from Opensubtitles :-)

Well yes, because they're not querying using a Danish IP :)

I can see there is a homepage called "opensubtitles.com" that I can access from Denmark. Maybe that could be an replacement for the .org if @pannal want to implement it in SZ.

Not really. It's only a matter of time until that one gets blocked as well.

ghost commented 4 years ago

@cphmichael I have solved that by using a VPN provider (Torguard). Last I checked it worked from their danish IPs too, not sure what they do. Haven't checked recently, but might be worth a try.

cphmichael commented 4 years ago

@tvebax I think $ 46.95 annually is to expensive for a proxy just for downloading from Opensubtitles ;-)

ghost commented 4 years ago

Entirely up to you, free country and all that jazz. Alternative ideas would be:

Untick the SZ options to delete other subtitles and enjoy both the ones Plex metadata agent grabs from opensubtitles + what SZ grabs
Google for "cheap vpn provider" (Torguard is not the only one out there..) Or even "free vpn provider"
Search the source code of Sub-zero and replace the opensubtitles url yourself, enjoy it as long as it works.
Write the danish government..

pannal / Sub-Zero.bundle

Opensubtitles.org blocked from Denmark #707