search

elgatito / script.elementum.burst

Development of this addon has been stopped!
MIT License
119 stars 117 forks source link

YggTorrent updates #434

Open Gregsr opened 5 months ago

Gregsr commented 5 months ago

YggTorrent adress has changed: https://www.ygg.re/ and the site structure has changed too. The site is now only accessible to members, and I assume the login process is different.

Thanks

antonsoroko commented 5 months ago

@Gregsr this probably will be fixed by https://github.com/elgatito/script.elementum.burst/issues/434 although, i can't really test it because of cloudflare anti-bot protection.

if you do not get cloudflare anti-bot protection then you can make local changes from PR to see if they work.

Gregsr commented 5 months ago

@Gregsr this probably will be fixed by #434 although, i can't really test it because of cloudflare anti-bot protection.

if you do not get cloudflare anti-bot protection then you can make local changes from PR to see if they work.

I made the changes and I have this error (my credentials are ok): T:13232 critical : [script.elementum.burst] [yggtorrent] Login failed: 403

antonsoroko commented 5 months ago

@Gregsr you need to enable debug log in kodi to see real root cause, see https://elementum.surge.sh/contribute/log/ as guide (you only need kodi part for this issue).

in my case i have

critical <general>: [script.elementum.burst] [yggtorrent] Login failed: 403
debug <general>: [script.elementum.burst] [yggtorrent] Failed login content: '<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title>...more HTML page content...

which is cloud flare anti-bot page.

Gregsr commented 5 months ago

@antonsoroko here is the log file https://pastecode.io/s/5o2by9ak

I notice that the search is still being conducted before logging in. Is this logical since it is now necessary to be logged in to see the results?

antonsoroko commented 5 months ago

@Gregsr see line 3951 - you also get cloudflare page.

the solution was to use cookies from browser, see https://github.com/elgatito/script.elementum.burst/issues/414 , but in recent versions of chrome that extension to sync cookies stopped working for me (either chrome or github changed something).

another solution is to install Jacket server and use https://github.com/fugkco/script.elementum.jackett

if you really need ygg - then Jacket probably will work for you, iirc Jacket uses "browser emulation" to bypass cloudflare, but it is a separate application that you should install somewhere and keep it running.

I notice that the search is still being conducted before logging in.

where you see it? i guess in UI - this just means that search process was started. see the log starting from line 3913 - burst does login, fails, stops - search was not conducted.

but login is a part of search process and done before search, every time (if you already have cookies - burst use cookies, but burst still does login to see if you have cookies). thus in kodi UI you anyway will see that search is started - maybe that confused you. so it is logical. otherwise it would be super hard to do all checks (if you have login/pass set, if those login works, if site wotks, etc) in advance and only then show you that search was started.

Gregsr commented 5 months ago

@antonsoroko thanks for the explanations and solutions. I will try it on Windows 10 and Google TV.

antonsoroko commented 5 months ago

@Gregsr actually SyncMyCookie works with latest google chrome (i use opera and opera has some issue with this extension), so you can try it - read https://github.com/elgatito/script.elementum.burst/issues/414 note that you need to catch "cloudflare anti-bot page" first, and then you can push cookies.

then you can start elementum and check if .kodi/temp/burst/common_cookies.jar was updated with cf_clearance cookie for ygg domain.

if not then it might be a bug of that chrome extension (nothing we can do), but you can add this cf_clearance cookie manually (you can take it from dev panel in your browser).

also you need to set custom "User Agent" in burst settings. You also can see its value in dev panel in browser.

if it is too complicated (cloudflare is tricky beast) - then try Jacket.

Gregsr commented 5 months ago

@antonsoroko I tested with SyncMyCookie. It seems that login is ok (I'm not a specialist) but I receive 0 result. Here is the log file https://pastecode.io/s/j0rf2hho

antonsoroko commented 5 months ago

@Gregsr can you please enable "Log debug messages for parsing" in burst advanced settings, and create log again. Then disable that setting.

Gregsr commented 5 months ago

@antonsoroko here is https://pastecode.io/s/dt7rgt2s

antonsoroko commented 5 months ago

@Gregsr right now you have some other error

<general>: [script.elementum.burst] b'https://www.ygg.re/auth/process_login' failed with ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))):

Looks like ygg was unreachable at that moment. (Or your internet provider blocked connection to ygg.)

Need a new log file with "Log debug messages for parsing".

Gregsr commented 5 months ago

@antonsoroko the parameter "login_path": "/auth/process_login" in providers.json seems to be not the good adress (via Chrome, it is a blank page). I have changed it by "login_path": "/auth/login" (via Chrome, I get to the login page) and I have got this message in the log file but 0 result. <general>: [script.elementum.burst] [yggtorrent] Login successful

Here is the log file https://pastecode.io/s/hnm97j77

antonsoroko commented 5 months ago

@Gregsr it is not about page, but about POST request, use dev panel in browser to see how it works and read some explanation about post and get in internet, if you are unfamiliar with web requests.

Anyway, as you can see in the log, right now you get login page instead of result page, which means that Login didn't work.

You see "Login successful" because you use wrong page and that page does not give "error message", so our check thinks that everything is ok.

Please change back login_path and make log with debug parsing, like I requested above. So we can see if you get cloudflare or something else.

Unfortunately, ygg always blocks me (probably because I'm not in France), so I can't test it.

Gregsr commented 5 months ago

@antonsoroko here is https://pastecode.io/s/0sz2jkg4 Thanks for your help

antonsoroko commented 5 months ago

@Gregsr hm, I still see html content of login page instead of results page. Have you pushed cookies in chrome extension after login? Maybe they expired, please push cookies again.

Also try without cookie sync (delete cookie for ygg in chrome extension).

Gregsr commented 5 months ago

@antonsoroko I deleted cookies in chrome extension and I have got results... (see log file https://pastecode.io/s/vkdwjtmk)

At the moment, I don't have the cloudflare page to push cookies before login (I tried after having deleted all my cookies from Chrome too)

elgatito commented 5 months ago

Set useragent setting in Burst to be the same as the browser that uses cookie sync.

antonsoroko commented 5 months ago

@Gregsr if you have results then it means that changes that I made works.

I am not sure if "cookies sync" still works for ygg correctly (it used to work, and as I can see they still use ygg_ cookie name for login cookie), when you will have problem with cloudflare again - try to do cookie sync again.

Also set user agent, like me and elgatito said above.

elgatito commented 2 months ago

@Gregsr @antonsoroko is there anything to fix/change? Or we can close this issue?

antonsoroko commented 2 months ago

@elgatito it works for me with cookie sync. without - cloudflare page. and for user - in log i see yggtorrent would send 39 torrents to Elementum. so in the end - it works.

Gregsr commented 2 months ago

It depends, and I use SyncMyCookie. Indeed, if it has to go through the Cloudflare page, it doesn’t work; otherwise, it works. I have the impression that when it works, my IP was previously recognized via Chrome on Windows by going through the Cloudflare page. At that point, I can use Kodi for a few hours or days.

I noticed in the file .kodi/temp/burst/common_cookies.jar that the Cloudflare cookies were not syncing, whereas the others were.

elgatito commented 2 months ago

@Gregsr

It depends, and I use SyncMyCookie. Indeed, if it has to go through the Cloudflare page, it doesn’t work; otherwise, it works.

It is the opposite. We sync cookies from the browser to pretend that we were already passing cloudflare protection pages (your browser was doing that). But of course, there are other possible checks. Cloudflare can even detect if you were accessing pages using different OS of the device, also the simplier UserAgent check that would be different with every Chrome update.

That is annoying, but probably only the periodic cookies update can help with it.

Gregsr commented 2 months ago

It seems to me that it was the same user agent when I tried some time ago, but I will try again and keep you updated.

Gregsr commented 1 month ago

Currently, it works. However, SyncMyCookie only synchronizes 3 out of 4 cookies. The cookie related to Cloudflare is not synchronizing, as can be seen in the screenshots. Is it the issue?

common_cookies.jar file cookies jar Cookies in Chrome for ygg.re cookies SyncMyCookie syncmycookie

antonsoroko commented 1 month ago

I have the same issue for ygg and some other trackers (so I had to manually add cf_clearance cookie into file). But this is SyncMyCookie chrome extension issue. And it's repository looks abandoned. https://github.com/Andiedie/sync-my-cookie

And looks like soon chrome will disable it (and many other extensions), since it was not updated to their new standards.

I do not know what we can do here. Only if someone will fork and update this extension and also fix issue with missing cookie.