No longer able to pull from javlibrary?

[damahala]

Now when try to pull data from javlibrary:

2024-04-27 20:55:05,407 (40d8) : DEBUG (runtime:717) - Handling request GET /:/plugins/com.plexapp.agents.javlibrary/messaging/function/X0FnZW50S2l0OlNlYXJjaA_/Y2VyZWFsMQoxCmxpc3QKMApyMAo/Y2VyZWFsMQoyCmRpY3QKZGljdAo2CnMyCnhuczQKbGFuZ2IwczYKbWFudWFsYjFzNwpwcmltYXJ5aTAKczcKdmVyc2lvbnIxCnM2Cmt3YXJnc3M1Ck1vdmllczEwCm1lZGlhX3R5cGU2CnMxNgowYzEyNjJjYjVlYTYxNTM5czE3Cm9wZW5TdWJ0aXRsZXNIYXNoczgKU3NpcyA5OTlzNApuYW1lczQ4ClclM0ElNUNzb3VyY2UlNUMlM0RTJTVDU1NJUy05OTklNUNTU0lTLTk5OSUyRW1wNHM4CmZpbGVuYW1lczQwCjU0NzFhZThhMjRlODYzYzNkNzc2MTBlOGRmY2EzMDFiMTA1M2Q2ZGNzOApwbGV4SGFzaHM3Cjk1NDA4OTFzOApkdXJhdGlvbnM2CjExMjg4MHMyCmlkcjAK 2024-04-27 20:55:05,407 (40d8) : DEBUG (runtime:49) - Received packed state data (80 bytes) 2024-04-27 20:55:05,407 (40d8) : DEBUG (runtime:814) - Found route matching /:/plugins/com.plexapp.agents.javlibrary/messaging/function/X0FnZW50S2l0OlNlYXJjaA_/Y2VyZWFsMQoxCmxpc3QKMApyMAo/Y2VyZWFsMQoyCmRpY3QKZGljdAo2CnMyCnhuczQKbGFuZ2IwczYKbWFudWFsYjFzNwpwcmltYXJ5aTAKczcKdmVyc2lvbnIxCnM2Cmt3YXJnc3M1Ck1vdmllczEwCm1lZGlhX3R5cGU2CnMxNgowYzEyNjJjYjVlYTYxNTM5czE3Cm9wZW5TdWJ0aXRsZXNIYXNoczgKU3NpcyA5OTlzNApuYW1lczQ4ClclM0ElNUNzb3VyY2UlNUMlM0RTJTVDU1NJUy05OTklNUNTU0lTLTk5OSUyRW1wNHM4CmZpbGVuYW1lczQwCjU0NzFhZThhMjRlODYzYzNkNzc2MTBlOGRmY2EzMDFiMTA1M2Q2ZGNzOApwbGV4SGFzaHM3Cjk1NDA4OTFzOApkdXJhdGlvbnM2CjExMjg4MHMyCmlkcjAK 2024-04-27 20:55:05,407 (40d8) : INFO (agentkit:961) - Searching for matches for {'openSubtitlesHash': '0c1262cb5ea61539', 'name': 'Xxxx 999', 'filename': 'W%3A%5Csource%5C%3DS%5CXXXX-999%5CXXXX-999%2Emp4', 'plexHash': '5471ae8a24e863c3d77610e8dfca301b1053d6dc', 'duration': '9540891', 'id': '112880'} 2024-04-27 20:55:05,407 (40d8) : DEBUG (networking:144) - Requesting 'http://127.0.0.1:32400/library/metadata/112880/tree' 2024-04-27 20:55:05,407 (40d8) : INFO (logkit:16) - Getting movie name from path: W:\source\=S\XXXX-999 2024-04-27 20:55:05,407 (40d8) : INFO (logkit:16) - Found movie name from path: XXXX-999 2024-04-27 20:55:05,516 (40d8) : INFO (logkit:16) - Searching for matches from url: http://www.javlibrary.com/ja/vl_searchbyid.php?keyword=XXXX-999 2024-04-27 20:55:08,423 (40d8) : DEBUG (runtime:88) - Sending packed state data (110 bytes) 2024-04-27 20:55:08,423 (40d8) : DEBUG (runtime:924) - Response: [200] str, 320 bytes

If copy the url to browser, it display normally, but if try to refresh metadata or find match, no result shows up.

[damahala]

It seems like the cloudscraper might not working properly. It can no longer bypass the cloudscraper's robot prevention so will always return Just a moment page. Can't simply upgrade cloudscraper as the latest 1.2.7.1 won't work on 2.7 python.

[damahala]

The easiest way to solve this issue might be just install the https://github.com/FlareSolverr/FlareSolverr docker and replace the cloudscraper's get to FlareSolverr. It works fine after replacing:

web_data = scraper.get(url).content.decode("utf-8")

with

    @staticmethod
    def get_scraper_contents(url):
        proxyUrl = "http://localhost:8191/v1"
        headers = {"Content-Type": "application/json"}
        data = {
            "cmd": "request.get",
            "url": url,
            "maxTimeout": 60000
        }
        response = requests.post(proxyUrl, headers=headers, json=data)
        jsonData = json.loads(response.text)

        return jsonData['solution']['response']

web_data = self.get_scraper_contents(url)