search

S1M0N38 / soccerapi

soccerapi, an unambitious soccer odds scraper ⚽️
MIT License
161 stars 36 forks source link

Bet365 is blocking Puppeteer access #29

Closed helderppb closed 3 years ago

helderppb commented 3 years ago

I'm facing a problem with the Puppeteer library to access the Bet365 website. It just started today. Does anyone has a workaround to solve this issue?

athuscavalini commented 3 years ago

Hi, @helderppb. I have also been experiencing problems with requests for about an hour. What response are you getting?

helderppb commented 3 years ago

Hi @athuscavalini, it looks like a CORS error. I'm doing a few tests using Puppeteer and also Playwright, but both are showing the same issue. image

artesea commented 3 years ago

@helderppb that isn't a CORS issue. Your "API" requests are being redirected to localhost, which usually means your IP address has been blacklisted. If you try to open Bet365 in your browser (on the same IP) you should only get the frame of the site and no matches or odds.

helderppb commented 3 years ago

@artesea No! I’m able to access the website normally using the browser. This issue is happening with other persons as well. Bet365 has made changes on their system to identify headless applications. I use to do the same operations a long time, but now I’m facing this problem. Try to do the process using Puppeteer. Thank you for your attention and response.

cardchase commented 3 years ago

@artesea No! I’m able to access the website normally using the browser. This issue is happening with other persons as well. Bet365 has made changes on their system to identify headless applications. I use to do the same operations a long time, but now I’m facing this problem. Try to do the process using Puppeteer. Thank you for your attention and response.

Yep, me too. The site opens in the browser but unable to scrape odds. @helderppb how could you use puppeteer with soccerapi? Considering I am assuming you could make it work. Thanks

S1M0N38 commented 3 years ago

from README

In order to obtain data from Bet365 you need to run a docker which posts on a local server a needed header to make requests to the Bet365 api. The docker run separeately from the api since it is written in JavaScript and it runs a chromedirver to run JavaScript and acquire the header. Checkout soccerapi-server to install the docker and run it.