search

dcts / opensea-scraper

Scrapes nft floor prices and additional information from opensea. Used for https://nftfloorprice.info
MIT License
184 stars 73 forks source link

Can't obtain any information from opensea due to cloudflare #68

Closed 0xMarsRover closed 1 year ago

0xMarsRover commented 1 year ago

I've added addtionalWait (up to 1 minute) to get html.content(), but it seems that opensea blocks me from it.

Note, it works when getting nft information from opensea testnet. Any tips?

image

dcts commented 1 year ago

this is a cloudflare issue, unfortunately I don't know how to solve, other than changing your environment. Launching the scraper from a cloud server like gcloud or firebase functions or heroku usually doesn't work. Using a private machine or using proxy services (expensive) should work though.

Good luck, please share if you are able to fix it.

0xMarsRover commented 1 year ago

Hi @dcts , Thanks for your reply. I just tried that AWS works for this part