search

iFraan / rocketleague.js

It's a scrapper/wrapper of TRNetwork stats site for Rocket League.
GNU General Public License v3.0
4 stars 2 forks source link

switch child process for fetch request #5

Closed ben-agnew closed 6 months ago

ben-agnew commented 7 months ago

Description

Removed the need for a child process by using a fetch request with appropriate headers (Origin & Referer).

I have been using this method myself for a while and have had no issues

iFraan commented 7 months ago

The need for curl was born as we must imitate a browser for Cloudflare to not catch us as bots. I was getting kicked when using axios hard enough. Never tried setting the Origin header tho, so i need to test it again.

ben-agnew commented 7 months ago

I have fixed the format and added a .prettierrc file

iFraan commented 7 months ago

Just tried and it works fine in node v18+; but when using v16 or lower i get "fetch is not defined." As in this post stackoverflow/referenceerror-fetch-is-not-defined, one of the alternatives is to use node-fetch or jump to a library like axios.

Just out of curiosity, did you have any issues with curl that push you to suggest the change?

ben-agnew commented 7 months ago

Yeah I'll have a go at testing Axios. I'm not sure because it has been like 6 months since I had trouble when I was testing the library but if I remember correctly there was an issue when using serverless and it wouldn't allow a child process.

iFraan commented 6 months ago

Wep, closing this. Tried axios again and my requests were blocked even when adding Origin/Referer headers. Added the .prettierrc file tho.