search

phantombuster / nickjs

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)
https://nickjs.org
ISC License
500 stars 48 forks source link

Error: timeout: load event did not fire after xxx ms #66

Closed heiyeliuying closed 4 years ago

heiyeliuying commented 4 years ago

Hello, when I use nickjs and phantombuster to grab 1688.com product data (for example, https://detail.1688.com/offer/521534045266.html), I always get a timeout error. It looks like the reqeust is blocked. Do you have any suggestions for this? Here is my error log : Error: timeout: load event did not fire after xxx ms

TASnomad commented 4 years ago

Hello @heiyeliuying

This log happens when the page didn't load after a certain amount of time (default 10 seconds). To be more precise this log happens when the loading spinner didn't disappear after a certain amount of time.

You can try to extend the page loading timeout with the NickJS constructor timeout option. You can also try to speed up the page loading by adding the loadImages to false on the NickJS constructor as well.

Example:

nick = new Nick({
  timeout: 60000, // waiting 60 seconds when open method is called
  loadImages: false, // don't load images on the page
})

If those 2 parameters didn't change the situation, try to catch this exception and check (with CSS selectors) if the page is loaded.