search

website-scraper / website-scraper-puppeteer

Plugin for website-scraper which returns html for dynamic websites using puppeteer
MIT License
323 stars 79 forks source link

initial page load wait seconds #76

Closed khanzzirfan closed 1 year ago

khanzzirfan commented 1 year ago

It is good to have page to load completely by setting wait for x seconds or a selector.

Currently page starts to load and closes the quickly before waitForIdle is executed.

s0ph1e commented 1 year ago

Hi @khanzzirfan

Thank you for the suggestion, it totally makes sense. As a workaround, you can try to use scrollToBottom as described in readme. Unfortunately, I do not have time to implement new functionality in this module, but contributions are welcome

s0ph1e commented 1 year ago

Hi again 👋

New gotoOptions have been just released in version 1.1.0. I think it resolves the issue you mentioned. If it doesn't - please don't hesitate to reopen the issue