search

skrapeit / skrape.it

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
https://docs.skrape.it
MIT License
790 stars 57 forks source link

[QUESTION] Tell SkrapeIT to wait until each JavaScript loaded completely #215

Open ablx opened 1 year ago

ablx commented 1 year ago

describe what you want to archive I want to scrape a website where some content is generated after loading via Javascript (SPA). However, SkrapeIT returns to fast, so that the content is missing. Instead, there is only a span with "Loading ..." which would be replaced later.

Is there anyway to tell SkrapeIT to either wait a given time or wait until an element is present before it returns? Other than that, great lib! Thanks!

UKMIITB commented 1 year ago

@ablx Were you able to find a solution for this ? I have a problem which requires this exact solution

@skrapeit @christian-draeger Team can you please check on this question

ablx commented 1 year ago

@UKMIITB no

ferreiravinicius commented 1 year ago

@ablx @UKMIITB Not sure if it's the case, but have you tried this approach?
https://docs.skrape.it/docs/dsl/extract-client-side-rendered-data