search

MontFerret / ferret

Declarative web scraping
https://www.montferret.dev/
Apache License 2.0
5.75k stars 302 forks source link

visualize For Declarative language #589

Open yance-dev opened 3 years ago

yance-dev commented 3 years ago

FQL (Ferret Query Language) is a strong way to make process for web scraping.

For reducing learning costs, Is there any possible to provide a web UI that uses page motion capture to generate FQL？

By the way, thanks for everyone who build this great project !

ziflex commented 3 years ago

That wold be great, but unfortunately I do not have time capacity for that. If anyone wants to write one, I can create a dedicated repo under the organization.

Swathi-L commented 2 years ago

Hi @ziflex / @yance-dev, Am interested to work on this enhancement, any more details on the requirement would be helpful.

yance-dev commented 2 years ago

Hi @ziflex / @yance-dev, Am interested to work on this enhancement, any more details on the requirement would be helpful.

I actually implemented something similar with my team at another job, inspired by "webscraper".

We use webscraper to generate page rules, and then use the parsing engine to drive the browser to perform crawling work.

Hope this information can help you

Wish all the best

ziflex commented 2 years ago

@Swathi-L hey, there is a VSCode plugin that supports syntax highlighting and script execution - I think it can be a good start.