search

get-set-fetch / extension

web scraping extension
MIT License
74 stars 6 forks source link

Current page scraping #65

Open ramiwi opened 3 years ago

ramiwi commented 3 years ago

Allow instant scraping of the currently open page without creating a "Start URL" project.

a1sabau commented 3 years ago

This idea has the potential to make one page scraping a LOT easier but .. how do you define the plugin options to be used ? Maybe use the configuration of an already saved project ?

ramiwi commented 3 years ago

Maybe by adding an option to choose current page or a url pattern instead of "start url" for crawling in the scraper definition step when creating a project. "Instant Data Scraper" and "Data Miner" chrome extensions opens a popup above the current page.

a1sabau commented 3 years ago

I'll add a checkbox option to have the "start url" dynamic, populated from current page url when creating a new project. All such projects will be available from a dropdown when clicking the extension icon. Scraped pages, regardless of domain, will be added to the same project.

Expect 2-3 weeks till I release a new extension version.

a1sabau commented 3 years ago

Those 2-3 weeks have turned in months... I'll definitely release a new version (this feature included) version sometime this fall.