weicheng2138 / crawler

crawl the web for remote work
https://hackmd.io/GYTmHYA4FMDYEMC0AjaBWAxogLAJmLoiMpOIpAAy6zLbDTQbYZA=?both#
GNU Affero General Public License v3.0
0 stars 0 forks source link

Project architecture and ideas #1

Open weicheng2138 opened 6 years ago

weicheng2138 commented 6 years ago

Implementation of chrome extension for this crawler or a new one?

I'm not really sure. It seems that I have to remove the server side electron and modules some new specific crawler. And implement it with chrome extension. Am I going to the right path haha? @serapath

serapath commented 6 years ago

yes - that would be sweet :-) Would be cool if the crawler browser extension has some kind of settings tab, where i can specify the scraper scripts i want to import - and maybe specify where i want to store the results of the scraper scripts.. maybe a "firebase" database or a custom one that i can host on my own server