search

kaiiyer / webtech

Identify the technologies used on websites. (Dig-deep into web tech from your terminal)
GNU Lesser General Public License v3.0
49 stars 39 forks source link

Kindly elaborate testing on the scraping script #46

Closed aayush1205 closed 4 years ago

aayush1205 commented 4 years ago

I have written the initial logic of the scraper as well as have a brief gist as to how to integrate it as an option in the actual software. Kindly review the PR, tell me if it works fine or not, and also how can I test it myself.

utkarsh-raj commented 4 years ago

You have the same file at two different locations @aayush1205 . Can you check for this possible duplication or is it justified?

aayush1205 commented 4 years ago

Yeah... While testing, webtech_example.py wouldn't allow me to port scraper as it was inside the webtech folder. If you can tell me if the script is okay or not, and hence how can I test if it is working with the argument parser, I can work better.

utkarsh-raj commented 4 years ago

I suppose you can add to the system path at runtime. A quick internet search will give you the way (and more better ones), but if we can avoid having the same file at two places, why not?

And the work is of incremental nature, i.e., we can keep adding more features all the time. That being said, the script looks great to me. A bit of testing must be done at the local end. Clone and extend and test. Keep us updated @aayush1205 !

kaiiyer commented 4 years ago

Looks fine !

aayush1205 commented 4 years ago

Opening another PR with code update.