search

scrapy / scrapely

A pure-python HTML screen-scraping library
1.86k stars 315 forks source link

How to scrape within Python using generated JSON from command line? #105

Open Shellcat-Zero opened 7 years ago

Shellcat-Zero commented 7 years ago

After doing:

python -m scrapely.tool myscraper.json
scrapely> ta http://pypi.python.org/pypi/w3lib/1.1
scrapely> a 0 w3lib 1.1 -n 0 -f name

How would I then use the myscraper.json from within Python for scraping?

I tried:

with open('myscraper.json') as f:
     s.fromfile(f)
     m = s.scrape('http://pypi.python.org/pypi/Django/1.3')

But it returns nothing.

Shellcat-Zero commented 7 years ago

I figured it out from #2. Documentation still needs to be updated to illustrate usage, thus leaving issue open.