search

niklasb / dryscrape

[not actively maintained] A lightweight Python library that uses Webkit to enable easy scraping of dynamic, Javascript-heavy web pages
http://dryscrape.readthedocs.io/
MIT License
533 stars 67 forks source link

Catch "save file" dialog box #38

Closed hsharrison closed 8 years ago

hsharrison commented 8 years ago

I'm not sure if I'm asking this correctly. I'm trying to download a file off a javascript page. I've succesfully clicked the right series of nodes. The page shows a message like "Building file to download..." and then (in Chrome at least) a "Save to..." dialog box appears. I've got the first part fine... rendering shows that the message is appearing and then disappearing. However, how do I catch the save file dialog box?

I've looked through the source code and the only thing that looks promising is session.set_attribute('local_storage_enabled'). I assume that is required but I can't figure out how to actually save to the local storage.

niklasb commented 8 years ago

Local storage is a Javascript feature which has little to do with this. Downloading files is not implemented.

trendsetter37 commented 8 years ago

@hsharrison Do you have of your python script you're using to attempt automated downloads? As niklasb said this is something you will have to implement yourself. Dryscrape can get you there but you have the to do the rest. Maybe passing the cookie for your particular session from dryscrape to a requests.Session() and stream the file to local storage that way.

Or I am I in left field here?