ScriptSmith / instamancer

Scrape Instagram's API with Puppeteer
http://adamsm.com/instamancer
MIT License
398 stars 61 forks source link

One file for all scraps #28

Closed arnaudjnn closed 5 years ago

arnaudjnn commented 5 years ago

I'm using batchfile to scrap multi accounts, is it possible to download one json file at the end of all scaps instead a file by username ?

ScriptSmith commented 5 years ago

This would be challenging to add because instamancer doesn't currently use line-delimited JSON.

It's simple to combine json files with something like jq:

jq -s '[.[][]]' *.json > output
ScriptSmith commented 5 years ago

So the entire pattern could be something like:

Create storage dir

mkdir users

Write batch.txt

user -c10 therock -o "users/[id].json"
user -c10 arianagrande -o "users/[id].json"

Retrieve and combine output:

instamancer batch batch.txt && jq -s '[.[][]]' users/*.json > users.json