excieve / dragnet

Catching the big fish
MIT License
2 stars 1 forks source link

There are no easy way to update aggregated fields of all documents after changes in views or post-processing #9

Open dchaplinsky opened 6 years ago

dchaplinsky commented 6 years ago

To do so you need to provide full list of document ids in the couchdb (and there are no easy way to generate such list).

excieve commented 6 years ago

Yes, it would be great to add a "full repump" mode to the pump.

dchaplinsky commented 6 years ago
fp = open("/tmp/full_state", "w")
with couchdb('XXXXX', 'XXXXXX', url='http://127.0.0.1:5984') as couch:
    db = couch["declarations"]
    with db.custom_result(include_docs=False, page_size=50000) as result:
        for i, r in enumerate(result):
            fp.write(r["id"] + "\n")
            if i and i % 50000 == 0:
                print(i)
fp.close()