search

python-bonobo / bonobo

Extract Transform Load for Python 3.5+
https://www.bonobo-project.org/
Apache License 2.0
1.59k stars 146 forks source link

Bulk loading recipe #307

Open nahuelange opened 5 years ago

nahuelange commented 5 years ago

Hello, I'm trying to have node that do bulk loading (like loading in database or ElasticSearch to SpeedUp loading. Globally this works well using the FixedWindow recipe, however, I have problem launching the process for the last buffer. Since, ContextProcessor.send method, sends it to the next node, and when I try to yield multiple times an updated buffer it raises this exception: raise RuntimeError('Context processors should not yield more than once.')

Well, what is the correct recipe to have a node buffer, and don't skip the last buffer run ?

hartym commented 5 years ago

Can you share some minimal code related to your issue please ?