Open showkeyjar opened 2 years ago
What do you mean, never stuck? Basically, the map caller spawns jobs to the workers... and essentially listens/waits for a reply. Are you thinking that you would like a timed map that gives up if results don't come back in enough time, or what?
@mmckerns yes,
I think we don't care a little bad data when we process batch works for big data, so there is no need to wait for all workers done, the bad or error data could be dropped.
Hi, I use pathos to solve my data process work,
it usually stuck or blocked at file w/r, network, sql w/r ...
it's too many exceptions to cache, and those exceptions makes me crazy.
so I wonder, if your guys can design a module or function that can not be stuck anytime?
it maybe very useful if this module be published.
hope your answer, best wishes!