search

ashander / ftprime

Forward-time simulation of the msprime data structure (for development)
2 stars 1 forks source link

concurrent simplify? #56

Open ashander opened 6 years ago

ashander commented 6 years ago

in principle the simplify step can be done on a separate thread

ashander commented 6 years ago

concern: seems a bit tricky to potentially avoid race conditions without expanding memory use.

ashander commented 6 years ago

@molpopgen commented

In my case, I have 1,000 generations of table stored up on the C++ side, which makes it easy for me to swap them into a new container that msprime will act on.

indicating we can probably do this easily after all code is using append columns (#41). seems like there is still a copy, but maybe nbd

molpopgen commented 6 years ago

indicating we can probably do this easily after all code is using append columns (#41). seems like there is still a copy, but maybe nbd

Yes, in the case of using MPI, there is a copy via pickling moving to/from different Python processes.