search

djhocking / Trout_GRF

Application of Gaussian Random Fields in a Dendritic Network
3 stars 0 forks source link

Write Lock in Parallel #10

Closed djhocking closed 8 years ago

djhocking commented 8 years ago

Problem: no file locking when writing in parallel so things seem to get messed up and written in the wrong places when creating the spatiotemporal simulation summary output table. Potential solutions:

  1. Write each line to separate files then recombine later (SLOW with loop but maybe use a bash script)
  2. Don't output anything else and let foreach recombine with rbind(). This is what I had done but then would have to rerun models to get results for making figures of the abundance across networks.
  3. Delay each iteration by a different amount to make less likely to write at exactly the same time (this is a hack that probably wouldn't work, definitely not consistently).
djhocking commented 8 years ago

Went with option 1 and used read_csv() to import so recombining went petty quickly. This is commit: 2b1c2b386f61aa39574c27172505ae5b90b0e49f