Error: object 'frm' not found when running farmR in parallel

[michstrauch]

@biopsichas With the foreach approach that is implemented to parallelize the farmR stuff (step 6), I got an error (object 'frm' not found). At least on our Windows it doesn't work. Some R6 class environment problem in the different workers.

Just to let you know and in case others run into the same problem, here my workaround:

##Run parallelization
library(doFuture)

future::plan(multisession, workers = 30)

txt_info <- foreach(d = m_dir, .options.future = list(packages = c("SWATfarmR", "tidyverse", "stringr"),
                                           globals = structure(TRUE, add = "frm"))) %dofuture% {write_mgt(d, periods)}

[biopsichas]

Hi Micha, if you use main branch you get this problem. But sv_dev is updated. Anyhow you have information here what to do https://github.com/biopsichas/cliwf/issues/1

[JoanaEichenberger]

Dear @michstrauch, I tried your workaround. I still get the error "object 'frm' not found" but a little later than without it. Did you also experience that?

[michstrauch]

No sorry @JoanaEichenberger, in my case it worked without any error, but I also had no issue with the project_type argument (I really wonder why this is causing still trouble as this should have been solved).
The error I got was purely related to parallelization on our Windows cluster, where executing functions from a local environment (frm) in each of the parallel workers did not work.

[kassaipiroska]

I am in the same boat as Joana and I got exactly the same error with Micha's solution too. I updated the workflow today as Svajunas recommended but still have the problem.

[biopsichas]

Sorry for this guys. I have properly tested on my version of SWATfarmR 3.2.0, but didn't find time to do it on Christoph's version of SWATfarm 4.0.2.

If I include this into the function, it runs for me. But I need a bit more time to say for sure it will provide the right result.

Maybe you can test this too

[JoanaEichenberger]

Dear all, I de-installed the SWATfarmR and installed it again. Although I have installed 3.2.0 before, something must have gone wrong the first time. Now it has been running already for 3h without any error message - I hope this means the problem is solved. Maybe you have to try that as well @kassaipiroska.

[kassaipiroska]

I also re-installed SWATfarmR 3.2.0 and it runs without errors now. Maybe something still wrong with the newest SWATfarmR to fit this workflow.

[biopsichas]

I updated the script. So now it should be able to understand, which version of SWATfarmR is used and run code accordingly.

[kassaipiroska]

@biopsichas How can I check that run parallelization (step 6) goes properly? I started it two days ago (it is still running) and I don't see any changes in my folders. Where can I check it?

[biopsichas]

Just go into results folder and search management.sch files. You should have 54 in the end. With each loop you should get closer to this number.