search

gadget-framework / rgadget

Rgadget is a set of useful utilities for gadget
6 stars 12 forks source link

gadget_iterative_stage_* mobilizes only 1 core #117

Open vbartolino opened 3 years ago

vbartolino commented 3 years ago

I can't explain why monitoring the system while running the procedure below shows that only 1 core is up running.

gadget_iterative_stage_1(gd, params.in = 'params.in') %>% 
       parallel::mclapply(gadget_optimize, mc.cores = parallel::detectCores()) %>% 
       gadget_iterative_stage_2() %>% 
       gadget_optimize()
bthe commented 3 years ago

There are two alternatives that might cause this:

vbartolino commented 3 years ago

parallel::detectCores() is not detecting the number of cores on your computer

that seems fine

> parallel::detectCores()
[1] 8

First, how many gadget directories come out of the stage 1 function? The function should generate a list of gadget directories of length as long as the number of likelihood components.

it generates the directory 'WGTS' with many files inside, repeated for each likelihood component, but not sub-directories within (see below). I'm mainly surprise by seeing that only 1 core runs...

> gadget_iterative_stage_1(gd,
                         grouping=list(
                          si_bits14=c("si.cod.bits1","si.cod.bits4")),
                         params.in = 'params.in',
                         wgts='WGTS') %>% 
  parallel::mclapply(gadget_optimize, mc.cores = parallel::detectCores(logical=T))
gadget_iterative_stage_1(gd,
+                          grouping=list(
+                           si_bits14=c("si.cod.bits1","si.cod.bits4")),
+                          params.in = 'params.in',
+                          wgts='WGTS') %>% 
+   parallel::mclapply(gadget_optimize, mc.cores = parallel::detectCores(logical=T))
$si_bits14
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.si.cod.bits1__si.cod.bits4"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.si.cod.bits1__si.cod.bits4"
attr(,"params_out")
[1] "WGTS/params.si.cod.bits1__si.cod.bits4"

$ldist.cod.bits1
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.ldist.cod.bits1"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.ldist.cod.bits1"
attr(,"params_out")
[1] "WGTS/params.ldist.cod.bits1"

$ldist.cod.bits4
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.ldist.cod.bits4"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.ldist.cod.bits4"
attr(,"params_out")
[1] "WGTS/params.ldist.cod.bits4"

$ldist.cod.act
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.ldist.cod.act"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.ldist.cod.act"
attr(,"params_out")
[1] "WGTS/params.ldist.cod.act"

$ldist.cod.pas
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.ldist.cod.pas"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.ldist.cod.pas"
attr(,"params_out")
[1] "WGTS/params.ldist.cod.pas"

$adist.cod.act
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.adist.cod.act"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.adist.cod.act"
attr(,"params_out")
[1] "WGTS/params.adist.cod.act"

$adist.cod.pas
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.adist.cod.pas"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.adist.cod.pas"
attr(,"params_out")
[1] "WGTS/params.adist.cod.pas"

$si.cod.bitsH
[1] "cod01.tmp"
attr(,"variant_dir")
[1] "WGTS"
attr(,"mainfile")
[1] "WGTS/main.si.cod.bitsH"
attr(,"class")
[1] "gadget.variant" "list"          
attr(,"params_in")
[1] "WGTS/params.si.cod.bitsH"
attr(,"params_out")
[1] "WGTS/params.si.cod.bitsH"
bthe commented 3 years ago

Yes, there should not be a number of folders created, just WGTS but the gadget_variant_dir refers to different main files with the WGTS folder. How do the params files look like?