search

rstudio / keras3

R Interface to Keras
https://keras3.posit.co/
Other
833 stars 282 forks source link

Some required packages prohibit parallel processing: ‘keras’ #1223

Open mbac opened 3 years ago

mbac commented 3 years ago

Hi,

I’m getting the titular warning when I try to {tune} a {workflowset} object comprising a keras neural network.

I’m running the following code that may be relevant:

set.seed(92511)
future::plan("multisession")

# Grid controls
wset_grid_ctrls <-
    control_grid(
        save_pred = TRUE,
        parallel_over = "everything",
        save_workflow = TRUE
    )

Is the code really not running in parallel processing, or is it a false alarm? Thanks!

t-kalinowski commented 3 years ago

Hi, can you please post a minimal reprex, so I can reproduce the issue?

FWIW, keras/tensorflow implements it's own parallelism and concurrency, and, in general, doesn't play nice when other tools try to parallelize it by forking its process and/or assuming concurrently running keras training sessions will equitably share resources (e.g., GPU RAM) with each other on the same machine.