Dagger @spawn can't launch workers from Pluto

[Oblynx]

Hi! I try to use Dagger.jl inside a Pluto cell. However, it can't seem to spawn even 1 worker when used from Pluto, while the same code runs with plain Julia.

Here's the Pluto notebook: https://gist.github.com/Oblynx/a0846ac1d1d021b52d52fa675c48aaa1

Here's the error message in Pluto:

On worker 1:

KeyError: key Dagger [d58978e5-989f-55fb-8d15-ea34adc7bf54] not found

[pankgeorg]

This is related to #300, and unfortunately it is not trivial to fix soon!

[Oblynx]

Oh this is indeed unfortunate! I wonder why Distributed can't nest.

[pankgeorg]

Pluto server runs on process 1 and the notebooks get a worker number. In general, apps that use Distributed assume they are process 1 and their children are processes 2:N. When you run from pluto, both assumptions break, as Distributed only has one nesting level.

[Pangoraw]

It seems that Dagger is trying to spawn the task from process id 1, so you can @spawn if you also import Dagger in the process where you launch Pluto from:

using Pluto, Dagger
Pluto.run()

I was then able to make your example work but note that there may be other issues down the line because of #300 as @pankgeorg said !