MUCollective / multiverse

R package for creating explorable multiverse analysis
https://mucollective.github.io/multiverse/
GNU General Public License v3.0
62 stars 5 forks source link

Execution of default analysis in the global environment conflicts with extract_variables #83

Closed abhsarma closed 2 months ago

abhsarma commented 4 years ago

Since we have the global env as the parent of the multiverse environments, and the default analysis is being executed in the global environment, its causing a few problems:

  1. Say there was an error in one of the universes. If you call extract_variables it will automatically get the object in the global environment instead of giving a NULL for that universe
  2. If the data is loaded outside the multiverse instead of in it, and the default execution rewrites it, it will cause a problem in the execution of the other universes and is a pain to debug.

I think the simple solution to both is to just not run anything in the global environment? I think (1) is a bigger problem than (2). @mjskay thoughts?

mjskay commented 4 years ago

Hmmm (1) is an interesting problem. One question is, what are the semantics of a universe that has been executed but returned an error? Like, what's the rule from which the appropriate behavior in both cases should be determined?

If the semantics are "variables from universes with an error are undefined", then extract_variables should return some version of "nothing" (NULL or NA?) for any variable, regardless of which environment it was defined in. That is one straightforward fix. On the other hand, I'm not sure those are the correct semantics (or maybe it would need a flag to allow it to return a value), because otherwise it might make debugging hard if someone wanted to inspect values up until the error occurred.

If the semantics are "variables from universes have whatever values they had immediately before the error occurred", then extract_variables should return the value from the global environment, no? Because it still was the value defined in that universe Except if both (1) and (2) are happening at the same time, which leads me to my next question...

What execution mode are we talking about? Interactive or non-interactive? It might help clarify the expected semantics by defining what execution mode we mean. My guess is they might lead to different conclusions, especially if we later want to add another execution mode that deals with packages that expect everything in the global environment (in which case we might be talking a model where every universe is executed end-to-end in the global environment in separate sessions or something).

Maybe it would help to clarify by writing out what the different execution modes are and what their expected semantics are.

abhsarma commented 2 months ago

At some point, and unfortunately, I don't remember when, a(n executive?) decision was made to not run any code in the global environment and I don't recall the rationale for it.

As a result, this problem mostly disappears. Currently the behavior is:

If there's an error: