Ensure that SKA env var has a consistent meaning and usage

[taldcroft]

In Skare (classic), SKA always means the root of a Ska runtime environment under which one can find data, share, bin, lib etc.

In Skare3, SKA is actually used mostly to locate data. The runtime environment code (including share) generally live somewhere entirely different. Standalone users are free to set SKA to wherever the data live.

There is a conflict with flt_envs, which will override SKA and set it to the code root. This is not a huge problem because in a standalone Ska3 one would not normally source flt_envs nor use the launchers. In addition there is always the option to make a soft link for $SKA/data to the actual data location.

In production, $SKA/data should definitely exist and link to (or contain) the required data.

In Skare3, the share directory is required to live within the code root as "configured / installed" files, so if a user has a different SKA (to point at their data), there will be an issue.

Here are specific recommendations:

• The SKA environment variable is (unambiguously) defined as the directory where a data directory with package non-static data can be found.
• The SKA_ARCH_OS env var is used in a production setting (e.g. task_schedule.cfg) to point at the environment root where one finds bin, lib, and share directories. A bash launcher can use code like the starcheck_sandbox. Python code can use sys.prefix.
• flt_envs / ska_envs will always set SKA to the conda root.
• If an environment is used with flt_envs / ska_envs, then SKA/data needs to exist (can be a soft link).
• Require that the SKA environment variable be set to the directory with data underneath. In Ska2 some packages default to /proj/sot/ska if SKA isn't defined, but for Ska3 don't do that, raise an exception otherwise. We don't need to immediately scrub for this, but fix along the way when package updates occur.
• Deprecate use of SKA_DATA or any other SKA_* env vars.
• Encourage use of ska_path again as a more concise Ska-wide way of getting to the data directory.
• Use of ska_path is discouraged. ${SKA}/data or Path(os.environ['SKA'], 'data') etc is more explicit and will always work. The trickery of the original ska_path is no longer needed.

Actions:

• [x] Update ska_path to require SKA. ~and special-case share to be w/respect to SKA_ROOT~. Also, I think that require_match should be the default. Done in https://github.com/sot/ska_path/pull/4, this just needs to be merged.
• [ ] Migrate the policies above into a Skare3 design doc

[jeanconn]

Mostly with you, I just don't understand the text "in a standalone Ska3 that is not the root env".

[taldcroft]

I struck out the "that is not the root env". That thought was not fully formed.

[taldcroft]

I edited the main description with (hopefully) resolution and actions.

[jeanconn]

Ah, didn't realize these were all actions for me.

[jeanconn]

Regarding the update to flt_envs, presently it overrides SKA

> # Use the convention that flt_envs must be in $SKA/bin to derive SKA and
> # OVERRIDE any existing value.
> ska = dirname(dirname(abspath(__file__)))  # from $SKA/bin/flt_envs to $SKA

I can't tell if you want it to continue this behavior or preserve SKA if already defined.

[taldcroft]

[EDIT] On reflection, I disagree with this approach and would prefer saying with $SKA_ARCH_OS. See discussion in https://github.com/sot/mica/pull/201#discussion_r303407178.

---

flt_envs should always modify SKA even if it is defined. That's very fundamental to our whole concept of ska_envs being used to unambiguously get into a clean environment.

So I edited the main description, but after looking at it this time around it seems there is actually no need for SKA_ROOT nor SKA_ARCH_OS. There are two cases:

1. flt_envs was run and user expects to be a in production environment where $SKA/data and $SKA/share are guaranteed to exist (among other things)
2. flt_envs not run, so only $SKA/data needs to exist.

So in the canonical example of acis_taco, https://github.com/sot/acis_taco/blob/master/task_schedule.cfg#L23 could just be:

bin_dir      $ENV{SKA}/share/acis_taco      # Bin dir (optional, see task def'n)

The original SKA_ARCH_OS is only needed for Ska2 compatibility. So we really don't need a new SKA_ROOT and we can just use SKA for Ska3-only apps.

Does this make sense? I've gone around in a few circles on this and not sure if it is all consistent.

[taldcroft]

An important point is that SKA_ROOT only comes into existence when flt_envs is run, so by that time SKA has been set or reset to exactly the same thing.

[jeanconn]

OK, so are we adding SKA_ROOT or not bothering because it is just SKA if we are in a configured-like-a-production-ska environment and have actually run flt_envs?

[taldcroft]

Not bothering. I just wanted to run this by you before editing the manifesto above.

[jeanconn]

Yes, I think I'm fine with it, I just got confused by your followup "An important point..." as if we're not making a SKA_ROOT there isn't much that is important about it :smile: