pachyderm / pachyderm

Data-Centric Pipelines and Data Versioning
https://www.pachyderm.com/
Apache License 2.0
6.18k stars 567 forks source link

set environment variables to all caps when running pipelines, provide naming guidance #3652

Open brokenjacobs opened 5 years ago

brokenjacobs commented 5 years ago

What is the goal / desired outcome? Per: http://docs.pachyderm.io/en/latest/reference/pipeline_spec.html#environment-variables pachyderm will set several environment variables for each job. Tools like shellcheck depend on environment variables being all caps (see: https://github.com/koalaman/shellcheck/issues/356)

In addition to this, several environment variables are named after repositories. If your repository contains an invalid character for an environment variable (like the '-' or '=' characters) this will break.

The open group guidelines for environment variables are:

Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation; applications shall tolerate the presence of such names.

If there is a way to accomplish this today via workaround, what does that require? Not that I know of

(Optional) What is your proposal for a feature to solve this? First I would capitalize all environment variable names, as well as leaving the existing environment variables in place for backwards compatibility reasons.

Looking to the future, I think some reasoning needs to be done about valid character sets for repository/pipeline names to be compliant with this guidance.

(related to Zendesk ticket #47)
gz#47

JoeyZwicker commented 5 years ago

Thanks for submitting @brokenjacobs . We'll take a look at tidying this up in an upcoming sprint

PachydermSupport commented 3 years ago
Agent Jan Schwoebel linked Freshdesk ticket 231 for this issue.
mindthevirt commented 3 years ago
Agent Dale Georg linked Freshdesk ticket 231 for this issue.