Standardize how to do environment variable parsing & substitution

[karrtikr]

There're two parts to this issue:

Parsing

We should probably always follow dotenv syntax:

• Circuit Python: https://docs.circuitpython.org/en/latest/shared-bindings/dotenv/index.html# It's almost very similar to what we do; except for the multiline support and comments on the same line, so good news is it won't break us if we decide to align.

• Python dot-env: https://pypi.org/project/python-dotenv/ Match with how it does variable substitution: https://github.com/microsoft/vscode-python/issues/18307

Substitution

There are currently two ways of substituting variables in a .env file:

VAR2="value;${env:VAR1}"

and

VAR2="value;${VAR1}"

and a combination of both has to used to resolve all variables in different situations. However, this is not standard syntax. Again we should probably rely on the dotenv syntax always.

• ${env:...} only works for process env variables (for eg. PATH) when debugging or when resolving env variables in settings.
  • Exception: There's a bug due to which debugger doesn't resolve ${env:...} in .env files when debugging a Python file: microsoft/vscode-python-debugger#159. Make sure https://github.com/microsoft/vscode-python-debugger/issues/159 is taken care of with this issue.
• "Recursive" variable substitution does not work with ${env:...}. It works only with ${...}. Need to correct the docs: https://github.com/microsoft/vscode-python/pull/17747#pullrequestreview-855395670.
• .env file is currently not supported when running code in terminal: https://github.com/microsoft/vscode-python/issues/944
• ${...} can not be used for process env variables when debugging: https://github.com/microsoft/debugpy/issues/821. Solution:
  • Debugging doesn't use environment variable substitution/resolution support in the extension. Expose https://github.com/microsoft/vscode-python/blob/456ac068e95ba600079ddcdc0cc7551d1659a699/src/client/common/variables/environmentVariablesProvider.ts#L56 as an API and have the debugger use it to fetch env variables.
  • Follow the standard python-dotenv syntax to resolve process env variables in debugger: https://pypi.org/project/python-dotenv/, i.e add support to resolve using ${...} in addition to ${env:...}.

[starball5]

Related on Stack Overflow: Why does VSCode show inline comments as part of environment variables from .env file?

[ababak]

Any updates on this? The lack of support for newlines is problematic.