Support for environment(env) variable format.

[ajays20078]

Can we have support for linux environment variable file.

Format is similar to properties file, just that while dumping the file, it should not have space on either side(Linux cannot source with spaces, it throws error). Fixing that line to

- stream.write("%s = %s\n" % (key, escape(val))) + stream.write("%s=%s\n" % (key, escape(val)))

will do it. So it can be a different backend altogether, or there could be some kind of a flag where we can specify whether it should be dumped as env file or properties file.

[ajays20078]

Actually it needs a different backend, because currently while dumping anyconfig, there are no quoted strings so if a file contains the below value

key = "value with spaces"

if we load and dump it, while dumping it, anyconfig dumps the properties file without double quotes. key = value with spaces but if we export it as env file it will set $key as "value" and not as "value with spaces".

[ssato]

First, thanks a lot for your suggestion!

AFAIK, there is no standard format of environment variables' definitions available. It's just a shell script and may have shell commands and other complex shell statements like conditionals and loops. I don't know how to distinguish between definitions of variables and others with stability. Also, anyconfig should need to have another backend for this support as you said. Right now, I think that more study is needed before starting to implement it.

BTW, shlex might help parsing quoted strings.

[ajays20078]

Agree that shell can have complex statements and commands, but can we not have support for the simplest environment variable file which will be a key=value pairs and in later versions try and add new features?

[ssato]

I want to make sure its use cases also. Shells can process them correctly even if there are complex shell statements and/or shell variable expansions are required. Is there any practical and valuable cases that these files are processed by python instead of shells?

Java properties is a similar case but it's easy to parse it and I thought that its support in anyconfig should be useful because python is lightweight compared to Java. Also its format feels well-defined for me.

Anyway, we have to define its format clearly if it's needed in advance. I guess that it should be hard to implement in python if the right hand value 'value' has some statements or shell expansions or defined in multiple lines. In other words, it can support only simple cases such as key=single_line_of_immediate_value (string).

[ajays20078]

I can't think of a specific use case but here are a few generic usecases:

1. If you want to develop a config parser which parses any type of config and may be even update certain keys (config service), which could be written in python.
2. If you want to do some automations and extract certain values to prepare a env file which should be sourced by shell or docker instances and if you are using python for the same. So for any system automations involving python , this feature could be really useful.

True it can support only simple cases, but can't we call out explicitly that for env files only the predefined format is supported?

[ssato]

Hi,

Excuse me for a late reply. I've been on summer holidays.

I added an experimental parser for files contain shell variables' definitions in the 'shellvars' branch (commit 0ed0d59). Could you please take a look at it and give me feedback? I think that it may cover this

[ajays20078]

Thank you for the feature addition. Testing it out, will revert back on this.

[ajays20078]

anyconfig.load doesn't seem to work here, am i missing something?

FileName - test.shellvars Contents: one="1" var2="something else" var3=dskjfkd

import anyconfig a= anyconfig.load("test.shellvars") a

[ssato]

Oops, sorry, I forgot to add its parser. I've pushed the required fix. Now it should work like this:

>>> anyconfig.loads("a='bbb'", ac_parser="shellvars")
{'a': 'bbb'}

And please note that because the extension of configuration files of this type is not clear, its type has to be given explicitly.

[ajays20078]

Hi,

The new parser works perfectly, can we have it auto detect the configuration files of type .env so that we can avoid specifying ac_parser while loading/dumping these files?

[ssato]

Hi @ajays20078,

I also want that happen ;-) but I'm not sure '.env' extension is popular enough. So I want to keep it is now until most appropriate ones are found.

[ssato]

Can I close this as the feature is in since 0.7.0?

[ajays20078]

Yeah it can be closed. Have one last question regarding this, does this automatically work with certain extension (.sh or .env) or should we still explicitly specify ac_parser?

[ssato]

Of course, I want to make it automatically detect as so by file extensions, but I'm not sure .sh/.env for such configuration files are popular enough. Rather than that, I guess that these configuration files tend to have no file extensions just like /etc/sysconfig/* do.

ssato@x1carbon% for f in $(locate .sh | grep -v home | grep etc); do grep -qE '^[A-Z]*=' $f 2>/dev/null && echo $f; done
/etc/X11/xinit/xinitrc.d/50-xinput.sh
/etc/kernel/postinst.d/51-dracut-rescue-postinst.sh
/etc/profile.d/qt.sh
/usr/share/doc/git-core-doc/contrib/examples/git-fetch.sh
ssato@x1carbon% for f in $(locate .env | grep -v home | grep etc); do grep -qE '^[A-Z]*=' $f 2>/dev/null && echo $f; done
ssato@x1carbon% for f in $(locate .sh | grep -v home | grep etc); do grep -qE '^[a-z]*=' $f 2>/dev/null && echo $f; done
/etc/NetworkManager/dispatcher.d/10-ifcfg-rh-routes.sh
/etc/NetworkManager/dispatcher.d/no-wait.d/10-ifcfg-rh-routes.sh
/etc/NetworkManager/dispatcher.d/pre-up.d/10-ifcfg-rh-routes.sh
/etc/X11/xinit/xinitrc.d/50-xinput.sh
/etc/profile.d/lang.sh
/usr/libexec/netcf-transaction.sh
/usr/share/doc/git-core-doc/contrib/examples/git-fetch.sh
/usr/share/doc/packagemaker-core/examples/list_etc_files_not_from_rpms.sh
ssato@x1carbon%                                     ~/repos/public/github.com/ssato/python-anyconfig.git