yaml out when yaml in

[splaisan]

This is sad, I would like to get text output from yaml input in order for instance to use the tool to declare variables from yaml as done at https://gist.github.com/pkuczynski/8665367

[4383]

Hello,

First, thank you for your proposal.

Please, can you giving to us an example of your needs?

If I understand you correctly you want to do something like this:

$ MY_YAML='{"foo": "bar", "fizz": {"buzz": ["zero", "one", "two", "three"]}}'
$ MY_VARIABLE=$(echo $MY_YAML | niet -f dquote "fizz")
$ echo $MY_VARIABLE
{'buzz': ['zero', 'one', 'two', 'three']}
$ echo $MY_VARIABLE | niet -f dquote "buzz[0:2]"
"zero" "one"

Then you can chain/pipe the niet call to reuse outputs with something like this who will be similar to my previous example:

$ MY_YAML='{"foo": "bar", "fizz": {"buzz": ["zero", "one", "two", "three"]}}'
$ echo $MY_YAML | niet -f dquote "fizz" | niet -f dquote "buzz[0:2]"
"zero" "one"

And store your output into a variable with something like this:

$ MY_YAML='{"foo": "bar", "fizz": {"buzz": ["zero", "one", "two", "three"]}}'
$ MY_VARIABLE=$(echo $MY_YAML | niet -f dquote "fizz" | niet -f dquote "buzz[0:2]")
$ echo $MY_VARIABLE
"zero" "one"

Please tell me if I correctly understood your request.

[splaisan]

my usage would be to get a bunch of variables in bash from a yaml file that is otherwise used in snakemake or R; I want to share this yaml info between bash and other languages for which the yaml was created in teh first place. This allows me to use the variables in bash and produce other content without having to redefine them in the bash script itself. minimal example:

config.yaml

# species database for enrichment with InterMine
mine: "YeastMine"
orgdb: "org.Sc.sgd.db"
threads: 80
pychopper: TRUE

Currently I use a custom function

# from: https://gist.github.com/pkuczynski/8665367
parse_yaml ()
{
    local prefix=$2;
    local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034');
    sed -ne "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" -e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" $1 | awk -F$fs '{
      indent = length($1)/2;
      vname[indent] = $2;
      for (i in vname) {if (i > indent) {delete vname[i]}}
      if (length($3) > 0) {
         vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
         printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, $2, $3);
      }
   }'
}

$ eval $(parse_yaml config.yaml)

Now I can use therm in bash like:

echo ${orgdb}
> org.Sc.sgd.d
# or 
echo ${threads}
> 80

(obviously not only to echo them)

I hope this is now more clear, although I am perfectly happy with the custom function , it is possible that more complex yaml structures like arrays or hashes do not translate well with it (or content including spaces or special characters)

Thanks for your help

[4383]

@splaisan thanks for your well done comment!

Indeed it can be interesting to introduce some new features related to your needs. It don't seem to be a big deal to implement them :)

I will try to spend some time to implement them soon and/or you can propose patches through pull requests to implement them.

[splaisan]

Nice, looking forward to test them on your next/test version Merci Hervé!

[4383]

@splaisan my pleasure. De rien :)

I'll ping you when I would have some updates to share with you.

[4383]

@splaisan feel free to test, take a look, and review: https://github.com/openuado/niet/pull/51

Thanks

[4383]

Now implemented, available in version 1.7.0:

• https://github.com/openuado/niet/releases/tag/1.7.0
• https://github.com/openuado/niet/commit/799b1cf23549f85ef0c92c8e57de1917b57b7b57
• https://pypi.org/project/niet/1.7.0/

Upgrade niet and get this feature by using:

$ pip install -U niet

Thanks.

[splaisan]

thanks Hervé, upgrade worked, I have now 1.7.0