Closed nickgarber closed 6 years ago
Bash removes the quotes. You will need to add a second single quote into the cli for it to actually be quoted.
salt-call sdb.set sdb://example/setitoff "'$(base64 -w0 my-demo-file)'"
Hey @gtmanfred, the nested quoting doesn't help. I think the quoting is working correctly. It's producing a single "word" and passing it to SDB, but somewhere along the line it's actually parsing inside the quotes looking for a keyword pair.
I haven't code dived but I approximate from testing that the parser is looking for keyword pairs from inside the string argument. Sounds kooky I know, but all three of the following produce the same error:
sdb.set sdb://example/setitoff 'jlkjhl='
sdb.set sdb://example/setitoff "jlkjhl="
sdb.set sdb://example/setitoff "'jlkjhl='"
sdb.set sdb://example/setitoff "'$(echo jlkjhl=)'"
How about "'\"jlkjhl=\"'"
?
Just to be sure, I'm going to rebuild my test environment and retry each of these question/tests. Will have an update for later in the day.
The "'\"jlkjhl=\"'"
input didn't error, but the quotes were recorded as part of the value which isn't particularly desirable.
salt-run sdb.get sdb://example/test
"jlkjhl="
Just to loop in on my more general question - is there a more preferable way to save base64-encoded binary data into a SQLite SDB than sdb.set from the CLI?
I'm trying to better understand this and came up with this question ... Where/How are the fun_args processed?
It looks like they are being split by the shell as positional parameters, then the positional arg is processed using type inference.
[root@salt-master-d1 ~]# salt-run sdb.set sdb://sdb_gema_exmpl__00a/testing123 ["a="] -l all ; salt-call pillar.item lookup:minion_nodegroup:SECRETS__sdb_gema_exmpl__00a:sdb_gema_exmpl__00a:entries
[DEBUG ] Sending event: tag = salt/run/20171227150914421022/new; data = {'fun': 'runner.sdb.set', 'fun_args': ['sdb://sdb_gema_exmpl__00a/testing123', ['a=']], 'jid': '20171227150914421022', 'user': 'sudo_vagrant', '_stamp': '2017-12-27T20:09:14.696940'}
[DEBUG ] LazyLoaded sqlite3.set
[DEBUG ] LazyLoaded local_cache.prep_jid
[DEBUG ] Sending event: tag = salt/run/20171227150914421022/ret; data = {'fun_args': ['sdb://sdb_gema_empl__00a/testing123', ['a=']], 'jid': '20171227150914421022', 'return': True, 'success': True, '_stamp': '2017-12-27T20:09:14.727089', 'user': 'sudo_vagrant
', 'fun': 'runner.sdb.set'}
[DEBUG ] LazyLoaded nested.output
[TRACE ] data = True
True
[INFO ] Runner completed: 20171227150914421022
[DEBUG ] Runner return: True
local:
----------
lookup:minion_nodegroup:SECRETS__sdb_gema_exmpl__00a:sdb_gema_exmpl__00a:entries:
----------
test:
"jlkjhl="
testing123:
- a=
Maybe this is to be expected, but I've found that if I wrap the string in syntax for a dictionary, set, or list, it works correctly, (though I still want a string value).
It may be I'm seeking a way to set the CLI character used to separate keyword pairs,
Something like --kwargs-separator
, inspired by https://github.com/saltstack/salt/pull/5497 ?
A general-case demo of this problem/question is below ...
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a base64_data
local:
----------
args:
('this', 'is', 'a', 'base64_data')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173115999653', '__pub_pid': 22302, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a "base64_data"
local:
----------
args:
('this', 'is', 'a', 'base64_data')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173127010847', '__pub_pid': 22380, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a "base64_data="
local:
----------
args:
('this', 'is', 'a')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173131314892', 'base64_data': '', '__pub_pid': 22458, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a "base64_data=="
local:
----------
args:
('this', 'is', 'a', 'base64_data==')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173134217461', '__pub_pid': 22536, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]#
Everything that you pass in has to first pass bash, and make it into python correctly, that is where I think the problem is here.
After that, it is then passed through pyyaml to load it so that dictionaries in json or yaml can be passed in from the commandline.
This looks to me like a problem introduced at the pyyaml processing layer, and I'd like to test this theory.
Is there a way to turn off the pyyaml processing for a positional argument?
Separate but related I noticed the --no-parse
option seems to serve a similar purpose, (to prevent pyyaml parsing of values https://github.com/saltstack/salt/pull/5497), within keyword arguments, but dont see a corresponding flag for positional arguments.
Alternatively, is there a way to pass the value of an sdb.set
command using a keyword argument (so that I can test using the --no-parse
flag)?
Thanks for your help on this!
As a footnote, I'm working around this for the time being by wrapping base64-encoded data in a list at the CLI. It doesn't address the parsing question/challenge here, but unfortunately it does solve my use-case adequately (by just using a different datatype).
To base64 encode a file:
salt-run sdb.set sdb://demo/sshkey_master_prv__b64_lst "[\"$(base64 -w0 ~/.ssh/id_rsa | sed -ne 's/.\{76\}/&", "/gp')\"]"
To subsequently retrieve it:
{{ sdb://demo/sshkey_master_prv__b64_lst | join("") }}
I am dumb, this is really simple, you just need to pass it to the keyword value
, and it won't parse anything after that until the next space.
[root@salt ~]# salt-call sdb.set sdb://mysqlite/setitoff value=jlkjhl=
local:
True
[root@salt ~]# salt-call sdb.get sdb://mysqlite/setitoff
local:
jlkjhl=
So you should be able to use this from the beginning example.
salt-call sdb.set sdb://example/setitoff value="$(base64 -w0 my-demo-file)"
And it will put whatever comes after the value=
and assign it to the value
keyword arguement from sdb.set
https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.sdb.html#salt.modules.sdb.set
Thanks, Daniel
Sweet! Thats a good remedy, thanks!
Are the function argument variables automatically available as kwargs at the CLI?
all function arguments can be passed as named arguments from the command line, and do not require passing them as positional arguments.
Description of Issue/Question
Setup
salt-call sdb.set sdb://example/setitoff "$(base64 -w0 my-demo-file)"
Outcome:
Expected Outcome:
Observation:
salt-call sdb.set
, and works as expected in every other module/case.Implication:
Steps to Reproduce Issue
Versions Report