Closed nickgarber closed 6 years ago
gtmanfred
commented
6 years ago Bash removes the quotes. You will need to add a second single quote into the cli for it to actually be quoted.
salt-call sdb.set sdb://example/setitoff "'$(base64 -w0 my-demo-file)'"
nickgarber
commented
6 years ago Hey @gtmanfred, the nested quoting doesn't help. I think the quoting is working correctly. It's producing a single "word" and passing it to SDB, but somewhere along the line it's actually parsing inside the quotes looking for a keyword pair.
I haven't code dived but I approximate from testing that the parser is looking for keyword pairs from inside the string argument. Sounds kooky I know, but all three of the following produce the same error:
sdb.set sdb://example/setitoff 'jlkjhl='
sdb.set sdb://example/setitoff "jlkjhl="
sdb.set sdb://example/setitoff "'jlkjhl='"
sdb.set sdb://example/setitoff "'$(echo jlkjhl=)'"How about "'\"jlkjhl=\"'"?
nickgarber
commented
6 years ago Just to be sure, I'm going to rebuild my test environment and retry each of these question/tests. Will have an update for later in the day.
nickgarber
commented
6 years ago The "'\"jlkjhl=\"'" input didn't error, but the quotes were recorded as part of the value which isn't particularly desirable.
salt-run sdb.get sdb://example/test
"jlkjhl=" Just to loop in on my more general question - is there a more preferable way to save base64-encoded binary data into a SQLite SDB than sdb.set from the CLI?
nickgarber
commented
6 years ago I'm trying to better understand this and came up with this question ... Where/How are the fun_args processed?
It looks like they are being split by the shell as positional parameters, then the positional arg is processed using type inference.
[root@salt-master-d1 ~]# salt-run sdb.set sdb://sdb_gema_exmpl__00a/testing123 ["a="] -l all ; salt-call pillar.item lookup:minion_nodegroup:SECRETS__sdb_gema_exmpl__00a:sdb_gema_exmpl__00a:entries
[DEBUG ] Sending event: tag = salt/run/20171227150914421022/new; data = {'fun': 'runner.sdb.set', 'fun_args': ['sdb://sdb_gema_exmpl__00a/testing123', ['a=']], 'jid': '20171227150914421022', 'user': 'sudo_vagrant', '_stamp': '2017-12-27T20:09:14.696940'}
[DEBUG ] LazyLoaded sqlite3.set
[DEBUG ] LazyLoaded local_cache.prep_jid
[DEBUG ] Sending event: tag = salt/run/20171227150914421022/ret; data = {'fun_args': ['sdb://sdb_gema_empl__00a/testing123', ['a=']], 'jid': '20171227150914421022', 'return': True, 'success': True, '_stamp': '2017-12-27T20:09:14.727089', 'user': 'sudo_vagrant
', 'fun': 'runner.sdb.set'}
[DEBUG ] LazyLoaded nested.output
[TRACE ] data = True
True
[INFO ] Runner completed: 20171227150914421022
[DEBUG ] Runner return: True
local:
----------
lookup:minion_nodegroup:SECRETS__sdb_gema_exmpl__00a:sdb_gema_exmpl__00a:entries:
----------
test:
"jlkjhl="
testing123:
- a=
Maybe this is to be expected, but I've found that if I wrap the string in syntax for a dictionary, set, or list, it works correctly, (though I still want a string value).
It may be I'm seeking a way to set the CLI character used to separate keyword pairs,
Something like --kwargs-separator, inspired by https://github.com/saltstack/salt/pull/5497 ?
A general-case demo of this problem/question is below ...
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a base64_data
local:
----------
args:
('this', 'is', 'a', 'base64_data')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173115999653', '__pub_pid': 22302, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a "base64_data"
local:
----------
args:
('this', 'is', 'a', 'base64_data')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173127010847', '__pub_pid': 22380, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a "base64_data="
local:
----------
args:
('this', 'is', 'a')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173131314892', 'base64_data': '', '__pub_pid': 22458, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]# salt-call test.arg_repr this is a "base64_data=="
local:
----------
args:
('this', 'is', 'a', 'base64_data==')
kwargs:
{'__pub_fun': 'test.arg_repr', '__pub_jid': '20171227173134217461', '__pub_pid': 22536, '__pub_tgt': 'salt-call'}
[root@salt-master-d1 ~]# Everything that you pass in has to first pass bash, and make it into python correctly, that is where I think the problem is here.
After that, it is then passed through pyyaml to load it so that dictionaries in json or yaml can be passed in from the commandline.
nickgarber
commented
6 years ago This looks to me like a problem introduced at the pyyaml processing layer, and I'd like to test this theory.
Is there a way to turn off the pyyaml processing for a positional argument?
Separate but related I noticed the --no-parse option seems to serve a similar purpose, (to prevent pyyaml parsing of values https://github.com/saltstack/salt/pull/5497), within keyword arguments, but dont see a corresponding flag for positional arguments.
Alternatively, is there a way to pass the value of an sdb.set command using a keyword argument (so that I can test using the --no-parse flag)?
Thanks for your help on this!
nickgarber
commented
6 years ago As a footnote, I'm working around this for the time being by wrapping base64-encoded data in a list at the CLI. It doesn't address the parsing question/challenge here, but unfortunately it does solve my use-case adequately (by just using a different datatype).
To base64 encode a file:
salt-run sdb.set sdb://demo/sshkey_master_prv__b64_lst "[\"$(base64 -w0 ~/.ssh/id_rsa | sed -ne 's/.\{76\}/&", "/gp')\"]"
To subsequently retrieve it:
{{ sdb://demo/sshkey_master_prv__b64_lst | join("") }}
gtmanfred
commented
6 years ago I am dumb, this is really simple, you just need to pass it to the keyword value, and it won't parse anything after that until the next space.
[root@salt ~]# salt-call sdb.set sdb://mysqlite/setitoff value=jlkjhl=
local:
True
[root@salt ~]# salt-call sdb.get sdb://mysqlite/setitoff
local:
jlkjhl=So you should be able to use this from the beginning example.
salt-call sdb.set sdb://example/setitoff value="$(base64 -w0 my-demo-file)"And it will put whatever comes after the value= and assign it to the value keyword arguement from sdb.set
https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.sdb.html#salt.modules.sdb.set
Thanks, Daniel
nickgarber
commented
6 years ago Sweet! Thats a good remedy, thanks!
Are the function argument variables automatically available as kwargs at the CLI?
gtmanfred
commented
6 years ago all function arguments can be passed as named arguments from the command line, and do not require passing them as positional arguments.
Description of Issue/Question
Setup
salt-call sdb.set sdb://example/setitoff "$(base64 -w0 my-demo-file)"Outcome:
Expected Outcome:
Observation:
salt-call sdb.set, and works as expected in every other module/case.Implication:
Steps to Reproduce Issue
Versions Report