Syntax clarification wanted

[HolQue]

In

https://github.com/test-fullautomation/python-jsonpreprocessor/issues/74

I wrote that the retest was successful. The issue has been closed.

But now I stumble about my implementation of the self test JPP_1001:

"param1" : {},
${param1}['subkey']['subkey']['subkey'] : "subkey value",

I asked myself why I implemented this line: "param1" : {},

Taking a look at the requirement

https://github.com/test-fullautomation/python-jsonpreprocessor/issues/74#issue-1767641397

it seems to me that this initialization "param1" : {}, should not be necessary. But when I remove the line, then I get the following error:

Could not set variable 'param1['subkey']['subkey']['subkey']' with value 'subkey value'! Reason: name 'param1' is not defined'

Taking a second look at the requirement I realize, that the scope of the example might be the TestsuitesManagement (because of the usage of params.global).

Here the TestsuitesManagement itself takes care of the creation of the top level dictionary.

But how about the pure JsonPreprocessor: Shall the initialization "param1" : {} necessary here, or not?

[namsonx]

Hello Holger,

Your concern makes me think of the following scenarios below:

• First, I agree that the definition "param1" : {}, before creating sub-element of param1 looks not nice, we could also create a data structure of the top level dictionary.
• By mistake users set **${paran1}**['subkey']['subkey']['subkey'] : "subkey value" where paran1 was not defined yet, then new variable **paran1** is created. But users use the variable **param1**['subkey']['subkey']['subkey'] in their program. Now, the error message caused by the variable **param1** is not available occurs while users may be not aware of their typo, and it's also not nice.

We should discuss in our meeting together with Thomas, then make a decision for this case.

Thank you, Son

[HolQue]

Hi Son,

hmm, typos. This is an interesting hint. My understanding of "implicit creation is": What does not exist, will be created. And this is for sure susceptible to typing mistakes.

This can happen at every sub key level, e.g.:

${param1}['subkey']['subbbbkey']['subkey']

The JsonPreprocessor cannot know which is the correct spelling and which is the incorrect spelling.

But you are right: We have to specify the requirements in more detail here.

[HolQue]

Another idea: How about a parameter to control the implicit creation?

dictReturned = json_preprocessor.jsonLoad(json_file, create_implicit=True)

Let's say: create_implicit=True is the default and can be omitted in jsonLoad. But with

dictReturned = json_preprocessor.jsonLoad(json_file, create_implicit=False)

the user has the chance to deactivate the implicit creation.

With implicit creation activated, the user gets all missing elements (dictionaries and keys of the dictionaries at all levels). Typing mistakes do not matter here.

With implicit creation deactivated, the JsonPreprocessor stops with an error message in case of any required element does not exist.

And this means we give the users the chance to decide for themself, if they want to go the more flexible (and more dangerous way) or if they prefer to be on the save (and not so much flexible) side.

[test-fullautomation]

Hi Holger,

unfortunately I disagree. Maybe we can do this switch later.

We should always do implicit creation. A switch increases the complexity in testing.

Thank you, Thomas

[HolQue]

Hi Thomas,

that's fine to me.

But I want to update the documentation. Therefore I urgently need to know: Is the first line of the following code still required or shall the initial creation of the dictionary be part of the implicit creation?

"param1" : {},
${param1}['subkey']['subkey']['subkey'] : "subkey value",

Currently the first line is required.

[test-fullautomation]

Hi Holger,

the first line shall not be required. When we create subkeys implicitly, then there is no argument why not to create the base key also.

Thank you, Thomas

[HolQue]

Hi Thomas, hi Son,

while updating the documentation I continued with playing around with possibilities. Outcome: Several issues need to be fixed:

(1) ${testdict.subKey.subKey.paramA} : "DEF" returns: Could not set variable ... Reason: name 'testdict' is not defined'

NOT OK

(2)

"testdict" : {},
${testdict.subKey.subKey.paramA} : "DEF"

returns: {'testdict': {'subKey': {'subKey': {'paramA': 'DEF'}}}}

OK

(3)

"testdict" : {},
"subKey"   : "ABC",
${testdict.subKey.subKey.paramA} : "DEF"

returns: {'subKey': 'ABC', 'testdict': {'subKey': {'subKey': {'paramA': 'DEF'}}}}

OK

(4)

"testdict" : {},
"paramA"   : "ABC",
${testdict.subKey.subKey.paramA} : "DEF"

returns: {'paramA': 'DEF', 'testdict': {}}

NOT OK

(5)

"paramA"   : "ABC",
${testdict.subKey.subKey.paramA} : "DEF"

returns: {'paramA': 'DEF'}

NOT OK

[test-fullautomation]

Hi Son,

all scenarios shall be possible. Please check.

Background: implicit creation has always the risk of typos which would create a new structure. But this applies for all levels. Therefore we an also create the top-level implicitly. And: This reduces the lines of code of config-files.

Thank you, Thomas

[HolQue]

Addendum:

(6) This implicit creation works like expected (except the fact that the first line should not be necessary, but currently is):

"testdict" : {},
${testdict.subKey.subKey.paramA} : "DEF"

(7) After the data structure is available I want to continue with working with this structure. I want to add another sub key.

This is now the code:

"testdict"   : {},
${testdict.subKey.subKey.paramA} : "DEF",
"anotherKey" : "ABC",
${testdict}['subKey']['subKey']['paramA']['anotherKey'] : "GHI"

Result:

{'anotherKey': 'GHI', 'testdict': {'subKey': {'subKey': {'paramA': 'DEF'}}}}

The value assignments went wrong: The single parameter anotherKey has got a new value, but should not. The testdict is not extended by another level.

(8) I removed the single parameter "anotherKey". This is now the code:

"testdict"   : {},
${testdict.subKey.subKey.paramA} : "DEF",
${testdict}['subKey']['subKey']['paramA']['anotherKey'] : "GHI"

Result:

Error: 'Could not set variable 'testdict['subKey']['subKey']['paramA']['anotherKey']' with value 'GHI'! Reason: 'str' object does not support item assignment'!

This completely went wrong.

And also: The original expression should be put to the error message. But in lots of cases the curly brackets and the dollar operator are missing.

Got: Could not set variable 'testdict['... But expected: Could not set variable '${testdict}['...

(9) Going back to simple standards, without implicit creation is involved:

This works like expected:

"testdict"   : {},
${testdict}['subKey'] : "A"

With implicit creation at two levels; works also like expected:

"testdict"   : {},
${testdict}['subKey']['subKey'] : "A"

The same data structure again, but without implicit creation; instead of this in two separate steps now:

"testdict"   : {},
${testdict}['subKey'] : "A",
${testdict}['subKey']['subKey'] : "A"

Outcome:

Error: 'Could not set variable 'testdict['subKey']['subKey']' with value 'A'! Reason: 'str' object does not support item assignment'!

(same problem again)

(10) Trying the same in dotdict notation.

Very simple use case; works:

"testdict"   : {},
${testdict.subKey} : "A"

Two sub key level in one step; works also:

"testdict"   : {},
${testdict.subKey.subKey} : "A"

The same in two steps; error is back:

"testdict"   : {},
${testdict.subKey} : "A",
${testdict.subKey.subKey} : "A"

Outcome:

Error: 'Could not set variable 'testdict['subKey']['subKey']' with value 'A'! Reason: 'str' object does not support item assignment'!

The same error with this version:

"testdict"   : {},
${testdict.subKey} : "A",
${testdict.subKey}['subKey'] : "A"

This are really strange dependencies - not only inside a separate line, but also between different lines.

[namsonx]

Hello Holger, Hello Thomas,

I pushed new commit ce79f1d7efa to stabi branch to cover all scenarios Holger mentioned in this ticket.

Thank you, Son

[HolQue]

Hi Son,

instead of returning wrong values, test case (7) now causes: Error: 'Could not set variable '...' with value 'GHI'! Reason: 'str' object does not support item assignment'!

But that's wrong. Expected are correct values. Also all other test cases still cause the same wrong "Could not set variable ..." error message.

[HolQue]

Hi Son,

that's it:

Does not work:

${testdict.subKey_1} : "A",
${testdict.subKey_1.subKey_2} : "B"

Does also not work:

${testdict.subKey_1.subKey_2} : "A",
${testdict.subKey_1.subKey_2.subKey_3} : "B"

But this works:

${testdict.subKey_1} : "A",
${testdict.subKey_2.subKey_3} : "B"

Outcome: You need to compare the entire hierarchy of levels to decide if something has to be created implicitly or if something already existing has to be extended:

testdict.subKey_1
testdict.subKey_1.subKey_2

[namsonx]

Hi Son,

instead of returning wrong values, test case (7) now causes: Error: 'Could not set variable '...' with value 'GHI'! Reason: 'str' object does not support item assignment'!

But that's wrong. Expected are correct values. Also all other test cases still cause the same wrong "Could not set variable ..." error message.

Hello Holger,

Regarding to the use case (7)

${testdict.subKey.subKey.paramA} : "DEF",
"anotherKey" : "ABC",
${testdict}['subKey']['subKey']['paramA']['anotherKey'] : "GHI"

First, you set ${testdict.subKey.subKey.paramA} : "DEF" it means testdict['subKey']['subKey']['paramA'] = "DEF" and has a datatype is str. Then, you create the sub-element 'anotherKey' of testdict['subKey']['subKey']['paramA'] which currently is in str datatype with value DEF. How a Python program can create sub-element for str datatype variable? So a program have to raise error message here.

Thank you, Son

[HolQue]

Hi Son,

yes, you are right! I overlooked the data type str of the sub level. The string cannot be converted to dict and a string has no key. My test code is wrong.

I corrected:

${testdict.subKey.subKey.paramA} : {"A" : 2},
${testdict}['subKey']['subKey']['paramA']['anotherKey'] : "GHI"

And this works!

[namsonx]

Hi Son,

yes, you are right! I overlooked the data type str of the sub level. The string cannot be converted to dict and a string has no key. My test code is wrong.

I corrected:

${testdict.subKey.subKey.paramA} : {"A" : 2},
${testdict}['subKey']['subKey']['paramA']['anotherKey'] : "GHI"

And this works!

Hello Holger,

Please consider for some other doesn't work cases you mentioned above as well:

${testdict.subKey_1} : "A",
${testdict.subKey_1.subKey_2} : "B"

and

${testdict.subKey_1.subKey_2} : "A",
${testdict.subKey_1.subKey_2.subKey_3} : "B"

These test scenarios are similar with the use case I mentioned in previous comment. Please let me know if I'm missing any test scenarios based on this ticket.

Thank you, Son

[HolQue]

Usage of parameters are not allowed for implicit creation. But if it's not implicit - are parameters allowed in this case to define key names? I am not sure about the requirements.

This works:

"paramD" : "D",
"paramE" : "E",
${testdict_3.paramD.paramE.paramD} : {"E" : 5},
${testdict_3.paramD.paramE.paramD}[${paramE}] : {"F" : 6}

(paramE is accepted because testdict_3.paramD.paramE.paramD already exists.)

This does not work (implicit creation error):

${testdict_4}[${paramD}] : {"G" : 7}

This is plausible, because testdict_4 does not exist.

Therefore I assume that I have to create testdict_4 before.

This is the code:

"paramD" : "D",
"paramE" : "E",
${testdict_4} : {},
${testdict_4}[${paramD}] : {"G" : 7}

But this does not work (implicit creation error).

Is this an error or not? Or in other words: Shall it be allowed to add a key to an empty dictionary by parameter or not?

Addendum: This is again my mistake. The value of paramE is "E", and this level is already created by {"E" : 5}.

I assume now the following:

Never it is allowed/possible to create a dictionary key based on a parameter. And it doesn't matter if this is a key anywhere in the middle of the hierarchy of sub keys (and if parent levels do not exist, we would call it "implicit creation") or if it's a new key on bottom level of a hierarchy of sub keys that already exists!

(Up to now I have had only the first case in my mind, but overlooked the second case.)

I will adapt the self tests accordingly.

[HolQue]

Retest successful. Issue can be closed.

[namsonx]

Merged to develop branch

[test-fullautomation]

solved with version 0.10.0