leanovate / gopter

GOlang Property TestER
MIT License
598 stars 40 forks source link

Decoding complex inputs upon test failure #58

Open msorens opened 5 years ago

msorens commented 5 years ago

Trying to get a handle on how WithLabel could help decoding inputs in the event of test failure. For the simple example shown on https://godoc.org/github.com/leanovate/gopter with two simple inputs, it seems fine. But here is what I have after I have put WithLabel on most everything in my test generators:

Project_Name, Project_Id, Rules: {{ﶄ⺗ມ𑌃ଃ b- {} [] 0} [{5p7 -
   𑊋﷼ⴐ🃭ἲ 1 [0xc0002e1a40 0xc0002e4c30 0xc0002e5e00 0xc000558ff0
   0xc00055c1e0] {} [] 515} {xm x ዓ𖭖🄉࠾᳆ 1 [0xc0005aba90
   0xc0005aec80 0xc0005afe50 0xc0005b7040 0xc0005fe8c0] {} [] 0} {h5 w
   ే𐬹𝕐𐄡Ӂ 0 [0xc000575130 0xc00053a690 0xc00053b900 0xc000554dc0
   0xc000538410] {} [] 0} {-y -y ৗए𑃰𬐐࿑ 0 [0xc000525450
   0xc000520f50 0xc00051ea00 0xc00051c460 0xc00051df40] {} [] 0} {2-9v a
   ූᩥᬥආⵣ 1 [0xc00050f8b0 0xc00050aff0 0xc000506730 0xc000507f40
   0xc000505720] {} [] 0}]}

First, it seems to have shown the labels only for the topmost items. Second, it put all the labels first, not intermixed with their values. Here is how those labels match the inputs: image Just to give some feel for how my generators are organized, here it is in pseudo-code:

createProjectAndRuleGen => 
    CombineGens(projectReqGen, gen.SliceOf(ruleReqGen)

projectReqGen => (id, name)
ruleReqGen => CombineGens(gen.Bool(), ruleReqGenEvent, ruleReqGenNode)

ruleReqGenEvent => (id, name, conditions1, and some other properties...)
ruleReqGenNode => (id, name, conditions2, and some other properties...)

conditions1 => gen.SliceOf(conditionsGenEvent)
conditions2 => gen.SliceOf(conditionsGenNode)

conditionsGenEvent => (type, value)
conditionsGenNode => (type, value)

Looking for tips on how to make the input blob be less indecipherable.

untoldwind commented 5 years ago

The "problem" here is the CombineGens. Unluckily go does not have something like tuples, therefor CombineGens is a somewhat feeble attempt to emulate those.

Simply put: When you have a generator with label "Im A" and a generator with label "Im B" then CombineGens will create a generate with label "Im A, Im B", which leads to the output shown.

To mitigate this the best idea I can up with is probable to write a specific parameter struct for you testcase (containing the three values "Project_Name", "Project_Id" and "Rules") an combine the generators with gen.Struct or gen.StructPtr. IThe you can implement a String() method for that parameter struct to beautify the output in case of an error.