microsoft / pict

Pairwise Independent Combinatorial Tool
www.pairwise.org
Other
1.27k stars 288 forks source link

Combinations lost of it #41

Open hahaha2019yz opened 4 years ago

hahaha2019yz commented 4 years ago

CONTENT OF PICT INPUT FILE(test.txt): A: a1 B: b1 C: c1,c2 D: d1,d2 E: e1,e2 F: f1,f2 G: g1,g2,g3,g4 H: h1,h2,h3,h4 I: i1,i2,i3,i4,i5

{A,B,C,E,D}@5 {F,G,H,I}@1

if [E] in {"e1"} then [G] in {"g1","g2"} else [G] in {"g3","g4"}; if [D] in {"d1"} then [I] in {"i1","i3"} else [I] in {"i2","i4","i5"}; if [E] in {"e1"} then [H]="h4" else [H]<>"h4";

AFTER GENERATION BY PICT:(command is: pict test.txt /o:1) A B C D E F G H I a1 b1 c1 d2 e2 f2 g3 h3 i4 a1 b1 c2 d1 e1 f1 g2 h4 i1 a1 b1 c2 d2 e2 f2 g4 h2 i2 a1 b1 c1 d1 e1 f1 g1 h4 i3 a1 b1 c2 d2 e2 f1 g4 h1 i5

parameters:ABCDE as@5 have 8 combinations and FGHI as@1 have 5 combinations. We predicted that the combinations of the two submodels have at least 8 results. But when we read and research the source codes of PICT we found that some random pick values of FGHI and constraints had confilcted with the 8 values of ABCDE combinations.

But in fact, we DO have some combinations to stay all of the 8 combination of ABCDE, such as: A B C D E F G H I 1 | a1 | b1 | c1 | d1 | e1 | f2 | g2 | h4 | i1 2 | a1 | b1 | c1 | d2 | e2 | f1 | g3 | h1 | i2 3 | a1 | b1 | c1 | d1 | e2 | f2 | g4 | h2 | i3 4 | a1 | b1 | c1 | d2 | e1 | f1 | g1 | h4 | i4 5 | a1 | b1 | c2 | d2 | e2 | f2 | g3 | h3 | i5 6 | a1 | b1 | c2 | d1 | e1 | f2 | g2 | h4 | i1 7 | a1 | b1 | c2 | d2 | e1 | f1 | g1 | h4 | i5 8 | a1 | b1 | c2 | d1 | e2 | f1 | g3 | h1 | i3

So in this situation, do we have some method to fix it still using PICT? Do we have other method to generate them? Or we must change the source code of PICT to generate them?

hahaha2019yz commented 4 years ago

@jaccz Hi, Jack, do you have any idea of this problem? Shall we exchange our options?

jaccz commented 4 years ago

The o:1 feature has a bug somewhere only recently discovered (https://github.com/microsoft/pict/issues/40). I haven't had time to root cause it yet. I predict your issue is a duplicate of that. One thing to check: if you use /o:2 instead of /o:1, will you see the same issue?

hahaha2019yz commented 4 years ago

The o:1 feature has a bug somewhere only recently discovered (#40). I haven't had time to root cause it yet. I predict your issue is a duplicate of that. One thing to check: if you use /o:2 instead of /o:1, will you see the same issue?

Hi, jaccz @jaccz , thanks for reply. I've saw the problem of #40, maybe it is not the same with mine. My problem is because the second combination is random, the PICT program gave the second submodel a set of random value, but this set of value was cutted off by the stricts(this maybe wouldn't happen by giving another random set of value). So if we could process this problem by taking all the parameters under a total root model(without generate submodels), and generate the test cases by using strength @1 of all the paramters, such as below:

E D C 0 0 0 0 0 0 0 0 E D B 0 0 0 0 E D A 0 0 0 0 E C B 0 0 0 0 E C A 0 0 0 0 E B A 0 0 D C B 0 0 0 0 D C A 0 0 0 0 D B A 0 0 C B A 0 0 I H 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I G 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I F 0 0 0 0 0 0 0 0 0 0 H G 255 255 0 0 255 255 0 0 255 255 0 0 0 0 255 255 H F 0 0 0 0 0 0 0 0 G F 0 0 0 0 0 0 0 0

As all of above values must at least appear once in the test cases, we suppose the problem I've mentioned above may be resolved.

hahaha2019yz commented 4 years ago

Hi, jaccz @jaccz , I also have another question about the PICT:

  1. What's the benifits of separate the problem of many submodels? The efficiency of algorithm, or to reduce the complex of the problem?
lakshmishub commented 4 years ago

Sub-models will help you to keep the number of cases low. you can only select the set of input variables that will directly impact the output , instead of all inout variables generating the combinations