cucapra / calyx-evaluation

4 stars 3 forks source link

Update benchmark numeric type format. #6

Closed cgyurgyik closed 3 years ago

rachitnigam commented 3 years ago

Since this repo doesn't have any CI, did you run runt to check if the results work? If so, merge it.

cgyurgyik commented 3 years ago

Since this repo doesn't have any CI, did you run runt to check if the results work? If so, merge it.

Running say,

runt -i polybench/linear-algebra-2mm.fuse -d

I'm getting different outputs:

...
   2│+   "A0_0": [
        3│+     10,
        4│+     98,
        5│+     38,
        6│+     18,
        7│+     34,
        8│+     97,
        9│+     61,
       10│+     71,
       11│+     42,
       12│+     99,
       13│+     90,
       14│+     76,
       15│+     93,
       16│+     89,
       17│+     64,
       18│+     4,
       19│+     17,
       20│+     79,
       21│+     0,
       22│+     24,
       23│+     78,
       24│+     47,
       25│+     2,
       26│+     98,
       27│+     49,
       28│+     18,
       29│+     76,
       30│+     82,
       31│+     12,
       32│+     92,
       33│+     80,
       34│+     36,
       35│+     37,
       36│+     34,
       37│+     60,
       38│+     9,
       39│+     73,
       40│+     12,
       41│+     87,
       42│+     7,
       43│+     69,
       44│+     46,
       45│+     80,
       46│+     77,
       47│+     69,
       48│+     50,
       49│+     82,
       50│+     15,
       51│+     89,
       52│+     2,
       53│+     52,
       54│+     13,
       55│+     11,
       56│+     4,
       57│+     65,
       58│+     49,
       59│+     14,
       60│+     80,
       61│+     81,
       62│+     61,
       63│+     32,
       64│+     81,
       65│+     32,
       66│+     85
       67│+   ],
    2  68│    "A_int0_0": [
    3  69│      [
         ~
   82 148│      ]
   83 149│    ],
      150│+   "B0_0": [
      151│+     39,
      152│+     50,
      153│+     44,
      154│+     33,
      155│+     9,
      156│+     13,
      157│+     49,
      158│+     54,
      159│+     77,
      160│+     21,
      161│+     35,
      162│+     70,
      163│+     24,
      164│+     99,
      165│+     51,
      166│+     65,
      167│+     69,
      168│+     91,
      169│+     33,
      170│+     61,
      171│+     65,
      172│+     75,
      173│+     0,
      174│+     36,
      175│+     67,
      176│+     14,
      177│+     92,
      178│+     92,
      179│+     47,
      180│+     6,
      181│+     0,
      182│+     63,
      183│+     60,
      184│+     73,
      185│+     86,
      186│+     23,
      187│+     12,
      188│+     45,
      189│+     47,
      190│+     76,
      191│+     63,
      192│+     14,
      193│+     51,
      194│+     60,
      195│+     74,
      196│+     16,
      197│+     15,
      198│+     87,
      199│+     89,
      200│+     73,
      201│+     32,
      202│+     63,
      203│+     46,
      204│+     52,
      205│+     96,
      206│+     45,
      207│+     67,
      208│+     42,
      209│+     83,
      210│+     8,
      211│+     40,
      212│+     43,
      213│+     81,
      214│+     73
      215│+   ],
...

So it looks like A0_0 is included in mine but not in the benchmark's expected?

rachitnigam commented 3 years ago

Odd, maybe worth checking what happens on an older commit of the Calyx compiler.

rachitnigam commented 3 years ago

Hey @cgyurgyik, this needs to be merged sooner rather than later since other PRs are blocked on it (specifically, the unify std_div and std_mod PR needs to run these tests)

cgyurgyik commented 3 years ago

Hey @cgyurgyik, this needs to be merged sooner rather than later since other PRs are blocked on it (specifically, the unify std_div and std_mod PR needs to run these tests)

With exams I won't have time to touch this until likely the weekend. Given that this is unsigned bitnum, I'm pretty confident that all my PR changes is the JSON format. It's hard to say whether the .expect file diffs is from my PR or an earlier one without integrated tests, and I haven't been able to follow your recommendation above yet.

rachitnigam commented 3 years ago

All green with https://github.com/cucapra/calyx/pull/430/commits/513287397dc6c84baa4a5073150190609b714499.