Codegen issues found via a testsuite

[chuggafan]

I found this nice C compiler testsuite https://github.com/ishiura-compiler/CF3 for arthemetic optimization and I've discovered at least 1 bug so far in OCC with it, specifically in the EXP_1 folder file test110.c function f61 terminates unexpectedly, when attempting to reproduce this with a smaller subcase I was unable to do so, however.

I also made a nice script to go with the testing repo I gave to measure the time it takes to compile each file to allow you to see problem spots in the code:

$dirs = dir -Directory .\testsuite
$sw = New-Object Diagnostics.Stopwatch
foreach($folder in $dirs)
{
    $files = Get-Childitem -Path .\testsuite\$folder -Filter '*.c'
    echo "Onto directory $folder"
    foreach($file in $files)
    {
        $exec = $file -replace ".c", ".exe"
        $sw.Restart()
        occ /! .\testsuite\$folder\$file /otestsuite\$folder\$exec
        $sw.Stop()
        $elapsed = $sw.ElapsedMilliseconds
        echo "Elapsed milis for file $file : $elapsed"
        Invoke-Expression .\testsuite\$folder\$exec
        if($LASTEXITCODE -ne 0)
        {
            echo "File $file failed from compilation"
        }
    }
    del *.o
}

I ran this testsuite by using the powershell command inside of git clone of the testsuite repository at the top level directory of OrangeC. There's several files that also take over a second to compile, and several more that take more than two seconds.

(edit): also EXP_1/test1182.c took THIRTEEN seconds to compile for me. However, this might be biased due to me having an anti-virus running atm as well, so I reached 100% disk utilization. shrug

[LADSoft]

ok we can look at this later... some programs still take a while to compile, the one out of the project is sqlite3.c... 8 seconds or so I think?

[GitMensch]

@chuggafan as there were different adjustments in the arithmetic parts since this issue was created - can you please recheck if the result is still the same?

[chuggafan]

test1.c in EXP_1 gives me a diag that says: "Diagnostic: Error compiling assembly instruction "add eax,byte170017912": Unknown operand:_main" so... it's gotten worse in some ways, I never achieved full knowledge of the times. And going very far with these tests can take hours because I have an A/V and I don't have some super-beefy Xeon computer + SSD set where OCC + the testsuite live on... so testing it completely is... unfeasable for me at this time.

[GitMensch]

@LADSoft I suggest to create a new issue for the assembly error raised by test1.c in EXP_1 and add this issue to the current milestone.

[GitMensch]

@chuggafan Do you mind trying to run the tests again?

[chuggafan]

I thought this was actually running in the CI at this point? I'm not even sure anymore... I probably will on friday tbh.

[GitMensch]

@chuggafan any news from that Friday? ;-)

[chuggafan]

None, I basically forgot about this for the most part, IIRC not much went on with this anyways and I was right that a large chunk of this is running in CI though. I've recently been completely metaphorically hit by a train though, which has slowed down everything for my work.

Running on the most up-to-date branch gives us bad things though:

Error: Duplicate public t13 in module .\testsuite\EXP_1\test1.c
    1 Errors, 0 Warnings
    Errors encountered, not creating output file

This type of error is being incredibly common for some unknown reason, I'll investigate why at a later point (if I can). This is currently the only type of failure case I'm seeing, though it will be quite some time before all 5k+ tests are done

[GitMensch]

Depending on how long this runs, it may should be included in the CI tests (either the default actions or the just suggested actions which would be "manually triggered shortly before release"). In any case it seems useful to recheck the current state, soon.

[chuggafan]

https://github.com/LADSoft/OrangeC/blob/master/tests/c-testsuite/makefile

A bunch of the problem testcases are actually already included, it seems, I keep forgetting about this through life and other work on this....