php8.4beta4 JIT erronous results

tbali0524 commented 3 months ago

Description

The following code:

// see `src/*.php` in the bug-report zip package (link below)
// These cli programs are reading from stdin and writing to stdout.
// see `./run.bat` to run them all with proper redirections (on Windows)

Resulted in this output:

// see `output-with-8.4beta4-JIT/*.txt` in the bug-report zip package (link below)

But I expected this output instead:

// see `output-with-8.4beta4-NO-JIT/*.txt` in the bug-report zip package (link below)

Notes

All files for bug reproduction (src, input text files, expected and wrong output text files) are attached or available in https://www.aviationfanatic.com/tmp/php84beta4-jit-bug.zip
Code WORKS fine with
- php 8.3.11 with JIT enabled, on Windows 11,
- php 8.3.11 with JIT disabled, on Windows 11,
- php 8.4beta4 with JIT disabled, on Windows 11,
- php 7.3 (no JIT ofc) on Linux (on CodingGame hosted runners)
Code FAILS (produces wrong, different output, despite having deterministic algorithms) with
- 8.4beta4 with JIT enabled, on Windows 11
- settings (full php.ini attached in bug-report zip):
  - opcache.enable_cli = 1
  - opcache.jit_buffer_size = 128M
  - opcache.jit = tracing
- same problem appeared with 8.4alpha4 and 8.4beta3 (only with JIT enabled), but I did not report it that time.
Tested on Windows 11 only.
I am very sorry for not being able to provide helpful bug description, just the fact of output mismatch. I don't know the root cause, and don't even know if these are a single JIT bug or multiple unrelated issues.
These JIT bug(s) discovered by re-running my 800+ CodinGame (algorithmic puzzles) CLI solutions (90k+ loc) with 6k+ test cases. Only < ~1% of the test cases failed, these are provided in this bug report.

php84beta4-jit-bug.zip

PHP Version

PHP 8.4beta4

Operating System

Windows 11

iluuu1994 commented 3 months ago

@tbali0524 Thanks for the report. It would help a lot if you could:

Extract and show only the cases that actually fail.
Try to reduce them as much as possible. I.e. try to isolate the issue in a small reproducer.

We get a lot of bug reports, and figuring out how to run your code takes a lot of time.

tbali0524 commented 3 months ago

Sorry for not being able to provide good bug description, actually I was hesitating to report it at all, because of this. I already filtered the failed tests from my 6k test runs for 800 scripts, so the 25 test runs for 14 scripts provided in the zip are ALL failing. The challenge for me is that the failure is only with JIT enabled, so if I try to see where it goes wrong with xdebug and step debugger in vs code, the bug does not even come up. If I manage to reduce the problem(s) to simpler code excerpts and better error symptom (not just output is different), I will file another bug report. But it takes try-and-error and lots of batch running.

iluuu1994 commented 3 months ago

I see, thanks for the response!

The challenge for me is that the failure is only with JIT enabled, so if I try to see where it goes wrong with xdebug and step debugger in vs code, the bug does not even come up.

This is indeed a challenge. Usually, I remove code from the example until the issue no longer reproduces (in this case, until the output of JIT and no JIT are equal), then revert, and try to simplify it in other way (remove branches, reduce loop iterations, etc.).

I will file another bug report

This issue can stay open, you can then report it here. Maybe somebody else will have a look at the examples.

iluuu1994 commented 3 months ago

After looking at your examples again, I understand why you had a hard time to simplify them, the code is quite complex. I was able to reproduce the first one with the tracing JIT on Linux:

php-dev -d opcache.enable_cli=1 -d opcache.jit=tracing src/event_spring-challenge-2024-q04.php < input/event_spring-challenge-2024-q04_i05.txt
[33,6,6]

@dstogov Is this enough for you to have a look?

tbali0524 commented 3 months ago

Thanks. Yes these are algorithmic puzzles, so lots of iterations, sometimes recursion or graph traversal, CPU heavy calculations (so prime use case for JIT)

PS: In the meantime I also re-ran my 9 seasons of Advent-of-Code solutions (225 similar CLI puzzles), from these 4 puzzles fail with 8.4beta JIT (and work with 8.3 and 8.4 no jit). I don't submit them as they might have been caused by same issue as with CG puzzles..

dstogov commented 3 months ago

Thanks for the report. It's completely enough to reproduce and analyse the JIT bugs. I'll try to fix the problems soon.

dstogov commented 2 months ago

https://github.com/php/php-src/pull/15732 should fix all the related issues. I'm going to merge it on Monday.

@tbali0524 thank you for finding so many bugs :)

tbali0524 commented 2 months ago

Thanks! i will wait for beta5 release and recheck if the 4 additional AoC bugs I discovered disappear with it. If they turn out to be unrelated, I will open a new issue.

tbali0524 commented 2 months ago

@dstogov I reran my test suite with beta5 and most problems gone, but a few test cases remained, where JIT gives wrong output. I attach a new zip where only the still failing test case inputs, expected and actual output and script sources are included. Or shall I open a new issue as this one is already closed? php84beta5-jit-bug.zip

dstogov commented 2 months ago

Interesting. I can't reproduce failures with DEBUG build on Linux, and I see 3 of 6 failures with RELEASE build. I'll try to take a look. No need for a new issue.

dstogov commented 2 months ago

The problem was not DEBUG vs RELEASE build, but in different JIT related settings in DEBUG php.ini. Now with default settings I reproduced all test failures with DEBUG build.

dstogov commented 2 months ago

The remaining problems should be fixed via 98f07fcfca2afa0ff41a14f003b3950f2ed0de03 Thanks for the reports!

php / php-src