Code editor 2: Syntax highlighting errors on load that change every time

KormexGit commented 6 months ago

Description

I have not had any luck reproducing this consistently in a smaller project, so I'm sending my whole main project. Almost every time I load up the project, there will be some sort of issue with the coloring of the code. What exactly the problem is and where it is will change every time. Making any edit on the line that's displaying incorrectly will often fix it, and sometimes fix other code below, but also sometimes cause code below to break in another different way. A common issue is for random parts of the code to remain grey

Or for a bunch of the code to be yellow, starting after a string that IS closed off properly, and continuing for a while until it hits another string that is closed off properly (but not necessarily the first string it hits)

Sometimes, a bunch of the code will be 100% yellow on load, but then fix itself after waiting a bit. For me at least, it takes several seconds for all of the highlighting to load in when opening the project for the first time, so I suspect some of the issues are related to that.

PS: Feel free to also use this project for testing #5371, because I have another tooltip stuck on top of this bug report window as I'm typing this lol

Steps To Reproduce

Open up oPlayer, and scroll around. There will likely be a variety of coloring issues.

You can also try reloading the project without restarting GM to trigger more/different issues.
Starting the project for the first time after opening GM will more frequently have the grey code issue
Making any sort of edit to the code can also shuffle the problems, fixing some of them but also potentially creating more elsewhere. The edit can be as little as adding or deleting a space.

Which version of GameMaker are you reporting this issue for?

IDE v2024.400.0.532 Runtime v2024.400.0.551

Which operating system(s) are you seeing the problem on?

Windows 10.0.22631.0

edd46ea1-1465-4a29-a740-b8f76cfa09a1

zreedy commented 4 months ago

A bunch of issues with how color coding was resolved have been fixed which should resolve this issue.

mgeddesGM commented 4 months ago

first time grey issue was still present in IDE v2024.600.0.561 Runtime v2024.600.0.579

opened oPlayer and sections were same colour as expected but this colour was mostly grey so no mixed colouring restarted ide and project, colours showed but some sections were still grey

KormexGit commented 4 months ago

Looking at it a bit myself, seems every global struct item is grey, which is probably a side effect of #5963 There's also some issues with things being grey when dot accessing an instance inside a with statement, which may be a specific issue rather than random color failure. Will see if I can reproduce it in a separate project.

Edit: I was able to consistently reproduce the issues in both of the screenshots immediately below, see #6029 and #6030

The fully random color errors are mostly fixed, though I do still see a just a couple issues here and there (pls don't judge my variable naming I swapped from all lower to camelCase halfway through the project and haven't gotten around to changing all the old variables lol)

This bracket coloring one actually happened again after restarting:

This one went away on restart, also I think it's in new coded I added since the report:

After reloading the project, this happened, it was the correct color prior to reloading:

Also seems the feather underline is underlining the wrong spot? That's a separate issue though

Reloading the project a 2nd time caused more grey code to appear, so it seems that sometimes reloading can make it worse:

There's actually quite a bit of random grey bits of code after the 2nd reload, seems to mostly be affecting numbers, strings, and sometimes orange keywords like function, if, or, exit, etc.

I scrolled even more after and the yellowfication made a return:

Emc1923 commented 3 months ago

These can happen on CE1 too.

zreedy commented 3 months ago

There are essentially the follow classes of color coding issues:

Client-sided TextMate Theme -> Token generation
Server-sided Semantic Token generation
Client-sided tmToken + smToken merging issues
Client-sided race conditions mangling token streams

Previous bug fixes have addresses issues found in all 4 classes but more have been found while investigating this ticket.

Now fixed:

Server-sided: Reported smTokens from functions outside of the current Source File are reported back in the same stream, resulting in a mangled token stream

Currently observed but unknown origin:

tmToken generation seems to either be timed out or overridden prematurely when Tasks are Yielded

zreedy commented 3 months ago

Discovered a class 4 issue that seems to explain the remainder of the off color coding issues. Unfortunately I cannot find the actual root of the race condition, however throwing a lock at the TokenizeLineBuffer level does seem to reliably fix the issue. It would be very ideal to find the root of this race condition and implement a more granular lock to improve performance. However, performance does not seem to be negatively impacted, even on extremely large scripts like unity builds of GMLive.

zreedy commented 3 months ago

Nature of the race condition has been discovered. An appropriate fix has been implemented and documented.

zreedy commented 3 months ago

All issues outlined here are fixed excluding the Feather specific bug where if type info is incomplete (e.g. resolves to type Any) resulting in gray variables. This issue in particular will require a lot of work on the Feather type system to better introspect these variable types.

caitlinrmcintyre commented 2 months ago

Verified fix in IDE v2024.800.0.591 Runtime v2024.800.0.615, thanks.

YoYoGames / GameMaker-Bugs