Feature request: syntax highlighting in o.expr.codebox

I've been writing very long lambda functions and I'm finding that I spend an inordinate amount of time tracking down basic errors in o.expr.codebox. I know this comes up all of the time, but can we please implement some form of basic text/syntax highlighting in the codebox? Even color coding parentheses and brackets would be a huge step forward. Extra points if we can build in some key commands for automatic formatting.

hey jeremy, totally agreed -- it's hypothetically possible do add syntax highlighting for o.expr.codebox using the MaxSDK, and of course things like the gen.codebox have that, ... but it might be easier and more flexible to make a syntax highlighting scheme for an external editor like atom, sublime, vs-code, textwrangler, etc. -- probably something like a javascript scheme could be edited to allow the leading / before variable names, and add an error for trailing commas.

for parenthesis checking, you can already paste o.expr.codebox text into a code text editor and it should help you find parenthesis pairs at least. also nice for paste/replace, etc.

I've been thinking of making an o.expr object that can read external files, seems pretty useful especially for composing in an external text editor,... not sure when that will be exactly, but I think "not too hard" to make.

Longstanding issue and talked about since ODOT's first builds. It's trivial and the code already exists in a sense (otherwise expr codebox wouldn't be able to parse text input and do something meaningful with it). The larger issue is twofold: o.display and o.compose also draw bundle text to the screen, but we wouldn't want those to be ~drawing~ parsing all color combinations, as this would introduce CPU tax in rapid updates. Consequentially, it feels kludgy to only have syntax colorization in expr codebox.

Because of this, Rama's suggestion or something similar would be a good alternative, basically having highlighting relegated to a scheme elsewhere. My thought on that back in the day was to have an "open in..." message that would open in whatever editor had been designated... Then there could be an autowatch similar to what you see in the [js] object's implementation.

From a workflow standpoint, it would be vastly preferable if the thing were fully self-contained. Even if the highlighting turns off when the codebox isn't being edited, it would make the thing much easier to troubleshoot, and I think ultimately, more teachable. If, in the meantime, we could provide hooks into/out of an external editor, that would be progress.

I suspect color is “free" in modern computers and OS’s.

On May 9, 2019, at 11:14, Jeffrey Lubow notifications@github.com wrote:

Longstanding issue and talked about since ODOT's first builds. It's trivial and the code already exists in a sense (otherwise expr codebox wouldn't be able to parse text input and do something meaningful with it). The larger issue is twofold: o.display and o.compose also draw bundle text to the screen, but we wouldn't want those to be drawing all color combinations, as this would introduce CPU tax in rapid updates. Consequentially, it feels kludgy to only have syntax colorization in expr codebox.

Because of this, Rama's suggestion or something similar would be a good alternative, basically having highlighting relegated to a scheme elsewhere. My thought on that back in the day was to have an "open in..." message that would open in whatever editor had been designated... Then there could be an autowatch similar to what you see in the [js] object's implementation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

It’s most certainly as free as not having it if you’re drawing in Max in the first place...

Since you probably don’t want to use the tokenizer we have for the language itself, the syntax highlighting rules can absolutely be compiled away from display and compose (those are different binaries anyway in the way we’ve been building odot). But even if they weren’t, even with the most naive, call-it-every-time branch to choose not to run color-code will not have any performance implications on event-thread code. (Neither would coloring it, either, unless you did it in some maliciously inappropriate ways.)

I took a stab at the odot highlighting in JUCE but found that the most important aspect was actually parentheses-matching (which I am willing to bet the code tokenizer doesn’t keep track of in the way most useful to a syntax highlighter). If you can activate that, that would be a feature that can fix 80-90% of the issues.

Writing odot code in Sublime Text for non-trivial things also worked for me, and was a faster fix than writing a syntax colorizer.

Best,

Ilya

On May 9, 2019, at 5:59 PM, Adrian Freed notifications@github.com wrote:

I suspect color is “free" in modern computers and OS’s.

On May 9, 2019, at 11:14, Jeffrey Lubow notifications@github.com wrote:

Longstanding issue and talked about since ODOT's first builds. It's trivial and the code already exists in a sense (otherwise expr codebox wouldn't be able to parse text input and do something meaningful with it). The larger issue is twofold: o.display and o.compose also draw bundle text to the screen, but we wouldn't want those to be drawing all color combinations, as this would introduce CPU tax in rapid updates. Consequentially, it feels kludgy to only have syntax colorization in expr codebox.

Because of this, Rama's suggestion or something similar would be a good alternative, basically having highlighting relegated to a scheme elsewhere. My thought on that back in the day was to have an "open in..." message that would open in whatever editor had been designated... Then there could be an autowatch similar to what you see in the [js] object's implementation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

I am aware that using an external text editor is easier than coding a new feature. That is why this is a feature request. ;-) But seriously, it would be nice if we put a little shine on odot with thoughtful features that encourage adoption and usability. If I open a text editor, I may as well be coding in js or java or c. Give me a reason to stay in odot.

On Thu, May 9, 2019, 9:40 PM ilya notifications@github.com wrote:

It’s most certainly as free as not having it if you’re drawing in Max in the first place...

Since you probably don’t want to use the tokenizer we have for the language itself, the syntax highlighting rules can absolutely be compiled away from display and compose (those are different binaries anyway in the way we’ve been building odot). But even if they weren’t, even with the most naive, call-it-every-time branch to choose not to run color-code will not have any performance implications on event-thread code. (Neither would coloring it, either, unless you did it in some maliciously inappropriate ways.)

I took a stab at the odot highlighting in JUCE but found that the most important aspect was actually parentheses-matching (which I am willing to bet the code tokenizer doesn’t keep track of in the way most useful to a syntax highlighter). If you can activate that, that would be a feature that can fix 80-90% of the issues.

Writing odot code in Sublime Text for non-trivial things also worked for me, and was a faster fix than writing a syntax colorizer.

Best,

Ilya

On May 9, 2019, at 5:59 PM, Adrian Freed notifications@github.com wrote:

I suspect color is “free" in modern computers and OS’s.

On May 9, 2019, at 11:14, Jeffrey Lubow notifications@github.com wrote:

Longstanding issue and talked about since ODOT's first builds. It's trivial and the code already exists in a sense (otherwise expr codebox wouldn't be able to parse text input and do something meaningful with it). The larger issue is twofold: o.display and o.compose also draw bundle text to the screen, but we wouldn't want those to be drawing all color combinations, as this would introduce CPU tax in rapid updates. Consequentially, it feels kludgy to only have syntax colorization in expr codebox.

Because of this, Rama's suggestion or something similar would be a good alternative, basically having highlighting relegated to a scheme elsewhere. My thought on that back in the day was to have an "open in..." message that would open in whatever editor had been designated... Then there could be an autowatch similar to what you see in the [js] object's implementation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CNMAT/CNMAT-odot/issues/380#issuecomment-491153195, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGWSI7VSGCDT33GJC7YMBTPUT4CRANCNFSM4HL4LZCA .

Forgive me if this came across as patronizing: was not my intent... I also failed to communicate that turning on the parentheses matching in the Max codebox may be a lot easier than writing your own was what I was thinking of.

I have no cycles to currently spare, sadly, but I did find the old tokenizing code specifically for syntax highlighting. I have no idea what it would take to make it usable in the Max GUI object, but can link whoever is interested. If you have access to my old didl repo, it’s in case_studies/EditorSketching/Source/EditorComponents—odot Tokenizer stuff is what you’re looking for.

It’s C++ and there’s at least one attempt to slightly optimize the keyword recognizer by separating keywords via tables of words of the same size and it’s all written with JUCE types so that’s also a bit meh... But JUCE did the scope / parens highlighting for free.

FWIW, today I would do it similarly: find what the max codebox does and get it to do more for me first. Then add the keywords if you really need them, or numbers if that’s the issue.

I wouldn’t do this from scratch, though...

On May 9, 2019, at 10:51 PM, Jeremy Wagner notifications@github.com wrote:

I am aware that using an external text editor is easier than coding a new feature. That is why this is a feature request. ;-) But seriously, it would be nice if we put a little shine on odot with thoughtful features that encourage adoption and usability. If I open a text editor, I may as well be coding in js or java or c. Give me a reason to stay in odot.

On Thu, May 9, 2019, 9:40 PM ilya notifications@github.com wrote:

It’s most certainly as free as not having it if you’re drawing in Max in the first place...

Since you probably don’t want to use the tokenizer we have for the language itself, the syntax highlighting rules can absolutely be compiled away from display and compose (those are different binaries anyway in the way we’ve been building odot). But even if they weren’t, even with the most naive, call-it-every-time branch to choose not to run color-code will not have any performance implications on event-thread code. (Neither would coloring it, either, unless you did it in some maliciously inappropriate ways.)

I took a stab at the odot highlighting in JUCE but found that the most important aspect was actually parentheses-matching (which I am willing to bet the code tokenizer doesn’t keep track of in the way most useful to a syntax highlighter). If you can activate that, that would be a feature that can fix 80-90% of the issues.

Writing odot code in Sublime Text for non-trivial things also worked for me, and was a faster fix than writing a syntax colorizer.

Best,

Ilya

On May 9, 2019, at 5:59 PM, Adrian Freed notifications@github.com wrote:

I suspect color is “free" in modern computers and OS’s.

On May 9, 2019, at 11:14, Jeffrey Lubow notifications@github.com wrote:

Longstanding issue and talked about since ODOT's first builds. It's trivial and the code already exists in a sense (otherwise expr codebox wouldn't be able to parse text input and do something meaningful with it). The larger issue is twofold: o.display and o.compose also draw bundle text to the screen, but we wouldn't want those to be drawing all color combinations, as this would introduce CPU tax in rapid updates. Consequentially, it feels kludgy to only have syntax colorization in expr codebox.

Because of this, Rama's suggestion or something similar would be a good alternative, basically having highlighting relegated to a scheme elsewhere. My thought on that back in the day was to have an "open in..." message that would open in whatever editor had been designated... Then there could be an autowatch similar to what you see in the [js] object's implementation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CNMAT/CNMAT-odot/issues/380#issuecomment-491153195, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGWSI7VSGCDT33GJC7YMBTPUT4CRANCNFSM4HL4LZCA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Iirc, at the time that the codebox was written, the Max SDK didn’t include the gen codebox—not sure if it does now or not, but it could be worth reaching out to someone at Cycling to see if they’d be willing to share some code. As Ilya says, I wouldn’t do this from scratch, and I certainly wouldn’t do it in the box that o.expr.codebox currently uses if that’s even possible.

John On May 10, 2019, 3:53 AM -0400, ilya notifications@github.com, wrote:

Forgive me if this came across as patronizing: was not my intent... I also failed to communicate that turning on the parentheses matching in the Max codebox may be a lot easier than writing your own was what I was thinking of.

I have no cycles to currently spare, sadly, but I did find the old tokenizing code specifically for syntax highlighting. I have no idea what it would take to make it usable in the Max GUI object, but can link whoever is interested. If you have access to my old didl repo, it’s in case_studies/EditorSketching/Source/EditorComponents—odot Tokenizer stuff is what you’re looking for.

It’s C++ and there’s at least one attempt to slightly optimize the keyword recognizer by separating keywords via tables of words of the same size and it’s all written with JUCE types so that’s also a bit meh... But JUCE did the scope / parens highlighting for free.

FWIW, today I would do it similarly: find what the max codebox does and get it to do more for me first. Then add the keywords if you really need them, or numbers if that’s the issue.

I wouldn’t do this from scratch, though...

On May 9, 2019, at 10:51 PM, Jeremy Wagner notifications@github.com wrote:

I am aware that using an external text editor is easier than coding a new feature. That is why this is a feature request. ;-) But seriously, it would be nice if we put a little shine on odot with thoughtful features that encourage adoption and usability. If I open a text editor, I may as well be coding in js or java or c. Give me a reason to stay in odot.

On Thu, May 9, 2019, 9:40 PM ilya notifications@github.com wrote:

It’s most certainly as free as not having it if you’re drawing in Max in the first place...

Since you probably don’t want to use the tokenizer we have for the language itself, the syntax highlighting rules can absolutely be compiled away from display and compose (those are different binaries anyway in the way we’ve been building odot). But even if they weren’t, even with the most naive, call-it-every-time branch to choose not to run color-code will not have any performance implications on event-thread code. (Neither would coloring it, either, unless you did it in some maliciously inappropriate ways.)

I took a stab at the odot highlighting in JUCE but found that the most important aspect was actually parentheses-matching (which I am willing to bet the code tokenizer doesn’t keep track of in the way most useful to a syntax highlighter). If you can activate that, that would be a feature that can fix 80-90% of the issues.

Writing odot code in Sublime Text for non-trivial things also worked for me, and was a faster fix than writing a syntax colorizer.

Best,

Ilya

On May 9, 2019, at 5:59 PM, Adrian Freed notifications@github.com wrote:

I suspect color is “free" in modern computers and OS’s.

On May 9, 2019, at 11:14, Jeffrey Lubow notifications@github.com wrote:

Longstanding issue and talked about since ODOT's first builds. It's trivial and the code already exists in a sense (otherwise expr codebox wouldn't be able to parse text input and do something meaningful with it). The larger issue is twofold: o.display and o.compose also draw bundle text to the screen, but we wouldn't want those to be drawing all color combinations, as this would introduce CPU tax in rapid updates. Consequentially, it feels kludgy to only have syntax colorization in expr codebox.

Because of this, Rama's suggestion or something similar would be a good alternative, basically having highlighting relegated to a scheme elsewhere. My thought on that back in the day was to have an "open in..." message that would open in whatever editor had been designated... Then there could be an autowatch similar to what you see in the [js] object's implementation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CNMAT/CNMAT-odot/issues/380#issuecomment-491153195, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGWSI7VSGCDT33GJC7YMBTPUT4CRANCNFSM4HL4LZCA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

would love to see this one too some day, perhaps similarly to how the js and node.js objects work (i.e. editor pops up on double-click)

Looks like this is probably the thing to try in order to do something similar do the js object: https://docs.juce.com/master/classCodeEditorComponent.html

CNMAT / CNMAT-odot

Feature request: syntax highlighting in o.expr.codebox #380