Improving Processing Error Messages

[WillRabalais04]

Processing Friendly Errors

We are building out friendly error message rewriting by extending the PreprocessIssueMessageSimplifier. To determine which errors to support, we worked with Raphaël and Sam to run a community survey.

Motivations

The results of the survey showed that although the error messages are usable, with an average clarity rating of 3.32 out of 5 there is room for improvement. The key elements that respondents identified as crucial for understanding error messages include the name and type of the error, specifics about what broke in the code, traceback to the specific line of the error, expected versus found outcomes, resources for contextualization and guidance, and possible solutions. Enhancing error messages based on the insights of the survey would greatly benefit the Processing user experience for beginners and advanced users alike. We hope to foster better understanding and more efficient problem-solving.

Currently Simplified Errors to Improve On

Strategy	Current Error Output	Acknowledged In Survey	URL
InvalidIdentifierStrategy	Bad identifier? Did you forget a variable or start an identifier with digits near ‘statement’	✅	Variables OR Variable Examples, Identifiers, Keywords & Statements
MismatchedInputSimplifierStrategy	Missing operator, semicolon, or ‘}’ near ‘statement’	✅	Operators, Operators Example, Semicolons, Curly Braces & Statements
InvalidAssignmentStrategy	Possible error on variable assignment near ‘statement’	✅	Variables OR Variable Examples, Assignment Operator & Statements
VariableDeclarationMissingTypeStrategy	Missing name or ; or type near ‘statement’	✅	Variables OR Variable Examples, Semicolons, Identifiers & Statements
MissingCurlyAtStartSimplifierStrategy	Missing left curly bracket “{”	✅	Curly Braces
MissingCurlyAtSemicolonSimplifierStrategy	Missing right curly bracket “}”	✅	Curly Braces
UnbalancedChevStrategy	Missing '<' OR Missing '>'	✅	Generics, Generic Methods & Generics & Subtyping
InvalidGenericDefinitionStrategy	Possibly missing type in generic near ‘statement’	❌	Generics, Generic Methods, Statements & Generics & Subtyping
MissingIdentifierSimplifierStrategy	Missing name or ; near ‘statement’	✅	Identifiers & Statements
KnownMissingSimplifierStrategy	Missing “statement”	❌	Statements
ExtraneousInputSimplifierStrategy	ArithmeticException statement or extra code near ‘statement’	❌	Arithmetic Exceptions, Operator Precedence & Statements
MissingClassNameStrategy	Missing name or ; near ‘statement’	❌	Classes , Identifiers, Semicolons & Statements
MethodMissingNameStrategy	Missing name or ; near ‘statement’	❌	Methods, Identifiers, Semicolons & Statements
ErrorOnParameterStrategy	Error on parameter or method declaration near ‘statement’	❌	Parameters, Methods & Statements
MissingSingleQuoteStrategy	Missing “'”	❌	Chars
MissingDoubleQuoteStrategy	Missing “"”	❌	Strings
UnbalancedCurlyStrategy	Missing "{" OR Missing "}"	✅	Curly Braces
UnbalancedParenStrategy	Missing "(" OR Missing ")"	✅	Parentheses
DefaultMessageSimplifier	Error Error on variable assignment near 'statement' Identifier cannot start with digits near 'statement' Error on parameter or method declaration near 'statement' Unexpected extra code near 'statement' Missing operator or semicolon near 'statement' Missing name near 'statement' Missing name or type near 'statement' Missing 'statement' Missing '}' Missing '{'	n/a	TBD

Errors to Add Strategies For

Priority	Error
First	ArrayIndexOutOfBoundException NullPointerException UnsupportedOperationException NumberFormatException StackOverflowError IndexOutOfBoundsException StringIndexOutOfBoundsException
Secondary	ClassCastException IllegalThreadStateException IllegalStateException ArrayStoreException NoSuchMethodError
TBD	FileNotFoundException IOException ConcurrentModificationException EnumConstantNotPresentException

Unsupported Errors

The following will still operate as usual without rewriting / error simplification.

• Variable is named after keyword
• GLSL/Shader errors
• Compatibility issues with Android extensions
• Library errors
• Python errors
• p5.js errors

Error Message Template

Optional?	Feature
❌	Processing found an error type error at [error line](link moves cursor to line number)!
❌	[error context]
✅	[reminder / additional context] OR We expected [expectation] but found [x]!
✅	We suggest [hint]!
✅	+ More Info on: -[additional resources]
❌	▶Click to See Original Error! [Toggles Original Error] *

* The toggle to see the original error will be added at some point but likely will not be in the first version of the improved error messages as it may be difficult to implement.

Example 1: Array Index Out of Bound Exception

🔵 Processing says: Oops! It looks like you're trying to look at an item in your 'numbers' array that doesn't exist. (On line 3 in sketch_230707c.pde)

Remember, counting in Processing starts from 0, not 1. This means your 'numbers' array, which has 3 items, has positions labelled from 0 to 2. If you're trying to find an item at position 3 and above (or below 0), it won't work because there's no item there!
To fix this, make sure you're looking at positions that exist in your 'numbers' array, that is, positions from 0 to 2.

For more:https://processing.org/examples/array.html

Example 2: Invalid Identifier

🔵 Processing says: Uh-oh! There's an issue with a "bad identifier" in your code. Specifically, the issue is with 'int 9'. (On line 48 in sketch230710a.pde)

In Processing, identifiers (like variable names, function names, etc.) can not start with a number and should only include letters, numbers, or underscores. Therefore, '9' is not a valid name.

Here's a quick fix: Try renaming your identifier so that it starts with a letter, dollar sign($) or an underscore(). Make sure the name you choose is not a reserved keyword like "setup" or "width".
For more:
https://processing.org/examples/variables.html
https://processing.org/examples/functions.html

Incorporating Additional Resources Approach

• When directing the users to external resources the goal is to direct them towards Processing resources when possible and towards official Java documentation as a last resort as it is significantly less user friendly than Processing resources.
• To not overwhelm the user with links to external resources we will link them inline with the error message if the keyword is conveniently in the message itself otherwise they will be in the "For more:" section at the bottom. The "For more:" section will also contain examples of the concepts when applicable.

[sampottinger]

This is great @WillRabalais04

[RichardDL99]

Syntax Error - Missing operator, semicolon, or ‘}’ near ‘load_primes’?

The error is that I have a function 'load_primes' but I have code not in a function, and I don't have setup (and draw) functions.

[sampottinger]

Hey @RichardDL99! I think you are referring to mixed modes where you are defining classes or functions but aren't using run or setup. We can take a look at that as well!

[sampottinger]

Dang sorry this one might have been my fault @WillRabalais04 - https://github.com/processing/processing4/issues/519. Looks like this slid off the radar.

[sampottinger]

Separately, it looks like we are doing both preprocessor issues and runtime issues here which is great! You'll want to extend PreprocessIssueMessageSimplifier for compile time errors and you'll want to do something similar to PreprocessIssueMessageSimplifier like a RuntimeIssueMessageSimplifier that intersects string messages going through EditorConsole

[sampottinger]

CC @SableRaf - the issue is cooking!

[SableRaf]

Hey @WillRabalais04,

Nice job on the "Invalid Identifier" error message! It's clear and helpful, but I think we can make it even friendlier, generic, and more direct.

Here's a tweaked version:

🔵 Processing says: Uh-oh! There's an issue with a "bad identifier" in your code. Specifically, the issue is with 'int 9'. (On line 48 in sketch_230710a.pde)

In Processing, identifiers (like variable names, function names, etc.) can not start with a number and should only include letters, numbers, or underscores. Therefore, '9' is not a valid name.

Here's a quick fix: Try renaming your identifier so that it starts with a letter or an underscore. Make sure the name you choose is not a reserved keyword like "setup" or "width".

For more:

• https://processing.org/examples/variables.html
• https://processing.org/examples/functions.html

I think it makes the message a bit more digestible.

I'm not entirely sure about linking to Oracle documentation pages as they can be overwhelming for beginners.

[WillRabalais04]

Thank you @SableRaf! I totally agree that Oracle documentation is not beginner friendly so it's not ideal to link to it.

In my approach thus far the only case in which there would be a link to Oracle documentation is when there is a keyword that a beginner may not know and that there is no Processing reference page for.

What would you suggest for this case instead? I can think of three alternatives to linking to Oracle documentation; including relevant information from the documentation in the error message itself, linking to Processing forum entries that pertain to the keyword or not attempting to define keywords.

These all have trade-offs to weight so a hybrid approach could be best.

• Including the information from the Oracle documentation within the error message itself comes at the risk of making it verbose. For example identifier, has a list of query language keywords that would be too long to include in the error message without making it extremely long.

• Linking to Processing forum articles would most likely be more user-friendly and comprehensible than Oracle documentation however there may not be an article for every issue and some discussions may include the information we want but also go off-topic which could be confusing for beginners.

• Not including any links to define keywords means beginners won't be confused by verbose documentation at the expense of not having any explanation for the keywords however it may be the best option.

However we do the keywords, I think the For More section at the bottom should stay the same as it is now and consist of examples from Processing reference.

Let me know what you think!

[sampottinger]

Hey @WillRabalais04! It occurred to me that I don’t think we are using an HTMLEditorKit for the console. I’d prefer to stay away from it but I’ll look at options. Until then, go ahead and assume you are writing out markdown formatted text. Thanks!

[SableRaf]

• Not including any links to define keywords means beginners won't be confused by verbose documentation at the expense of not having any explanation for the keywords however it may be the best option.

That would be my pick too. In my opinion, messages should be as self-contained as possible, introducing unfamiliar keywords as necessary within the message, and avoiding unnecessary jargon to keep things beginner-friendly. The primary focus should be to guide users towards a solution to their issue, with only a secondary aim of teaching programming terminology, if and when necessary.