Figure out error story for FFI

[Manishearth]

Right now FFI just returns bools for error cases.

We should eventually figure out how we want to expose error states over FFI and write error enums for these. The current design is easily extended to error enums.

The hard part here is mostly handling complex error states.

[sffc]

Discussion:

• @zbraniecki - Mozilla has tooling that returns line numbers from errors. But perhaps plain enums would be fine.
• @Manishearth - Plain enums would be the easiest path forward.
• @sffc - Could we use a thread-local error?
• @Manishearth/@shadaj - No, libc and Python does this and it's painful
• @sffc - So we would write a mapping from the complex error types to the C enums?
• @Manishearth - Yes; some of the mappings could be auto-generated with our helper macro.
• @zbraniecki - Could we return an array or vector of errors? This may be useful but I'm not sure.
• @sffc - I think being able to associate metadata with errors is still very useful. For example, with I/O errors, being able to know what file path caused the I/O error is important.
• @Manishearth - Memory allocation issues asside, for FFI, enums are still better than strings for reasons @zbraniecki mentioned.
• @sffc - Could the errors we return from ICU4X be a plain C enum paired with a string (the Display impl of the Rust error), as the standard type we always use?

Conclusion:

• For now, keep returning bool. @Manishearth or @shadaj to follow up with a design doc later in the summer once we have more of an idea around Diplomat and more cases to inspect.

[sffc]

Make a plan for what to do this quarter, and implement in Q1 2022.

[sffc]

Discussion 2022-05-27:

• @sffc - A few approaches:

1. Enums over diplomat corresponding to enums in Rust
2. Strings built from the Display impl
3. Strings in debug mode; bool or similar in release mode
4. Static strings
5. Opaque error type that has a to_string
6. Global flat error enum

• @Manishearth - I think (4) is a less good version of (1). If we do (4), we should return an opaque type that has a to_string function on it. With any approach that involves returning strings, we need to do some weird logic in diplomat::Result.
• @Manishearth - Regarding losing functionality over FFI. Rust has really good ways of handling errors and enums that are difficult to replicate in other languages.
• @robertbastian - DataError has no allocations, as do most of our errors. Why not just use them?
• @Manishearth - Over FFI, we have allocations because we need to make it opaque, because we can't make it a foreign type, since it is a dataful enum.
• @sffc - For option 5, in debug mode, we can make to_string return a string, and in release mode, it panics, or we rely on DCE to remove the function.
• @robertbastian - By hiding this in an opaque error, we preclude langauges that have better stories for error handling.
• @sffc - For Option 5, do we have a single opaque error type for all of ICU4X, or different ones for each function?
• @sffc - I think I prefer separate error types, which is better for DCE. I think I prefer Diplomat auto-generating this for us.
• @Manishearth - If we do it ourselves instead of Diplomat, we can have more control over things like cfg.
• @zbraniecki - I'd like to avoid two scenarios: if all errors are one error, or if all errors are just a string. Because there are a sufficient number of cases where you want to read the error and pivot based on it.
• @Manishearth - That would point to the error enums.
• @sffc - I think it's okay to rely on introspecting strings. It's something that all programming languages understand, and developers often do it anyway. JavaScript has relied on string matching for errors for a long time.
• @zbraniecki - I'm concerned about the idea of relying on string stability for errors. I would prefer not to lower that velocity. Just the ability to say "data payload error is this ID" is useful.
• @Manishearth - A common error approach in C libraries (editorial note: including ICU4C) is to have a single flat error enum. We could have Into impls into the big giant common error enum.
• @sffc - With a common error enum, it covers the introspecting error use case well, but it doesn't do a good job for debugging. If we do this, we should at least log the Display of the error based on the log feature.
• @Manishearth - A common enum is a low denominator.
• @zbraniecki - JS users sometimes write a regex against an error string to capture different versions of the error.
• @sffc - I'm okay with an enum approach with logging strings as a compromise.
• @robertbastian - I feel a massive enum is good because it is the product of the expressiveness we have in Rust. I would still prefer a general solution that changes based on programming languages, but I understand that is more complicated. We could still write wrappers in other languages that do something nicer.

Consensus: Single common error enum, with strings optionally logged using the Rust logger.

LGTM: @QnnOkabayashi @robertbastian @nordzilla @Manishearth @sffc @zbraniecki

[Manishearth]

So I was getting ready to start implementing this and I realized that in Diplomat, enums become strings in JS, converted after being received as integers. For the JS backend are we okay with that or would we want to return an integer instead?

Ideally Diplomat has support for consts so we can return an integer that you can check against a constant list.

[sffc]

So I was getting ready to start implementing this and I realized that in Diplomat, enums become strings in JS, converted after being received as integers. For the JS backend are we okay with that or would we want to return an integer instead?

Ideally Diplomat has support for consts so we can return an integer that you can check against a constant list.

IMO, I think it's fairly standard for errors to be strings in JS, and this would be my expected behavior. The main thing is that the strings are stable, which for enums they should be.

[zbraniecki]

Hah, that's exactly why I'd prefer not to use strings. Because if we make strings stable we corner ourselves away from ability to improve error messages. I'm ok with string identifiers or numerical error codes plus payload.

[Manishearth]

yeah these would be string identifiers like "ICU4X_PLURALS_OOPSIE" (and an enum variant of the same name in C++) which we would stabilize

[zbraniecki]

I'm happy with ICU4X_PLURALS_OOPSIE. More, I now demand that we add it.