Open WardBrian opened 4 months ago
Attention: Patch coverage is 73.75000%
with 21 lines
in your changes are missing coverage. Please review.
Project coverage is 89.76%. Comparing base (
8bc6ba0
) to head (49381b7
). Report is 3 commits behind head on master.:exclamation: Current head 49381b7 differs from pull request most recent head 450cad4. Consider uploading reports for the commit 450cad4 to get more accurate results
Isn't there sub slicing when we peel off _lpdf
suffixes in sampling statements?
Isn't there sub slicing when we peel off
_lpdf
suffixes in sampling statements?
Y ~ foo(...)
) to the thing with the suffix (target += foo_lpdf(Y | ...)
), so even there it is a concatenation problem more than a subslicing.θ_lpdf
, the cut point in the string of bytes would still just be 5 bytes from the endBut, that is definitely one area of this that would need much more testing before it could be merged.
I know for a fact that this requires a few changes in
stan-dev/stan
's json data handler to recognize unicode names, which is just one of several reasons this is a draft.The basic overview: OCaml strings should be treated mostly like arrays of bytes, and
ocamllex
handles inputs as sets of bytes. We can define rules that recognize UTF-8-compatible bytes, and then do validation on them after the fact based on the the Unicode Annex 31: Unicode Identifiers standard.We then pretend for most of the compiler like it's just bytes, which is fine, because we never do things like subslice variable names.
Finally, at output time, we already had string escaping (since #952), so most of the code-gen works fine. Recent C++ standards require that compilers support UTF-8 names based on the same UAX31 rules linked above, but older ones may not. For now I've got it generating "Universal character names" which seem like the legacy version of this, which hopefully means older compilers will be happy with it.
Submission Checklist
Release notes
stanc3 can now accept a flag
--allow-unicode
which enables the use of non-ascii characters in Stan files. All files are expected to be encoded in UTF-8. This is experimental and may not work with older C++ compilers.Copyright and Licensing
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the BSD 3-clause license (https://opensource.org/licenses/BSD-3-Clause)