search

projectfluent / fluent-rs

Rust implementation of Project Fluent
https://projectfluent.org
Apache License 2.0
1.08k stars 97 forks source link

Unable to parse multiline messages formatted with CRLF with breaks that don't match the indentation of the rest of the message #213

Closed kellpossible closed 3 years ago

kellpossible commented 3 years ago

When I have a message like:

multiline-message = 
    This is a multiline message 1.

    This is a multiline message 2.

Fluent successfully parses when the file is formatted with LF, but has a parse error when the file is formatted with CRLF.

However if the gap between the lines contains indentation matching the other lines, then it successfully parses with both LF and CRLF.

multiline-message = 
    This is a multiline message 1.

    This is a multiline message 2.

Editors often automatically remove this indentation, or don't place it in the first place.

kellpossible commented 3 years ago

I have created a minimal reproduction of the issue in the multiline-break-indentation-bug branch of this repo

Pike commented 3 years ago

Fly-by note, the ´crlf´ test is disabled, https://github.com/projectfluent/fluent-rs/blob/cd8d6c4e7a20ff69762ff86c7a169df790d50ddd/fluent-syntax/tests/parser_fixtures.rs#L21-L23

Did you test your example on other implementations by any chance?

Pike commented 3 years ago

Also, can you set .gitattributes on your example repo to ensure that we all look at the same line endings?

kellpossible commented 3 years ago

Also, can you set .gitattributes on your example repo to ensure that we all look at the same line endings?

I'm not familiar with .gitattributes but I gave it a try, please let me know if you think it's correct

Pike commented 3 years ago

Yes, thanks. Sadly not through the gh UI, but when cloning it does.

Also tested the python ipml just now, that parses OK

zbraniecki commented 3 years ago

This should be now fixed by https://github.com/projectfluent/fluent-rs/commit/0195099c8a71dcfa2acd05578e94ed6d1203549a - can someone verify?

zbraniecki commented 3 years ago

Closing. Reopen if needed.