It turns out we can make these work with very minimal complexity because the LF is still in the right place either way. This also lets us easily support mixtures of LF and CR+LF line endings gracefully. We create the line structures around the LF bytes and then have the byte-dispatch loop notice a CR followed by an LF and skip to the LF behavior.
Rather than add the remaining complexity around supporting bare CR and LF+CR sequences (both of which are quite rare now), this just adds diagnostics when we encounter a CR byte that won't fall out of our CR+LF handling. This is a better experience for users than the alternative. We still have a TODO to handle the full complexity of vertical whitespace, but I've updated it to reflect that the common case should be handled already.
This isn't complete though: we need to add support in string literal lexing, and we need to teach the diagnostic rendering to handle the error messages above better. But those will be future PRs, this is enough to unblock folks who happen to edit a Carbon source file with notepad on Windows which seems important.
It turns out we can make these work with very minimal complexity because the LF is still in the right place either way. This also lets us easily support mixtures of LF and CR+LF line endings gracefully. We create the line structures around the LF bytes and then have the byte-dispatch loop notice a CR followed by an LF and skip to the LF behavior.
Rather than add the remaining complexity around supporting bare CR and LF+CR sequences (both of which are quite rare now), this just adds diagnostics when we encounter a CR byte that won't fall out of our CR+LF handling. This is a better experience for users than the alternative. We still have a TODO to handle the full complexity of vertical whitespace, but I've updated it to reflect that the common case should be handled already.
This isn't complete though: we need to add support in string literal lexing, and we need to teach the diagnostic rendering to handle the error messages above better. But those will be future PRs, this is enough to unblock folks who happen to edit a Carbon source file with notepad on Windows which seems important.