Open blake-watkins opened 7 months ago
Thanks for the suggestion.
I remember having some thoughts about the proper solution relying on external format mechanism of the Lisp implementation. The idea is that the byte sequences 0d 0a
for Windows and just 0a
for UNIX are both turned into the Lisp character #\Newline
when they pass through the external format associated with the input stream. If I remember correctly, CCL has worked like that for a long time and SBCL now also supports different newline encodings in the external format (Since release 2.3.11, "enhancement: external formats for unibyte encodings and utf-8 now support newline variants.").
Is the actual use-case about parsing from a string, or this that just an artificial example to demonstrate the problem?
Thanks for your response. The example is just artificial to show the issue, actually I was trying to use aws-sdk
to make some AWS calls. It tries to get its configuration from config files. I had copied an example and since I was on windows the lines ended in CRLF. aws-sdk
calls parser.ini:parse
here with the name of the config file, but the parse was failing.
It's not an problem for me since I just switched the line endings, but thought it would be good to fix for others (there is an open issue in aws-sdk
that I think is probably the same as I ran into).
Thanks for the info on the external format, unfortunately the latest SBCL binary for windows is 2.3.2 and I've been having some permission issues building from source this morning so I can't test to see if updating the SBCL version would also fix it. I'll try again when I have time.
I was looking at
aws-sdk
, which usesparser.ini
to parse configuration and credential INI files, which uses this library as a parser. The parse was failing since windows ends its lines with#\Return #\Newline
, not just#\Newline
.Could you add
#\Return
to the whitespace definition inparser.common-rules/src/rules-whitespace.lisp
? Or change the whitespace definition inparser.ini
?As a test case, the following call is currently failing, but adding
#\Return
to the list gives the correct answer: