Bug: CRLF and LF line ending handling in preprocessor

[ijsf]

I've been trying to make & run the test-wcc in WSL.

Apart from some shell script adjustments, most of the tests seem to run okay, but there seems to be an odd problem with line ending handling in the compiler.

When I check out all files with CRLF (Windows) line endings, which is default behaviour, and make test-wcc, I get some odd preprocessor errors:

make test-wcc
make -C tests clean && make -C tests test-wcc
make[1]: Entering directory './xcc_wasm_test/xcc_wsl/tests'
rm -rf table_test util_test parser_test initializer_test print_type_test \
        valtest dvaltest fvaltest link_test \
        a.out tmp* *.o mandelbrot.ppm \
        *.wasm
make[1]: Leaving directory './xcc_wasm_test/xcc_wsl/tests'
make[1]: Entering directory './xcc_wasm_test/xcc_wsl/tests'
## test.sh
Compile=[../wcc], Run=[../tool/runwasi.sh tmp.XNbuJeSBI5.wasm]
  Basic: OK: 9
  Struct: OK: 9
  Bitfield: OK: 6
  Initializer: OK: 20
  Function: OK: 18
  Error: OK: 90
  Error line no: OK: 6
../wcc -o valtest.wasm valtest.c
## valtest
../tool/runwasi.sh valtest.wasm
  all: OK: 237
  basic: OK: 48
  struct: OK: 30
  bitfield: OK: 43
  initializer: OK: 33
  function: OK: 34
  vla: OK: 14
../wcc -o dvaltest.wasm fvaltest.c
## dvaltest
../tool/runwasi.sh dvaltest.wasm
  number: OK: 29
  mix: OK: 4
../wcc -o fvaltest.wasm fvaltest.c
## fvaltest
../tool/runwasi.sh fvaltest.wasm
  number: OK: 29
  mix: OK: 4
All tests PASS!
make[1]: Leaving directory './xcc_wasm_test/xcc_wsl/tests'
make -C libsrc clean-test && make -C libsrc test-wcc
make[1]: Entering directory './xcc_wasm_test/xcc_wsl/libsrc'
rm -rf printf_test file_test stdlib_test math_test longjmp_test \
        a.out *.wasm tmp*
make[1]: Leaving directory './xcc_wasm_test/xcc_wsl/libsrc'
make[1]: Entering directory './xcc_wasm_test/xcc_wsl/libsrc'
../wcc -oprintf_test.wasm tests/printf_test.c
./xcc_wasm_test/xcc_wsl/libsrc/tests/xtest.h(29): `__VA_ARGS__' undeclared
  static void (*const tests[])(void) = {__VA_ARGS__}; \
                                        ^~~~~~~~~~~
./xcc_wasm_test/xcc_wsl/libsrc/tests/xtest.h(29): warning: cannot convert value from type `int' to `void(*)(void)'
  static void (*const tests[])(void) = {__VA_ARGS__}; \
                                        ^~~~~~~~~~~
./xcc_wasm_test/xcc_wsl/libsrc/tests/xtest.h(29): Initializer must be constant
  static void (*const tests[])(void) = {__VA_ARGS__}; \
                                        ^~~~~~~~~~~
./xcc_wasm_test/xcc_wsl/libsrc/tests/xtest.h(29): Unexpected character `\'(92)
  static void (*const tests[])(void) = {__VA_ARGS__}; \
                                                      ^
make[1]: *** [Makefile:155: printf_test.wasm] Error 1
make[1]: Leaving directory './xcc_wasm_test/xcc_wsl/libsrc'
make: *** [Makefile:262: test-wcc] Error 2

Furthermore, the line numbers ((29)) seem to be wrong.

I suspected this to be a bug in the compiler with line ending handling. So I checked out all files with LF (Unix) line endings, and all tests passed.

This seems to confirm that that some of compiler (e.g. the multi-line preprocessor code) makes an assumption about the line endings being LF despite that both CRLF and LF are possible depending on platform. This leads to wrong line numbers and incorrectly parsed preprocessors.

[tyfkda]

Reproduced.

[ijsf]

Can no longer reproduce, fix seems to be working 👍