dlang-community / libdparse

Library for lexing and parsing D source code
https://libdparse.dlang.io
Boost Software License 1.0
114 stars 56 forks source link

Program exited with code -11 on not Windows and ldc-1.29.0 #457

Closed outlandkarasu closed 2 years ago

outlandkarasu commented 2 years ago

When using DLexer with sse42 = true on not Windows and ldc-1.29.0, DLexer constructor has crashed with exited code -11.

Reproducing code is bellow.

/+dub.sdl:
dependency "libdparse" version="~>0.19.1"
+/
import std;

void main()
{
    import dparse.lexer : LexerConfig, StringCache, DLexer;

    auto sourceCode = q{
        int x;
    };  

    LexerConfig config;
    auto cache = StringCache(StringCache.defaultBucketCount);
    auto lexer = DLexer(sourceCode, config, &cache); // Program exited with code -11 

    // this code reproduce the error too.
    // auto lexer = DLexer(sourceCode, config, &cache, true);

    // But this code does not reproduce the error
    //auto lexer = DLexer(sourceCode, config, &cache, false);
}

This error is not occurred with ldc-1.28.0.

Error location is skip function insrc/dparse/lexer.d.

asm pure nothrow @nogc
{
    naked;
    movdqu XMM1, [RDX];
    mov R10, constant;
    movq XMM2, R10;
    mov RAX, charsLength;
    mov RDX, 16;
    pcmpestri XMM2, XMM1, flags;
    add [RSI], RCX;
    add [RDI], RCX;
    ret;
}
WebFreak001 commented 2 years ago

@kinke LDC regression? I would assume an assembler function should not change between compiler versions.

kinke commented 2 years ago

v1.29 came with an extern(D) ABI change that was highlighted in the change/release log, especially wrt. naked asm assuming params in specific registers.

maxhaton commented 2 years ago

Some data on something like 650K lines of D code in one go.

LDC 1.28.1 w/ inline asm enabled.

dparseCount -> 73 ms, 701 μs, and 1 hnsec 3619150 tokens
sdcCount -> 75 ms and 67 μs 3568151 tokens
dmdFrontendCount -> 74 ms, 92 μs, and 9 hnsecs 3568151 tokens

Ditto but without inline asm enabled

dparseCount -> 83 ms, 751 μs, and 9 hnsecs 3619150 tokens
sdcCount -> 76 ms, 621 μs, and 9 hnsecs 3568151 tokens
dmdFrontendCount -> 73 ms, 689 μs, and 2 hnsecs 3568151 tokens

Similar trends on an M1 macbook pro.

We could fix the asm but honestly lexing D code is so much faster than analysing it, it wouldn't hurt to just get rid.