Reduce stack usage when parsing

[WebFreak001]

Can be tested by doing ldc2 -O2 ./run_tests.sh

[codecov[bot]]

Codecov Report

Merging #477 (0519089) into master (fd5efb4) will increase coverage by 0.00%. The diff coverage is 100.00%.

Additional details and impacted files

[](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community) ```diff @@ Coverage Diff @@ ## master #477 +/- ## ======================================= Coverage 83.46% 83.47% ======================================= Files 11 11 Lines 8493 8498 +5 ======================================= + Hits 7089 7094 +5 Misses 1404 1404 ``` | [Impacted Files](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community) | Coverage Δ | | |---|---|---| | [src/dparse/parser.d](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community#diff-c3JjL2RwYXJzZS9wYXJzZXIuZA==) | `91.38% <100.00%> (ø)` | | | [src/dparse/stack\_buffer.d](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community#diff-c3JjL2RwYXJzZS9zdGFja19idWZmZXIuZA==) | `93.75% <100.00%> (+0.13%)` | :arrow_up: | | [test/tester.d](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community#diff-dGVzdC90ZXN0ZXIuZA==) | `99.65% <100.00%> (+<0.01%)` | :arrow_up: | ------ [Continue to review full report in Codecov by Sentry](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community). Last update [fd5efb4...0519089](https://codecov.io/gh/dlang-community/libdparse/pull/477?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dlang-community).

[github-actions[bot]]

✅ PR OK, no changes in deprecations or warnings

Total deprecations: 0

Total warnings: 0

Build statistics:

 ------ libdparse statistics ------

 statistics (-before, +after)
 library size=3511132 libdparse.a
 rough build time=16s

 ------ DCD statistics ------

 statistics (-before, +after)
 client size=1055768 bin/dcd-client
 server size=3018608 bin/dcd-server
-rough build time=75s
+rough build time=76s

-DCD run_tests.sh   Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.80
-DCD run_tests.sh   Maximum resident set size (kbytes): 9900
+DCD run_tests.sh   Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.79
+DCD run_tests.sh   Maximum resident set size (kbytes): 7932

 short requests: (215x)
     min request time =     0.007ms
-    10th percentile  =     0.110ms
-    median time      =     0.405ms
-    90th percentile  =     0.691ms
-    max request time =     1.375ms
+    10th percentile  =     0.111ms
+    median time      =     0.412ms
+    90th percentile  =     0.690ms
+    max request time =     1.399ms

 top 5 GC sources in server:
 bytes allocated, allocations, type, function, file:line
         7554112              81399 void[] std.array.Appender!(DSymbol*[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577
         3768320                460 void[] std.array.Appender!(TokenStructure!(ubyte, "\n    import dparse.lexer : Token;\n\n    this(Token token) pure nothrow @safe @nogc {\n        this(token.type, token.text, token.line, token.column, token.index);\n    }\n\n    int opCmp(size_t i) const pure nothrow @safe @nogc {\n        if (index < i) return -1;\n        if (index > i) return 1;\n        return 0;\n    }\n\n    int opCmp(ref const typeof(this) other) const pure nothrow @safe @nogc {\n        return opCmp(other.index);\n    }\n")[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577
         2277376                278 void[] std.array.Appender!(char[][]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577
         1830176              57193 std.array.Appender!(dsymbol.symbol.DSymbol*[]).Appender.Data std.array.Appender!(DSymbol*[]).Appender.this /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3452
-        1659904                621 void[] std.array.Appender!(const(TokenStructure!(ubyte, "import dparse.lexer:TokenTriviaFields,TriviaToken; mixin TokenTriviaFields;"))[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577
+        1487872                608 void[] std.array.Appender!(const(TokenStructure!(ubyte, "import dparse.lexer:TokenTriviaFields,TriviaToken; mixin TokenTriviaFields;"))[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577

Full build output

``` DUB version 1.31.1, built on Mar 12 2023 LDC - the LLVM D compiler (1.32.0): based on DMD v2.102.2 and LLVM 15.0.7 built with LDC - the LLVM D compiler (1.32.0) Default target: x86_64-unknown-linux-gnu Host CPU: icelake-server http://dlang.org - http://wiki.dlang.org/LDC Registered Targets: aarch64 - AArch64 (little endian) aarch64_32 - AArch64 (little endian ILP32) aarch64_be - AArch64 (big endian) amdgcn - AMD GCN GPUs arm - ARM arm64 - ARM64 (little endian) arm64_32 - ARM64 (little endian ILP32) armeb - ARM (big endian) avr - Atmel AVR Microcontroller bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) hexagon - Hexagon lanai - Lanai mips - MIPS (32-bit big endian) mips64 - MIPS (64-bit big endian) mips64el - MIPS (64-bit little endian) mipsel - MIPS (32-bit little endian) msp430 - MSP430 [experimental] nvptx - NVIDIA PTX 32-bit nvptx64 - NVIDIA PTX 64-bit ppc32 - PowerPC 32 ppc32le - PowerPC 32 LE ppc64 - PowerPC 64 ppc64le - PowerPC 64 LE r600 - AMD GPUs HD2XXX-HD6XXX riscv32 - 32-bit RISC-V riscv64 - 64-bit RISC-V sparc - Sparc sparcel - Sparc LE sparcv9 - Sparc V9 systemz - SystemZ thumb - Thumb thumbeb - Thumb (big endian) ve - VE wasm32 - WebAssembly 32-bit wasm64 - WebAssembly 64-bit x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 xcore - XCore Upgrading project in /home/runner/work/libdparse/libdparse/ Starting Performing "release" build using /opt/hostedtoolcache/dc/ldc2-1.32.0/x64/ldc2-1.32.0-linux-x86_64/bin/ldc2 for x86_64. Building libdparse 0.22.0+commit.9.g8d650ae: building configuration [library] STAT:------ libdparse statistics ------ STAT: STAT:statistics (-before, +after) STAT:library size=3511132 libdparse.a STAT:rough build time=16s STAT: STAT: STAT:------ DCD statistics ------ STAT: { "name": "dcd", "description": "The D Completion Daemon is an auto-complete program for the D programming language", "copyright": "Copyright © 2015-2020, Brian Schott", "authors": [ "Brian Schott" ], "license": "GPL-3.0", "dependencies": { ":dsymbol": "*", "libdparse": {"path":".."}, ":common": "*", "emsi_containers": "~>0.9.0" }, "subPackages": ["dsymbol", "common"], "versions": ["built_with_dub"], "configurations": [ { "name": "library", "targetType": "library", "excludedSourceFiles": [ "src/dcd/client/*", "src/dcd/server/main.d" ] }, { "name": "client", "targetType": "executable", "targetPath": "bin/", "targetName": "dcd-client", "excludedSourceFiles": [ "src/dcd/server/*" ] }, { "name": "server", "targetType": "executable", "targetPath": "bin/", "targetName": "dcd-server", "excludedSourceFiles": [ "src/dcd/client/*" ] } ] } { "fileVersion": 1, "versions": { "dsymbol": "0.14.1", "emsi_containers": "0.9.0", "libdparse": {"path":".."}, "msgpack-d": "1.0.4", "stdx-allocator": "2.77.5" } } STAT:statistics (-before, +after) STAT:client size=1055768 bin/dcd-client STAT:server size=3018608 bin/dcd-server STAT:rough build time=76s STAT: unix:tc001: ... Pass unix:tc002: ... Pass unix:tc003: ... Pass unix:tc004: ... Pass unix:tc005: ... Pass unix:tc006: ... Pass unix:tc007: ... Pass unix:tc008: ... Pass unix:tc009: ... Pass unix:tc010: ... Pass unix:tc011: ... Pass unix:tc012: ... Pass unix:tc013: ... Pass unix:tc014: ... Pass unix:tc015: ... Pass unix:tc016: ... Pass unix:tc017: ... Pass unix:tc018: ... Pass unix:tc019: ... Pass unix:tc020: ... Pass unix:tc021: ... Pass unix:tc022: ... Pass unix:tc023: ... Pass unix:tc024: ... Pass unix:tc025: ... Pass unix:tc026: ... Pass unix:tc027: ... Pass unix:tc028: ... Pass unix:tc029: ... Pass unix:tc030: ... Pass unix:tc031: ... Pass unix:tc032: ... Pass unix:tc033: ... Pass unix:tc034: ... Pass unix:tc035: ... Pass unix:tc036: ... Pass unix:tc037: ... Pass unix:tc038: ... Pass unix:tc039: ... Pass unix:tc040: ... Pass unix:tc041: ... Pass unix:tc042: ... Pass unix:tc043: ... Pass unix:tc044: ... Pass unix:tc045: ... Pass unix:tc046: ... Pass unix:tc047: ... Pass unix:tc048: ... Pass unix:tc049: ... Pass unix:tc050: ... Pass unix:tc051: ... Pass unix:tc052: ... Pass unix:tc053: ... Pass unix:tc054: ... Pass unix:tc055: ... Pass unix:tc056: ... Pass unix:tc057: ... Pass unix:tc058: ... Pass unix:tc059: ... Pass unix:tc060: ... Pass unix:tc061: ... Pass unix:tc062: ... Pass unix:tc620: ... Pass unix:tc_access_modifiers: ... Pass unix:tc_accesschain_type: ... Pass unix:tc_anon_class: ... Pass unix:tc_anon_struct: ... Pass unix:tc_bang_op_or_template: ... Pass unix:tc_base_template_type: ... Pass unix:tc_body_var: ... Pass unix:tc_calltip_in_func: ... Pass unix:tc_char_dot: ... Pass unix:tc_complete_kw: ... Pass unix:tc_currmod_fqn: ... Pass unix:tc_ditto_scopes: ... Pass unix:tc_empty_module: ... Pass 00000unix:tc_empty_requests: ... Pass unix:tc_erroneous_body_content: ... Pass unix:tc_extended_ditto: ... Pass unix:tc_extended_types: ... Pass unix:tc_if_auto_array: ... Pass unix:tc_if_var: ... Pass unix:tc_import_symbol_list: ... Pass unix:tc_incomplete_switch: ... Pass unix:tc_issue558: ... Pass unix:tc_locate_ufcs_function: ... Pass unix:tc_middle_of_utf: ... Pass unix:tc_module_scope_op: ... Pass unix:tc_named_mixin: ... Pass unix:tc_opaque_structs: ... Pass unix:tc_pointer_type_printing: ... Pass unix:tc_pointers: ... Pass unix:tc_recursive_public_import: ... Pass unix:tc_rm_import: ... Pass unix:tc_scope_mess: ... Pass unix:tc_selective_import_list: ... Pass unix:tc_super_scope: ... Pass unix:tc_template_param_props: ... Pass unix:tc_traits: ... Pass unix:tc_ufcs_alias_this_completion: ... Pass unix:tc_ufcs_array_type_completion: ... Pass unix:tc_ufcs_calltip_in_func: ... Pass unix:tc_ufcs_fundamental_types_completion: ... Pass unix:tc_ufcs_pointer_type_completion: ... Pass unix:tc_ufcs_struct_completion: ... Pass STAT:DCD run_tests.sh Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.79 STAT:DCD run_tests.sh Maximum resident set size (kbytes): 7932 STAT: STAT:short requests: (215x) STAT: min request time = 0.007ms STAT: 10th percentile = 0.111ms STAT: median time = 0.412ms STAT: 90th percentile = 0.690ms STAT: max request time = 1.399ms STAT: STAT: unix:tc001: ... Pass unix:tc002: ... Pass unix:tc003: ... Pass unix:tc004: ... Pass unix:tc005: ... Pass unix:tc006: ... Pass unix:tc007: ... Pass unix:tc008: ... Pass unix:tc009: ... Pass unix:tc010: ... Pass unix:tc011: ... Pass unix:tc012: ... Pass unix:tc013: ... Pass unix:tc014: ... Pass unix:tc015: ... Pass unix:tc016: ... Pass unix:tc017: ... Pass unix:tc018: ... Pass unix:tc019: ... Pass unix:tc020: ... Pass unix:tc021: ... Pass unix:tc022: ... Pass unix:tc023: ... Pass unix:tc024: ... Pass unix:tc025: ... Pass unix:tc026: ... Pass unix:tc027: ... Pass unix:tc028: ... Pass unix:tc029: ... Pass unix:tc030: ... Pass unix:tc031: ... Pass unix:tc032: ... Pass unix:tc033: ... Pass unix:tc034: ... Pass unix:tc035: ... Pass unix:tc036: ... Pass unix:tc037: ... Pass unix:tc038: ... Pass unix:tc039: ... Pass unix:tc040: ... Pass unix:tc041: ... Pass unix:tc042: ... Pass unix:tc043: ... Pass unix:tc044: ... Pass unix:tc045: ... Pass unix:tc046: ... Pass unix:tc047: ... Pass unix:tc048: ... Pass unix:tc049: ... Pass unix:tc050: ... Pass unix:tc051: ... Pass unix:tc052: ... Pass unix:tc053: ... Pass unix:tc054: ... Pass unix:tc055: ... Pass unix:tc056: ... Pass unix:tc057: ... Pass unix:tc058: ... Pass unix:tc059: ... Pass unix:tc060: ... Pass unix:tc061: ... Pass unix:tc062: ... Pass unix:tc620: ... Pass unix:tc_access_modifiers: ... Pass unix:tc_accesschain_type: ... Pass unix:tc_anon_class: ... Pass unix:tc_anon_struct: ... Pass unix:tc_bang_op_or_template: ... Pass unix:tc_base_template_type: ... Pass unix:tc_body_var: ... Pass unix:tc_calltip_in_func: ... Pass unix:tc_char_dot: ... Pass unix:tc_complete_kw: ... Pass unix:tc_currmod_fqn: ... Pass unix:tc_ditto_scopes: ... Pass unix:tc_empty_module: ... Pass 00000unix:tc_empty_requests: ... Pass unix:tc_erroneous_body_content: ... Pass unix:tc_extended_ditto: ... Pass unix:tc_extended_types: ... Pass unix:tc_if_auto_array: ... Pass unix:tc_if_var: ... Pass unix:tc_import_symbol_list: ... Pass unix:tc_incomplete_switch: ... Pass unix:tc_issue558: ... Pass unix:tc_locate_ufcs_function: ... Pass unix:tc_middle_of_utf: ... Pass unix:tc_module_scope_op: ... Pass unix:tc_named_mixin: ... Pass unix:tc_opaque_structs: ... Pass unix:tc_pointer_type_printing: ... Pass unix:tc_pointers: ... Pass unix:tc_recursive_public_import: ... Pass unix:tc_rm_import: ... Pass unix:tc_scope_mess: ... Pass unix:tc_selective_import_list: ... Pass unix:tc_super_scope: ... Pass unix:tc_template_param_props: ... Pass unix:tc_traits: ... Pass unix:tc_ufcs_alias_this_completion: ... Pass unix:tc_ufcs_array_type_completion: ... Pass unix:tc_ufcs_calltip_in_func: ... Pass unix:tc_ufcs_fundamental_types_completion: ... Pass unix:tc_ufcs_pointer_type_completion: ... Pass unix:tc_ufcs_struct_completion: ... Pass STAT:top 5 GC sources in server: STAT:bytes allocated, allocations, type, function, file:line STAT: 7554112 81399 void[] std.array.Appender!(DSymbol*[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577 STAT: 3768320 460 void[] std.array.Appender!(TokenStructure!(ubyte, "\n import dparse.lexer : Token;\n\n this(Token token) pure nothrow @safe @nogc {\n this(token.type, token.text, token.line, token.column, token.index);\n }\n\n int opCmp(size_t i) const pure nothrow @safe @nogc {\n if (index < i) return -1;\n if (index > i) return 1;\n return 0;\n }\n\n int opCmp(ref const typeof(this) other) const pure nothrow @safe @nogc {\n return opCmp(other.index);\n }\n")[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577 STAT: 2277376 278 void[] std.array.Appender!(char[][]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577 STAT: 1830176 57193 std.array.Appender!(dsymbol.symbol.DSymbol*[]).Appender.Data std.array.Appender!(DSymbol*[]).Appender.this /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3452 STAT: 1487872 608 void[] std.array.Appender!(const(TokenStructure!(ubyte, "import dparse.lexer:TokenTriviaFields,TriviaToken; mixin TokenTriviaFields;"))[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577 ```