Adds a ignore specification region where ignorable terminals can be specified. The parse stage was modified to scan ignorable terminals into item "shadow vectors", allowing the parse tree to be constructed as if the ignored terminals were a part of the item's grammar production. This incurs a small overhead even when ingnorable terminals are not used, however this will be improved when #32 is implemented.
Note that #32 is much more important after this change, as symbols will potentially be cloned much more often during the parse phase if shadow vectors are cloned often. If the symbols are encoded as usize instead of String, these clone operations will be much faster.
Adds a
ignorespecification region where ignorable terminals can be specified. The parse stage was modified to scan ignorable terminals into item "shadow vectors", allowing the parse tree to be constructed as if the ignored terminals were a part of the item's grammar production. This incurs a small overhead even when ingnorable terminals are not used, however this will be improved when #32 is implemented.Note that #32 is much more important after this change, as symbols will potentially be cloned much more often during the parse phase if shadow vectors are cloned often. If the symbols are encoded as
usizeinstead ofString, these clone operations will be much faster.Closes #93 and unblocks #94.