hylo-lang / Lotsawa

A Swift implementation of the MARPA algorithms
Apache License 2.0
18 stars 2 forks source link

Consider more efficient lookup of partial parses to advance #2

Closed dabrahams closed 1 year ago

dabrahams commented 2 years ago

Auxilliary hashtables probably only become efficient when there are many partial parses active in each Earleme, but may be necessary to guarantee a strict O(N) upper bound on parsing.

Also possible: sort the elements of an Earleme so we can use binary search. Likely also only a win when there are many partial parses in each Earleme. Slightly complicated by the fact that the current earleme can't be sorted (though #6 would make that a non-issue).

Small linear searches are extremely hard to beat—this suggests that the break-even point for int-sized data is around 60 elements, and this suggests 22 elements for int triples.

Probably Aycock&Horspool optimization into a pushdown automaton would reduce the number of partial parses in each Earleme, so in the end this may never become a real win for any practical grammar.

Need to measure, obviously

dabrahams commented 1 year ago

We are now doing binary search in each Earleme.