search

utk-se / WorldSyntaxTree

Language-agnostic parsing of World of Code repositories
Other
20 stars 0 forks source link

Analysis step node text indexing order #13

Closed robobenklein closed 3 years ago

robobenklein commented 3 years ago

Need to evaluate difference between operations:

  1. Index and uniquify node text entries during insertion

Or

  1. Insert raw node text entries, nonunique
  2. Index node texts

  3. Parallel reduction either after or during collection to uniquify node text entries

    Most interested in total time to final DB state.