Upon ingestion, we read each JSON line, we parse it and create the proper nodes and edges in the graph database.
Each insertion/update was done in its own traversal, which had an impact on ingestion speed.
This PR proposes to put together the operations to do to ingest a derivation in a single, shared traversal, to reduce the count of transactions and boost performances.
Upon ingestion, we read each JSON line, we parse it and create the proper nodes and edges in the graph database. Each insertion/update was done in its own traversal, which had an impact on ingestion speed.
This PR proposes to put together the operations to do to ingest a derivation in a single, shared traversal, to reduce the count of transactions and boost performances.