This patch changes the migration script to index after every commit instead of all at once at the end. That way, the indexer will decide if the index should actually need to be run, and it's less likely that we'll run into memory issues migrating larger ledgers.
It also creates a unique file changes channel for each commit, and merges those channels into the changes channel that was passed in to the migrator. This is so we can run the indexer after each commit because it closes the changes channel after it runs, but we'd like to keep the original changes channel open in this case.
This patch changes the migration script to index after every commit instead of all at once at the end. That way, the indexer will decide if the index should actually need to be run, and it's less likely that we'll run into memory issues migrating larger ledgers.
It also creates a unique file changes channel for each commit, and merges those channels into the changes channel that was passed in to the migrator. This is so we can run the indexer after each commit because it closes the changes channel after it runs, but we'd like to keep the original changes channel open in this case.