Closed frheault closed 1 year ago
@arokem It is still a bit rought, but is this what you had in mind? For me it is working pretty well.
In the tracking_introduction_eudx.py
example from Dipy doing this works:
streamlines_generator = LocalTracking(csa_peaks, stopping_criterion, seeds,
affine=affine, step_size=.5)
#streamlines = Streamlines(streamlines_generator)
trx = TrxFile.from_lazy_tractogram(streamlines_generator, hardi_img, extra_buffer=100000)
Using an extra_buffer of 0 will recreate a new TrxFile at each chunk (few thousand streamlines) which is a lot of new temporary folders and all and rewriting data. Using a bigger extra_buffer will create an empty TrxFile that can use preallocations and only get resized (if needed) a few times and the function finishes with a final resize.
I did not benchmark it.
Is this still WIP?
Is this still WIP?
With #37 passed and the test passing, I think this is ready for testing.
I need to test with larger files to quantify how much time is saved or how much allocations are avoided.
I know it works but I want to be able to have a few numbers to back it up.
Sounds good! No rush.
Create TRX file from Generator or LazyTractogram and write to memmap in chunks. Support a buffer to avoid resizing the TRX at every chunk.
Added tests to verify if it is working.