Closed fralik closed 6 years ago
Please read the information in the master_file_example.m about these file paths. The RegFileRoot is supposed to be an SSD (which doesn't have unlimited capacity). The code will be much slower if you use your normal hard drives. Also, at least in our case, and potentially many others, the data is located on a server, not locally, so people may want to save registered tiffs there (RegFileTiffLocation). We don't use the registered tiff functionality, but others do. The RegFileBinLocation is somewhere (not your SSD, because your SSD has limited capacity, a binary file can be around 100gb) where you can copy the binary if you want to keep it for further processing.
You can do whatever you want with the pipeline, but your speed will suffer if you choose to do all the processing on a hard drive. I'll fix the readme RegFileRoot description.
I see. I read the master file example of course. Perhaps the naming of options is cryptic. My point was that it is very unintuitive to figure out what files will be saved where by option names and comments.
I see these things from your description:
You can make this happen like this:
ResultsSavePath
- local or remote location where result files should be kept. Let's assume unlimited capacity there.FastStoragePath
- location on (path to) SSD that will be used to store intermediate results. If omitted, everything goes through ResultsSavePath
. Both plane binary and interpolated plane binary files can be stored there.SaveIntermediateResults
- boolean. If true, then copies things from FastStoragePath
over to ResultsSavePath
. One can, of course, demand more flexibility and make it in a similar manner as RegFileTiffLocation
, i.e. instead of boolean make IntermediateSavePath
.RegFileTiffLocation
- if present, then save binary as tiff files in that location. However, SaveRegistrationTiff
: boolean can be better. If set to true, save tiff files in ResultsSavePath
.I end up with the same number of parameters - 4, but only 2 of them are paths. The other two can be boolean. Also, having this FastStoragePath
, you can actually store everything you want (for speed) there before you transfer it to a different location.
Here is how the usage table will look like after the pipeline is finished:
ResultsSavePath | FastStoragePath | SaveRegistrationTiff | SaveIntermediateResults | |
---|---|---|---|---|
regops | x | |||
plane binary | x | true | ||
interpolated | x | true | ||
plane binary | x | false | ||
interpolated | x | false | ||
plane tiff | x | true |
Fast storage will be used during processing regardless of SaveIntermediateResults
value. It is only that the results are copied/moved to a bigger storage if SaveIntermediateResults
is true.
My naming is perhaps not the best and different words should be chosen. With paths from the master file example it will be like this:
ResultsSavePath = 'D:/DATA/F';
FastStoragePath = 'C:/DATA/';
SaveRegistrationTiff = false;
SaveIntermediateResults = false;
Hi!
I was wondering if there are any plans on straightening file paths for registration output? What we have now:
RegFileRoot
- location for binary file.ResultsSavePath
- some results go here, seems not to be registration related.RegFileTiffLocation
- if provided, saves registration results as tiff files in that folder. The sole purpose of this option seems to specify whether to save registration results as tiff files or not.RegFileBinLocation
- if provided, copies bin file with registration results to specified location, but not only this.Looks kind of OK, but the usage is cryptic. Here is how these variables are used. There are several files created:
ResultsSavePath
. Why not inRegFileRoot
?!RegFileRoot
.RegFileBinLocation
. Why not inRegFileRoot
?!ResultsSavePath
.RegFileTiffLocation
.From this list of 5 file types it seems like 4 are registration related. However, all of them are saved in different folders! Here is an overview in table form.
My suggestion: make it simple and clean. Basically, throw away all the options but one -
ResultsSavePath
. If you think that registration results must be placed in a different user-specified folder, then have one additional optionRegistrationSavePath
. Otherwise, just put it underResultsSavePath
, i.e.ResultsSavePath/registration
.RegFileTiffLocation
option is in realty a booleansaveRegistrationTiff
option, i.e. it is sole purpose is to give user an option whether to save results as tiff(s) or not.It is also worth mentioning that the usage of
RegFileRoot
contradicts README file where it says All of these filepaths are completed with separate subfolders per animal and experiment.RegFileRoot
is not completed with subfolders.I can make a pull request for it if you think this sounds OK. This is however a breaking change. I do not know how you handle Suite2P versions and what your release plans are..
I am abstracting all these things away in my work I mentioned in #131, but it will be much better if we could clear things up as a first step.