Closed rly closed 5 years ago
For NWB:N files I don't see a reason why you would not always want to cache the spec. The amount of data is minimal and it helps ensure that the file remains accessible.
The only use-case I could think of where you may not want to do this is if you want to write partial files, e.g., a single TimeSeries container, rather than a full NWB:N file. The question of partial files has come up in the context of integration with data management. However, even in this case, I think you are probably fine with caching the spec.
The tests are considerable slower when caching the spec
On Wed, May 29, 2019, 5:36 PM Oliver Ruebel notifications@github.com wrote:
For NWB:N files I don't see a reason why you would not always want to cache the spec. The amount of data is minimal and it helps ensure that the file remains accessible.
The only use-case I could think of where you may not want to do this is if you want to write partial files, e.g., a single TimeSeries container, rather than a full NWB:N file. The question of partial files has come up in the context of integration with data management. However, even in this case, I think you are probably fine with caching the spec.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NeurodataWithoutBorders/pynwb/issues/967?email_source=notifications&email_token=AAGOEETYINEULAEGJGNZQHTPX4OSLA5CNFSM4HQY3TDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWRAKOY#issuecomment-497157435, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGOEEWU2KY4VTIAQS7VSPDPX4OSLANCNFSM4HQY3TDA .
The tests are considerable slower when caching the spec
Which just means that writing the spec to the file takes a lot of time or?
I'm not sure if it's writing or reading, just something I noticed. I think it would be worthwhile to profile this to see what the time penalty is.
Would there be any issues in loading namespaces by default on read? If specs are cached by default, they should be read by default, right?
i.e. make load_namespaces=true
the default when calling NWBHDF5IO(path, 'r', load_namespaces=true)
I think that would replace any custom classes that have already been imported and/or registered
Is there a reason why we would not want to cache the spec when writing a file?
Advantages of always caching spec: fewer complex use cases and interactions to handle and tests to make.
Parallel issue on hdmf: https://github.com/hdmf-dev/hdmf/issues/76