[Feature]: Remove or enchance the warning about ignoring cached name spaces

h-mayorquin commented 8 months ago

What would you like to see added to HDMF?

I get this warning to standard error sometimes and I don't know what to make out of it:

/home/heberto/miniconda3/envs/neuroconv_env/lib/python3.10/site-packages/hdmf/spec/namespace.py:531: UserWarning: Ignoring cached namespace 'hdmf-common' version 1.5.0 because version 1.8.0 is already loaded.
  warn("Ignoring cached namespace '%s' version %s because version %s is already loaded."
/home/heberto/miniconda3/envs/neuroconv_env/lib/python3.10/site-packages/hdmf/spec/namespace.py:531: UserWarning: Ignoring cached namespace 'core' version 2.4.0 because version 2.6.0-alpha is already loaded.
  warn("Ignoring cached namespace '%s' version %s because version %s is already loaded."
/home/heberto/miniconda3/envs/neuroconv_env/lib/python3.10/site-packages/hdmf/spec/namespace.py:531: UserWarning: Ignoring cached namespace 'hdmf-experimental' version 0.1.0 because version 0.5.0 is already loaded.
  warn("Ignoring cached namespace '%s' version %s because version %s is already loaded."

It comes from here: https://github.com/hdmf-dev/hdmf/blob/41fabd4eea4dc1e2a2c55af5b81358d1de86f3b1/src/hdmf/spec/namespace.py#L527-L532

I have the feeling that warnings should be actionable by the person that receives them. I feel that displaying this to end users is obscure (as I assume most users are not familiar with the concept of namespaces in the context of hdmf) and non-actionable. It is not clear what (and if) there is something wrong and what should be done about it.

What solution would you like?

If this is something that the end users should be concerned about, maybe enchance the warning to add instructions of what should be done or try.

If this is something that end users should not be concerned about maybe we could eliminate or restrict the warning so only the people that should be concerned get it.

Can this be done?

Do you have any interest in helping implement the feature?

Yes.

oruebel commented 8 months ago

I get this warning to standard error sometimes and I don't know what to make out of it:

These warnings can happen when loading a namespace. In practice this is usually the case when opening a file with load_namespaces=True (which is now the default). HDMF/PyNWB does not allow for multiple different versions of the same namespace (e.g., and extension) to be used at the same time. This warning indicates that a version of a namespace you are trying to load has already been loaded and therefore the namespace is being ignored.

In most cases, this currently does not lead to downstream issues, but it can if different versions of an extension are incompatible with each other. As such, I do think we should keep some form of a warning. However, I do agree that this should be improved to clarify what this means for the user. How about the following:

Instead of issuing a separate warning for each namespace, we could collect all the namespaces that are being ignored in a list (since this is happening in a single loop) and issue a single warning after the loop
We could add a statement along the lines of If the versions of the already loaded namespaces are compatible with the expected versions then this warning can be ignored.
For hdmf-common and the NWB core namespace we could potentially add some more sophisticated logic to check if the versions are compatible and only warn for those if they are actually incompatible. This should reduce the number of times we warn about this.

rly commented 8 months ago

I could have sworn we had already made an issue on this but maybe we just discussed it.

For hdmf-common and NWB core, I think we should raise the warning only if the cached namespace is newer than the loaded namespace, and thus we cannot ensure compatibility. The warning should recommend updating to the latest version of PyNWB. If the cached namespace is older or equal to than the loaded namespace, then if we have written HDMF and PyNWB well, then there should not be any compatibility issues. This is the most common case.

sneakers-the-rat commented 4 weeks ago

Came across this just now and looked to see if this was already raised because it seems like odd behavior to me - i would expect when I load a file that has embedded some schema that those schema would be used. I could understand if i had multiple files open with different versions why there would be problems (although i'm not sure why the namespace catalog can't hold multiple versions at a time, seems simple enough to do), but it's especially puzzling when i am only loading one file fresh after booting up my interpreter.

Seems like the correct behavior would be to replace any cached namespaces with the ones that are correct and provided by my file, or else what's the purpose of embedding them along with my file? I couldn't figure out how to force this behavior even with load_namespaces = True. that incidentally solves the problem of the warning, because there shouldn't be a warning in most cases :)

hdmf-dev / hdmf