hdmf-dev / hdmf

The Hierarchical Data Modeling Framework
http://hdmf.readthedocs.io
Other
46 stars 26 forks source link

Cannot move containers or add/copy existing containers to another file #691

Open rly opened 2 years ago

rly commented 2 years ago

Description

See https://github.com/NeurodataWithoutBorders/helpdesk/discussions/21

Users may want to reorganize their files to conform to best practices or other reasons. However, moving containers from one location to another does not work, regardless of whether the file has been written or the user is using the export function. This is because the parent of the container is set once and cannot be reset.

Similarly, although HDMF (HDF5IO) supports taking datasets read from one file and writing them to a new file as a link or copy, taking entire containers that are read from one file and adding/writing them to a new file is not currently supported. This is because the parent of the container is set once and cannot be reset. Copying entire containers without calling the constructor and copying each field one by one has not yet been implemented. See https://github.com/hdmf-dev/hdmf/issues/331 and https://github.com/hdmf-dev/hdmf/pull/108

One solution is to allow users to reset the parent of a container, at their own peril. Another is to resolve copying of containers but that has proven to be more difficult than expected.

Environment

Python Executable: Conda
Python Version: Python 3.9
Operating System: Windows
HDMF Version: dev

Checklist

mavaylon1 commented 5 months ago

@rly could you give me an example of this in pynwb where one container moves.

Response: If a user accidentally puts data in acquisitions but it needs to be in processing. We need to think of multiple cases: 1) Say a user has not written the file. The user can use the reset_parent method and assign the new parent. This needs documentation. 2) Say a user already wrote the file. The user could a) Open in append mode, reset_parent, assign new parent. b) Similar idea with export

The concern is to make sure that the source attribute is updated (if needed).

TODOS: