where "non-DANDI layout" file is found to be not uploaded.
I think overall implementation, upon processing command upload for a specific (sub)path (could be top of dandiset), and having obtained list of paths which were uploaded, should then call find_unused_paths(path, used_paths), which would os.walk that path, and if encountering folder, check if any path among user_paths is under that folder path, if not -- add entire folder path into result, if some is, descend into that folder (recurse); if encountering a non-folder, see if was among used_paths and if not -- add to result.
Common known to be ignored paths (like vcs folders and some OSX trash folders) should be ignored altogether.
Then WARNING could report total number of skipped entities (folders of files) with some (up to 10?), and log the rest at DEBUG level (into log file).
Might need to be implemented along with
and overall provide partial (UX) resolution to e.g.
where "non-DANDI layout" file is found to be not uploaded.
I think overall implementation, upon processing command upload for a specific (sub)
path(could be top of dandiset), and having obtained list of paths which were uploaded, should then callfind_unused_paths(path, used_paths), which wouldos.walkthatpath, and if encountering folder, check if any path amonguser_pathsis under that folder path, if not -- add entire folder path into result, if some is, descend into that folder (recurse); if encountering a non-folder, see if was amongused_pathsand if not -- add to result.Common known to be ignored paths (like vcs folders and some OSX trash folders) should be ignored altogether.
Then WARNING could report total number of skipped entities (folders of files) with some (up to 10?), and log the rest at DEBUG level (into log file).