jedwards1211 / metacave

One cave survey data format to rule them all
5 stars 3 forks source link

Add support for breaking up MetaCave into multiple files #15

Open vpicaver opened 3 years ago

vpicaver commented 3 years ago

In an effort to make cave data more version control friendly, I propose that the metacave format can be broken up into multiple files. Any JSON object or array in the metacave file can be referenced in a seperate file using a url format. Files can not reference each other (cyclic dependency). We might want to constrain this to particular variables.

Url format is a string: Relative filename (recommend): "file://subFilename.metacave"

Absolute "file:///home/user/sauce.metacave"

Internet? "http://cavewhere.com/superCave.metacave"

jedwards1211 commented 3 years ago

Hmmm I'm totally for being able to split up into multiple files. I think I'd rather not support absolute paths? (David McKenzie's old walls workspace was full of absolute paths, yay...) I'm even tempted to say only relative paths to siblings and subdirectories be allowed, with no ../ so that janky stuff doesn't happen (I remember seeing some messed up relative paths in the Walls Sistema Oaxaca data...) and to make it less likely that people forget to include everything necessary in a zipfile.

As far as internet, the main risk there is that someone would download a datafile without knowing it's got an internet URL in it, go on an expedition to somewhere they don't have internet and then not be able to open the data. Apps could cache downloaded data if the file is opened with internet access, but that wouldn't help people if they didn't open the file before going somewhere they don't have internet.

I wish Git had some builtin JSON diffing, oh well.

vpicaver commented 3 years ago

Yea, I think we should only support relative paths.

I also agree that ../ can cause some major problems.

Yea urls on the internet is probably something we don't want to support either.

We should probably have a prefix like: relativePath://the/path/to/child/directories/cave.metacave

where relativePath:// is the prefix