Open diamondap opened 2 years ago
I was just starting to file issues for implementations. I would also note:
fetch.txt
are supposed to be encoded the same as manifest paths. I have not done a survey of fetch.txt
implementations, but I would be surprised if there was not a general problem here as well. I'm not sure how this could be addressed in a backward compatible way.Thanks. The fallback for backwards compatibility is a good idea.
It occurred to me that another approach to backwards compatible validation would be to only decode %0D
, %0A
, and %25
when decoding paths in manifests. Normally, when percent-decoding, you'd decode all encoded characters as described here. However, by only decoding these three a correct BagIt 1.0 implementation would still be able to validate most bags produced by existing implementations.
For example, if a bag contains the file test%201.txt
, then an existing implementation would write it to the manifest as data/test%201.txt
when it should actually be data/test%25201.txt
. However, if you only decode %0D
, %0A
, and %25
, then the paths are equivalent.
This approach does not work for files that naturally include these three strings. For example, if a bag has a file named test%251.txt
. Existing implementations would write it to the manifest as data/test%251.txt
when it should be data/test%25251.txt
. These paths are not equivalent. The first decodes to data/test%1.txt
and the second decodes to data/test%251.txt
.
While not perfect, I think this approach would greatly improve validation compatibility.
This is a general bug filed against the manifest spec at https://github.com/LibraryOfCongress/bagit-spec/issues/46.
TLDR: No existing baggers correctly implement the BagIt spec's filename percent-encoding requirements. Fixing this in DART likely means breaking compatibility with all other baggers and validators.
Wait to see how the community moves on this.