Closed mikejritter closed 4 years ago
@mikejritter : If it isn't too much trouble I think it is probably a good idea to do this before I test it:
Finally... when the filename in the archive do not match the filename of the archive (e.g. test-bag/ and sample-bag.tar), the BagDeserializers will return an incorrect path. This can be fixed before the PR is merged if desired.
@dbernstein I think there are a few additional updates for this for all the tests to pass from the changes #139 brought in. I'll get changes for that tomorrow in addition to seeing if I can get in a fix for the issues you commented on.
@dbernstein I made adjustments according to your comments and also updated the BagDeserializers
to prioritize the name of the path in the archive. I opted to have it fallback to the filename minus the extension if one is not found, which felt like a sane default.
The BagDeserializerTest
was also updated to check for this mismatch, and while I was updating it I made it a Parameterized
test since all the tests were the same.
@mikejritter : I ran the test, but was running into the following error (after modifying the tar commands - I changed them from tar xf ...
to tar cf ...
and tar xzf ...
to tar czf ...
:
Here's the the tar import: (tar.gz file also failed for similar reasons).
Daniels-MBP$ java -Dfcrepo.log.importexport=DEBUG -jar ~/code/fcrepo-import-export/target/fcrepo-import-export-0.4.0-SNAPSHOT.jar --mode import -r http://localhost:8080/rest --dir fcrepo-3205.tar.gz --binaries -g default -G bag-config.yml -u fedoraAdmin:fedoraAdmin
DEBUG 21:56:43.704 (ArgParser) Command line argments weren't valid for specifying a config file.
DEBUG 21:56:44.114 (SerializationSupport) /Users/danielbernstein/tmp/btr/fcrepo-3205.tar.gz: application/gzip
ERROR 21:56:44.119 (ImportExportDriver) Error performing import/export: BagProfile does not allow application/gzip. Accepted serializations are:
application/tar
DEBUG 21:56:44.122 (ImportExportDriver) Stacktrace:
java.lang.RuntimeException: BagProfile does not allow application/gzip. Accepted serializations are:
application/tar
at org.fcrepo.importexport.common.SerializationSupport.deserializerFor(SerializationSupport.java:132) ~[fcrepo-import-export-0.4.0-SNAPSHOT.jar:na]
at org.fcrepo.importexport.importer.Importer.loadBagProfile(Importer.java:193) ~[fcrepo-import-export-0.4.0-SNAPSHOT.jar:na]
at org.fcrepo.importexport.importer.Importer.
@dbernstein there are two things I think we can do for this:
x-tar/zip/gzip/x-gzip
to the supported serialization types for the default bag profileSerializationSupport
to have the x-
prefixed content types go to the "normal" form, e.g. application/x-tar -> application/tar
. Since some of the bag profiles were specifying things like application/tar
and application/x-tar
I had kept them separate in the mapping, but I don't see any harm in unifying them.Otherwise I think I had been testing with the beyondtherepository
profile, but I do think it would be good to at least update the default profile with more supported types.
@mikejritter : were you going to add all the supported types that you had in beyond the repository to the default? ie
"application/x-tar",
"application/x-gzip",
"application/x-7z-compressed"
Once that's in I'll merge it (as I tested successfully with the btr profile.
@dbernstein I updated the default profile to have all serialization formats, it should be good to go. I have a note for myself about further coercion for potentially doing x-tar -> tar
, and I think that's something which should stay as a note for now since the profile has the content types enumerated.
Resolves: https://jira.lyrasis.org/browse/FCREPO-3205
Adds classes to deserialize zip, tar, and gzip BagIt bags
New Dependencies
application/x-compressed-zip
on Windows,application/zip
on linux). The only issue was that it is precise in the content type it returns, so I did coercion of similar types in theSerializationSupport
to map types likegtar
andx-gtar
totar
andx-tar
.Notes
\\.zip
forexample-bag.zip
was used in order to get onlyexample-bag
. I that this might not always be the case (maybe a gzipped tarball is just named .tar) so I switched this to the current implementation of getting the last index of a.
. I'm still not 100% on this, though.GZipBagDeserializer
only perform deflation of the gzip file. This was done in case there was not a tarball in the underlying file... though I'm not sure if this is the flexibility we need. There would likely be an exception thrown either way, so theGZipBagDeserializer
could be made to look like the otherBagDeserializer
s fairly easily.test-bag/
andsample-bag.tar
), theBagDeserializer
s will return an incorrect path. This can be fixed before the PR is merged if desired.Testing
bag-config.yml
for export