search

crkn-rcdr / cihm-metadatabus

Documentation and Docker build environment for key portions of the metadata bus.
BSD 2-Clause "Simplified" License
0 stars 1 forks source link

Add in md5 check for uploads and retry #44

Closed RussellMcOrmond closed 2 years ago

RussellMcOrmond commented 2 years ago

43

All tools need to properly check the MD5 of the file that was sent, as it can sometimes store a 0 length file.

russell@russell-XPS-13-7390:~/git/cihm-metadatabus$ find CIHM-Meta/ -type f -exec grep object_put {} /dev/null \;
CIHM-Meta/lib/CIHM/Meta/Smelter/Process.pm:        $r = $self->swiftaccess->object_put( $self->swift_access_metadata,
CIHM-Meta/lib/CIHM/Meta/Smelter/Process.pm:            die "object_put didn't return matching etag\n";
CIHM-Meta/lib/CIHM/Meta/Smelter/Process.pm:                  $self->swiftaccess->object_put( $self->swift_access_files,
CIHM-Meta/lib/CIHM/Meta/Smelter/Process.pm:                    warn "MD5 mismatch object_put("
CIHM-Meta/lib/CIHM/Meta/Dmdtask.pm:          $self->swift->object_put( $self->access_metadata, $object,
CIHM-Meta/lib/CIHM/Meta/Dmdtask.pm:            warn "object_put($object) didn't return matching etag\n";
CIHM-Meta/lib/CIHM/Meta/ImportOCR/Process.pm:                              $self->swift->object_put( $self->access_metadata,
CIHM-Meta/lib/CIHM/Meta/ImportOCR/Process.pm:"object_put $object didn't return matching etag\n";
CIHM-Meta/lib/CIHM/Meta/ImportOCR/Process.pm:                          $self->swift->object_put( $self->access_metadata,
CIHM-Meta/lib/CIHM/Meta/ImportOCR/Process.pm:"object_put $object didn't return matching etag\n";
CIHM-Meta/lib/CIHM/Meta/ImportOCR/Process.pm:                    my $putresp = $self->swift->object_put( $self->access_files,
CIHM-Meta/lib/CIHM/Meta/ImportOCR/Process.pm:                        die(    "object_put of $object returned "
CIHM-Meta/lib/CIHM/Meta/Ocrtask.pm:      $self->swift->object_put( $container, $object, $fh,
CIHM-Meta/lib/CIHM/Meta/Ocrtask.pm:        die(    "object_put of $object into $container returned "
CIHM-Meta/lib/CIHM/Meta/Ocrtask.pm:    warn "Etag mismatch during object_put of $object into $container: $md5digest != "
russell@russell-XPS-13-7390:~/git/cihm-metadatabus$