search

samvera / hydra-works

A ruby gem implementation of the PCDM Works domain model based on the Samvera software stack
Other
24 stars 14 forks source link

Out of Memory issue for large files during virus check #311

Closed carolyncole closed 7 years ago

carolyncole commented 7 years ago

This code cuases an out of memory error for large files since it reads the file entirely into memory and then uploads writes it to disk: https://github.com/projecthydra/hydra-works/blob/master/lib/hydra/works/services/virus_checker_service.rb#L33-L34

escowles commented 7 years ago

Maybe we should pull CC:WorkingDirectory down into HW so we can use the same logic for all the times we want to have a local copy of a file? In addition to this issue, this code would also fail to take advantage of a locally-cached file, and create a duplicate local copy.

carolyncole commented 7 years ago

@escowles I tried just pulling working directory in, but it started to get really complicated based on the Rails dependencies, since hydra-works in not a rails project. I also noted that the working directory code was not going to be any better due to this comment: https://github.com/projecthydra/curation_concerns/blob/master/app/services/curation_concerns/working_directory.rb#L32