search

vanstyn / Catalyst-Controller-SimpleCAS

General-purpose content-addressed storage (CAS) for Catalyst
0 stars 4 forks source link

add($filename) moves file instead of copying #8

Open adherzog opened 3 years ago

adherzog commented 3 years ago

When calling $cas->add($filename), on a system where a filesystem link cannot be established, the CAS is incorrectly (or, at least, unexpectedly) moving the source file into the CAS instead of copying it.

See: https://github.com/vanstyn/Catalyst-Controller-SimpleCAS/blob/master/lib/Catalyst/Controller/SimpleCAS/Store/File.pm#L59

vanstyn commented 3 years ago

Hmmm. I believe the reason it is like this is for performance, assuming that in almost all cases the file in question is a temp file that just got created from an http upload. And, if this is a large file, this could be a non-trivial performance hit, of essentially having to do double the the disk writes. That said, I agree that conceptually this is incorrect. May I ask your use-case where this came up?

adherzog commented 3 years ago

There's a separate method, add_content_file_mv(), which does what you describe, and seems to be the preferred method for the various upload methods where you're importing a temp file.

In any case, since add_content_file() will sometimes copy (link) the file and sometimes move the file, the behavior is inconsistent.

The specific use case is migrating many, many files from an external CDN into the local CAS. So I have a script that processes the files from the system, calling $cas->add($filename) to get them into the CAS, then updating some DB links appropriately, and finally moving the original file to a separate directory on the filesystem for archiving. Essentially, I want the original file to stick around, just in case something goes wrong or we need to reimport it.

In the meantime, I've gotten around the issue by supplying the actual content to ->add() instead of the filename.