jon1caleb2 / jmm

Automatically exported from code.google.com/p/jmm
0 stars 1 forks source link

Speed up - Hashing and moving file over network #487

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
If I am not totally wrong, JMMServer hashes the files over the network which 
probably result in the file getting stored in some random temp folder (or maybe 
even in the RAM...), and after its done hashing the file it tries to copy the 
file to the drop location - if you have configured JMMServer this way.

Windows (the SMB Protokol) will than again download the file to a temp folder 
just to upload it to the destination folder.

So in my believe it would be faster, if JMMServer figures out that its working 
with Networkdrives as source/drop location. 
Than download a copy of the file from source destination to a random temp 
folder, do the hashing locally (which should also be faster - and alot faster 
with the tempfolder being on SSD instead of a network drive) after it was 
successfully hashed upload it to drop location and delete the file in the 
source location aswell as in the temp folder.

With this method, you kill the purpose of downloading the file twice.
Once during the hashing progress and once during the move to the drop location. 
Considering movies with 4 GB plus this can be alot of time saved.

If I am totally wrong and just impatient with JMMServer hashing 5900 files 
right now, than just say so. :)

Original issue reported on code.google.com by buzzplay...@gmail.com on 25 Sep 2013 at 8:49

GoogleCodeExporter commented 8 years ago
[deleted comment]
GoogleCodeExporter commented 8 years ago
The hasher actually loads the file in chunks (and does the hashing in memory)
It is possible that copying the file locally first would improve performance 
though, so I will investigate it

Original comment by werndly...@gmail.com on 6 Aug 2014 at 12:02

GoogleCodeExporter commented 8 years ago
https://github.com/japanesemediamanager/jmmserver/issues/70

Original comment by werndly...@gmail.com on 20 Aug 2014 at 3:05