search

animetosho / ParPar

High performance PAR2 create client for NodeJS
195 stars 21 forks source link

Generating the files takes a long time when the source size exceeds 50 GB #53

Closed BakasuraRCE closed 11 months ago

BakasuraRCE commented 1 year ago

Hello!

This is not exactly an issue but rather a question. I have the following parameter configuration, and what I do is pass a set of files (between 10 to 20) that in total add up to between 50 to 100 GB. Could you help me understand why it takes approximately 1 hour to generate the par2 files?

It is running on a 12-core CPU and 16 GB of RAM, with an M.2 disk.

parpar -s5M -r1n*0.6 --threads 14 --memory 2048M -p1l --progress stdout -q file1 file2 file3 ...

animetosho commented 1 year ago

Are you noticing that it's much faster if the source set is under 50GB?
Can you post the full output of ParPar? What version are you running?
Is your CPU or I/O at full usage? Do you have enough free RAM?