What is the reasoning for -N?

[ole-tange]

In the discussion of #125 -N has come up.

In general I do not understand why lrzip has -N. Why not just let the user use nice if he feels it needs to be niced, and leave it out if he feels running lrzip is more important? Where can I read about the reasoning behind implementing -N and the reasoning behind the default value?

[ckolivas]

The coder- me - is a CPU scheduler hacker for the Linux kernel and chose sane defaults for nice levels since lrzip is extremely resource intensive but I liked the idea of letting users choose.

On Tue, 29 Oct 2019, 19:13 Ole Tange, notifications@github.com wrote:

In the discussion of #125 https://github.com/ckolivas/lrzip/issues/125 -N has come up.

In general I do not understand why lrzip has -N. Why not just let the user use nice if he feels it needs to be niced, and leave it out if he feels running lrzip is more important? Where can I read about the reasoning behind implementing -N and the reasoning behind the default value?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ckolivas/lrzip/issues/130?email_source=notifications&email_token=AABYWONYYD2GMRH4CUWBZMTQQ7WDDA5CNFSM4JGE5RO2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HU7JMLA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYWOMSUAGX2LZISHFVHBLQQ7WDDANCNFSM4JGE5ROQ .

[pete4abw]

I should add that if you choose -N0 or even -N[1-20] you will see speed improvements but at the expense of other running processed. $ lrzip -vf kernel.tar The following options are in effect for this COMPRESSION. Threading is ENABLED. Number of CPUs detected: 8 Detected 8341676032 bytes ram Compression level 7 Nice Value: 19 Show Progress Verbose Overwrite Files Temporary Directory set as: ./ Compression mode is: LZMA. LZO Compressibility testing enabled Heuristically Computed Compression Window: 53 = 5300MB Output filename is: kernel.tar.lrz File size: 1968363520 Will take 1 pass Beginning rzip pre-processing phase kernel.tar - Compression Ratio: 20.286. Average Compression Speed: 17.380MB/s. Total time: 00:01:47.86

With Nice = 0 kernel.tar - Compression Ratio: 20.286. Average Compression Speed: 20.856MB/s. Total time: 00:01:30.22

With Nice = -20 kernel.tar - Compression Ratio: 20.286. Average Compression Speed: 21.330MB/s. Total time: 00:01:28.38 A negligible improvement.

Interestingly, when setting a nice value of -20, warnings appear Warning, unable to set nice value on thread

A review of stream.c shows these lines as the offender.

if (unlikely(setpriority(PRIO_PROCESS, 0, control->nice_val) == -1))
               print_err("Warning, unable to set nice value on thread\n");

In main.c. the main process priority is set here:

       if (control->nice_val > 0 && !NO_COMPRESS) {
                if (unlikely(setpriority(PRIO_PROCESS, 0, control->nice_val / 2) == -1))
                        print_err("Warning, unable to set nice value\n");
        } else {
                if (unlikely(setpriority(PRIO_PROCESS, 0, control->nice_val) == -1))
                        print_err("Warning, unable to set nice value\n");
        }

I'm thinking the control->nice_val > 0 may be the culprit, but some testing will be required.

Perhaps, this modification may solve it?

       if (!NO_COMPRESS) {
                if (unlikely(setpriority(PRIO_PROCESS, 0, control->nice_val / 2) == -1))
                        print_err("Warning, unable to set nice value\n");
        } else {
                if (unlikely(setpriority(PRIO_PROCESS, 0, control->nice_val) == -1))
                        print_err("Warning, unable to set nice value\n");
        }

[pete4abw]

I should have realized. Setting a nice value < 0 as a user may not work at all. Running lrzip as root is the only way I see to assure that niceness using a higher scheduling priority will work.

In the case of the above example, running lrzip as root, with niceness as -20, the total compression time was: kernel.tar - Compression Ratio: 20.286. Average Compression Speed: 23.462MB/s. Total time: 00:01:20.17

Which is not insignificant. But beware, CPU temps may increase, and other system/user processes may slow.

[pete4abw]

See #136 . Niceness handling improved.