This change makes st a bit quicker for large inputs. The main boost comes from de-duplicating the validate check for every number. The validation check is also replaced with Scalar::Util's looks_like_number, which passes the existing test suite, is part of core, and is implemented in C.
This change, as it is now, would require a version bump on the library since previously, process would die on invalid input. This change assumes the caller has validated the input. It should be straightforward to avoid that if necessary.
I've included some simple runs on a 40mb file showing the before and after below. The run time for this file was reduced to 14s, originally taking 21s. Both times were taken after multiple runs of tests to account for page caches being warm etc.
Current st:
-bash-4.2$ time st --complete /tmp/input.txt
N min q1 median q3 max sum mean stddev stderrvariance
4.45998e+06 0.002401 0.00371 0.004048 0.004653 0.069798 28189.6 0.00632058 0.00852295 4.03574e-06 7.26406e-05
real 0m21.341s
user 0m20.551s
sys 0m0.790s
Proposed:
-bash-4.2$ time st --complete /tmp/input.txt
N min q1 median q3 max sum mean stddev stderrvariance
4.45998e+06 0.002401 0.00371 0.004048 0.004653 0.069798 28189.6 0.00632058 0.00852295 4.03574e-06 7.26406e-05
real 0m14.079s
user 0m13.255s
sys 0m0.823s
This change makes
st
a bit quicker for large inputs. The main boost comes from de-duplicating thevalidate
check for every number. The validation check is also replaced with Scalar::Util'slooks_like_number
, which passes the existing test suite, is part of core, and is implemented in C.This change, as it is now, would require a version bump on the library since previously,
process
would die on invalid input. This change assumes the caller has validated the input. It should be straightforward to avoid that if necessary.I've included some simple runs on a 40mb file showing the before and after below. The run time for this file was reduced to 14s, originally taking 21s. Both times were taken after multiple runs of tests to account for page caches being warm etc.
Current
st
:Proposed: