oils-for-unix / oils

Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!
http://www.oilshell.org/
Other
2.85k stars 159 forks source link

Calling the `get()` function without a default value can cause a performance regression #2123

Open quexxon opened 2 hours ago

quexxon commented 2 hours ago

I tried unsuccessfully to boil this down to a simpler reproducible example. Something about the interplay of get without a default value in the filter procedure appears to cause the issue. Sample code for reproduction:

#!/usr/bin/env ysh

proc filter (; predicate) {
    for line in (io.stdin) {
        if (io->evalExpr(predicate, vars={_val: fromJson8(line)})) {
            write -- $line
        }
    }
}

var f = $(mktemp)

for i in (0..=100_000) {
    json write ({}, space=0) >> $f
}

write -- u'get() with default\n'
write -- 'BEFORE ------------------------------------------------------'

for _ in (0..<3) {
    time for line in (io.stdin) {
        call fromJson8(line)
    } < $f
    write
}

filter [get(_val, 'missing-key', 0) === 0] < $f >/dev/null

write -- 'AFTER -------------------------------------------------------'

for _ in (0..<3) {
    time for line in (io.stdin) {
        call fromJson8(line)
    } < $f
    write
}

write -- u'get() without default\n'
write -- 'BEFORE ------------------------------------------------------'

for _ in (0..<3) {
    time for line in (io.stdin) {
        call fromJson8(line)
    } < $f
    write
}

filter [get(_val, 'missing-key')] < $f >/dev/null

write -- 'AFTER -------------------------------------------------------'

for _ in (0..<3) {
    time for line in (io.stdin) {
        call fromJson8(line)
    } < $f
    write
}

rm $f

Output on my machine:

ysh-0.24.0$ uname -a
Linux rodan 6.8.0-48-generic #48-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 14:04:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
ysh-0.24.0$ ./bug.ysh
get() with default

BEFORE ------------------------------------------------------
real    0.071
user    0.071
sys     0.000

real    0.070
user    0.070
sys     0.000

real    0.070
user    0.070
sys     0.000

AFTER -------------------------------------------------------
real    0.071
user    0.071
sys     0.000

real    0.070
user    0.070
sys     0.000

real    0.071
user    0.071
sys     0.000

get() without default

BEFORE ------------------------------------------------------
real    0.071
user    0.071
sys     0.000

real    0.070
user    0.070
sys     0.000

real    0.070
user    0.070
sys     0.000

AFTER -------------------------------------------------------
real    0.928
user    0.927
sys     0.001

real    0.929
user    0.928
sys     0.001

real    0.937
user    0.932
sys     0.001
quexxon commented 2 hours ago

I'm on 5a67993f