Unexpected Slow Performance and Error in is_reference(x, quote(expr = ))

[smarr]

Not sure whether this is helpful or not, but just in case, here an R script that runs rather slow: https://gist.github.com/smarr/6ba736373f46cac851d1b52e337ab753 This R script is a version of ReBenchDB's Rmarkdown report.

Here the numbers, first Ř, and then GNU R.

1: ../rir/bin/Rscript somns.min.R
            Mean                Std.Dev.    Min         Median      Max
real       41.494+/-3.5537      1.054      40.695      40.804      42.983
user       41.375+/-3.5183      1.043      40.587      40.690      42.850
sys         0.121+/-0.0358      0.011       0.111       0.116       0.136

1: Rscript somns.min.R
            Mean                Std.Dev.    Min         Median      Max
real        2.519+/-0.0185      0.005       2.513       2.519       2.526
user        2.458+/-0.0160      0.005       2.451       2.461       2.462
sys         0.064+/-0.0232      0.007       0.054       0.066       0.071

For completeness, the version details:

 ../rir/bin/Rscript --version
R scripting front-end version 3.6.2 (2019-12-12)

Rscript --version
R scripting front-end version 3.4.4 (2018-03-15)

Ř was built from git HEAD with cmake -DCMAKE_BUILD_TYPE=release .

To run it, you'll need two data files:

• https://rebench.stefan-marr.de/rebenchdb/get-exp-data/381 (TruffleSOM-381.qs)
• https://rebench.stefan-marr.de/rebenchdb/get-exp-data/380 (TruffleSOM-380.qs)

There also seems to be an issue with default arguments to functions, or at least that's what the error suggests when line 15 is not commented out.

Error in is_reference(x, quote(expr = )) :
  argument is missing, with no default
Calls: <Anonymous> ... abort -> validate_signal_args -> is_missing -> is_reference
Environment:
  1: (function (x, ...)
UseMethod("print"))(x)
  2: print.ggplot(x)
  3: ggplot_gtable(data)
  4: ggplot_gtable.ggplot_built(data)
  5: plot_theme(plot)
  6: mapply(validate_element, theme, names(theme), MoreArgs = list(element_tree
  7: (function (el, elname, element_tree)
{
    eldef <- element_tree[[elname]]
  8: abort(glue("Theme element `{elname}` is not defined in the element hierarch
  9: validate_signal_args(.subclass)
  10: is_missing(subclass)
  11: is_reference(x, quote(expr = ))

For executing the script you'll need the following libraries:

library(ggplot2)
library(ggstance)
library(qs)
library(dplyr)
library(stringr)

[smarr]

Ah, and for good measures, after commenting out the bit that breaks on the original ReBenchDB report, these are the performance numbers, first Ř, and then GNU R:

1: ./render-rir.sh TruffleSOM-380.qs TruffleSOM-381.qs
            Mean                Std.Dev.    Min         Median      Max
real        257.235+/-11.6012     3.440       254.553     255.061     262.091
user        256.573+/-11.7868     3.495       253.906     254.303     261.511
sys         0.657+/-0.2475        0.073       0.576       0.642       0.754

1: ./render.sh TruffleSOM-380.qs TruffleSOM-381.qs
            Mean                Std.Dev.    Min         Median      Max
real        83.027+/-4.3162      1.280       82.037      82.210      84.835
user        82.411+/-4.0607      1.204       81.447      81.678      84.109
sys         0.605+/-0.2860       0.085       0.531       0.559       0.724

[o-]

Thanks for the report. We are always happy to look at new benchmarks and it is well possible that you see slowdowns, when exercising parts of the language we did not look at so far. It is vast unfortunately (hundreds of builtins and specials)...

Unfortunately I can't run your script yet, it fails with:

> not_in_both <- stats %>%
+   filter(is.na(ratio)) %>%
+   droplevels()
Error: (converted from warning) Factor `commitid` contains implicit NA, consider using `forcats::fct_explicit_na`
Environment:
  1: stats %>% filter(is.na(ratio)) %>% droplevels()
  2: withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
  3: eval(quote(`_fseq`(`_lhs`)), env, env)
  4: eval(quote(`_fseq`(`_lhs`)), env, env)
  5: `_fseq`(`_lhs`)
  6: freduce(value, `_function_list`)
  7: withVisible(function_list[[k]](value))
  8: function_list[[k]](value)
  9: droplevels(.)
  10: droplevels.data.frame(.)
  11: lapply(x[ix], droplevels)
  12: x[ix]
  13: `[.grouped_df`(x, ix)
  14: grouped_df(y, group_names, group_by_drop_default(x))
  15: grouped_df_impl(data, unname(vars), drop)
  16: .signalSimpleWarning("Factor `commitid` contains implicit NA, consider usin
  17: withRestarts({
    .Internal(.signalCondition(simpleWarning(msg, call), msg
  18: withOneRestart(expr, restarts[[1]])
  19: doWithOneRestart(return(expr), restart)

[JanJecmen]

Hi Stefan, thanks! I managed to run this and observe similar timings. I will have a look.

[smarr]

Thanks @o- and @JanJecmen.

it is well possible that you see slowdowns, when exercising parts of the language we did not look at so far. It is vast unfortunately (hundreds of builtins and specials)...

Yeah, I figured :) So, this isn't a complaint. It's just me trying Ř. I think, I promised doing it already after the DLS paper last year. And, I am neither an R expert nor in anyway sure that my R code isn't at fault. It's really mostly curiosity on my part.

Unfortunately I can't run your script yet, it fails with:

Hm, perhaps the input files went the wrong way around? Not entirely sure what might be going wrong.

[o-]

So, this isn't a complaint.

no worries, just some expectation management.

Hm, perhaps the input files went the wrong way around? Not entirely sure what might be going wrong.

i reinstalled all packages and that did the trick. maybe some incompatibility with an old version or I had a broken installation of some package on machine.