Closed etiennebacher closed 1 year ago
Thanks @etiennebacher. I did know of this limitation but not in this context. @CGMossa thinks its related to https://github.com/extendr/extendr/issues/397.
Here's something interesting. The number 50000
has something to do with it.
library(h3o)
# its exactly 50000 that causes an issue
n <- 49999
pnts <- tibble::tibble(
x = runif(n, -5, 10),
y = runif(n, 40, 50)
) |>
sf::st_as_sf(
coords = c("x", "y"),
crs = 4326
)
foo <- h3_from_points(pnts$geometry, 4)
#> Error: protect(): protection stack overflow
# set to 50000
n <- 50000
pnts <- tibble::tibble(
x = runif(n, -5, 10),
y = runif(n, 40, 50)
) |>
sf::st_as_sf(
coords = c("x", "y"),
crs = 4326
)
# watch error
foo <- h3_from_points(pnts$geometry, 4)
#> Error: protect(): protection stack overflow
50000 * 2 = 10000. 50000 robjs (Doubles) coming in and 50000 robjs (externalptrs) going out its exactly 10000. Can we turn that number up to 11?
The number 50000 has something to do with it.
I don't understand why, the example also fails with 49,999
The number can change by ± 5 or so I've noticed. But pretty reliably around that limit. But only for lists I've found.
simple R package designed to illustrate the issue https://github.com/JosiahParry/stackoverflow/blob/main/example.R
Using code from this package I have boiled this down to:
code <- 'fn stack_overflow_list(x: List) -> List {
x
.into_iter()
.map(|(_, x)| {
let dbs = Doubles::from_robj(&x).unwrap();
[dbs[1].0, dbs[0].0]
})
.collect::<List>()
}'
rextendr::rust_function(code)
n <- 49984
pnts <- lapply(1:n, \(x) runif(2))
stack_overflow_list(pnts)
n <- 49993
pnts <- lapply(1:n, \(x) runif(2))
stack_overflow_list(pnts)
reprex
does not render this for me.rextendr
. Has this been reported there?Hello @rstub. This is a bug in extendR
. @JosiahParry have helped extensively with troubleshooting it. I wrote a PR for it here https://github.com/extendr/extendr/pull/540, and it got reviewed and approved.. I just need to write one more test to merge it.
This will necessitate the use of extendr
on github, in order to avoid this. I would say that you can circumvent this, if it is truly necessary.
Thanks for investigating it as well!
Thanks both! Yeah, this bothered me for quite some time. For now, the only way I am aware of to avoid it is to pre-allocate the List and assign elements in a for loop using the .elt() method.
On Mon, May 8, 2023 at 04:17 CGMossa @.***> wrote:
Hello @rstub https://github.com/rstub. This is a bug in extendR. @JosiahParry https://github.com/JosiahParry have helped extensively with troubleshooting it. I wrote a PR for it here extendr/extendr#540 https://github.com/extendr/extendr/pull/540, and it got reviewed and approved.. I just need to write one more test to merge it. This will necessitate the use of extendr on github, in order to avoid this. I would say that you can circumvent this, if it is truly necessary. Thanks for investigating it as well!
— Reply to this email directly, view it on GitHub https://github.com/JosiahParry/h3o/issues/3#issuecomment-1537947830, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADHIKLGSAEMSZDPM63MSDMTXFCTZPANCNFSM6AAAAAAVY5D5ZE . You are receiving this because you were mentioned.Message ID: @.***>
@etiennebacher can you install the new version from https://josiahparry.r-universe.dev/h3o and let me know if this is fixed?
Hi @JosiahParry, I don't have this error anymore, thanks
@etiennebacher brilliant :) thanks
Hello, I've been following a bit your repos that use
extendr
since I'm trying to learn it. I ran some of your examples with more points to see if the performance gains scale well and I came across this error inh3_from_points()
:I know it's still early in development and I'm not using these geospatial features anyway, but I just thought you'd want to know.
Session info if needed