data.table: another non-exportable object.

[JamesThompsonC]

Thank you Henrik for your generosity in developing and sharing this package.

I discovered another non-exportable object. Here's the repex:

library(data.table) library(tibble) library(foreach) library(fulure.callr) library(doFuture)

doFuture::registerDoFuture() plan(callr, workers = 3)

This gives me what I'm expecting. a three row, two column data.table, each cell a data.table:

o <- foreach(ii = 1:3) %do% { data.table(df_a = list(data.table(e = 1:10)), df_b = list(data.table(d = 1:10))) }

rbindlist(o)

This returns an error:

o <- foreach(ii = 1:3) %dopar% { data.table(df_a = list(data.table(e = 1:10)), df_b = list(data.table(d = 1:10))) }

Error: Detected a non-exportable reference (‘externalptr’) in the value (of class ‘list’) of the resolved future

I got %dopar% to work by using tibble's instead of data.tables.

My problem is that most of my functions return data.table's.

o <- foreach(ii = 1:3) %dopar% { tibble(df_a = list(tibble(e = 1:10)), df_b = list(tibble(d = 1:10))) }

rbindlist(o)

I'm not sure you can do anything about it. If not, maybe include it in your "A Future for R: Non-Exportable Objects" vignette.

~Jim

[HenrikBengtsson]

Hi. I think you have:

options(future.globals.onReference = "error")

set when doing this, cf. ?future::future.options. Because without that set, the future framework will not look for "non-exportable reference" and therefore not throw an error in this case.

Section 'Packages that rely on external pointers' of the 'Non-Exportable Objects' vignette mentions data.table as a special type of objects holding "external pointers" but still can be exported (back and forth).

I hope this helps

PS. Please have a look at https://docs.github.com/en/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax on how to format your code examples here in GitHub to make them easier to read.