futureverse / future.apply

:rocket: R package: future.apply - Apply Function to Elements in Parallel using Futures
https://future.apply.futureverse.org
211 stars 16 forks source link

IDEA: Add argument 'future.label' for producing future labels #15

Closed HenrikBengtsson closed 4 years ago

HenrikBengtsson commented 6 years ago

(Cut'n'pasting some old notes-to-self of mine):

Future labels are informative in some case, e.g. users of future.batchtools who will be able to see them in the job queue. The same may be true when we get a framework for progress updates in place (e.g. Issue https://github.com/HenrikBengtsson/future/issues/172).

Current, the future.apply package does not provide a way to control future labels. Exactly how this could/should be done is not obvious - we don't want the same label for everything. Here's some thoughts/sketches:

Add argument 'future.label' for producing future labels, e.g.

x <- 1:10
names(x) <- letters[seq_along(x)]

## Labels: names(x), if they exist, otherwise seq_along(x)
y <- future_lapply(x, FUN = function(x) x^2, future.label = TRUE)

## Labels: future-<names[1]>, future-<names[2]>, future-<names[3]>, ...
y <- future_lapply(x, FUN = function(x) x^2, future.label = "future-%s")

## Function of mapply(seq_along(x), names(x), FUN = function(idx, name) ...)
y <- future_lapply(1:10, FUN = function(x) x^2, future.label = function(idx, name) sprintf("future-%d-%s", idx, name))

How should the above work when multiple elements are processed per chunk? Pass vector of indices and names, e.g. future.label = function(idxs, names) { ... })?

UPDATE 2018-07-04: Add comment on chunks.