Closed SebKrantz closed 1 year ago
Hi Sebastian,
Great, thank you for the hint! For the latest CRAN release, we changed a lot of code from a for
-loop to way more efficient split
-approach, so this seems like a further easy speed-up!
rsplit
does not seem to support matrices unlike base R's split
does (via its default method)?
What we do a lot is this:
# split matrix X by individual and store in list
X.col <- NCOL(X)
tX.list <- split(X, ind) # gives list of vectors
tX.list <- lapply(tX.list, function(m) matrix(m, ncol = X.col)) # transform list of vectors to list of matrices
Where the last line is somewhat annoying but necessary as split
splits out vectors and not matrices.
Hi Kevin, that's right, rsplit()
has no matrix method. I could add it for the next release though. The main reason it is not there is because I never required it. The good news is that gsplit()
has the option to keep the first argument empty (NULL
), which will return indices that can be used to subset the matrix:
library(collapse)
X = qM(mtcars)
f = qF(mtcars$cyl)
X_spl = lapply(gsplit(g = f), function(i) X[i, ])
str(X_spl)
#> List of 3
#> $ : num [1:11, 1:11] 22.8 24.4 22.8 32.4 30.4 33.9 21.5 27.3 26 30.4 ...
#> ..- attr(*, "dimnames")=List of 2
#> .. ..$ : chr [1:11] "Datsun 710" "Merc 240D" "Merc 230" "Fiat 128" ...
#> .. ..$ : chr [1:11] "mpg" "cyl" "disp" "hp" ...
#> $ : num [1:7, 1:11] 21 21 21.4 18.1 19.2 17.8 19.7 6 6 6 ...
#> ..- attr(*, "dimnames")=List of 2
#> .. ..$ : chr [1:7] "Mazda RX4" "Mazda RX4 Wag" "Hornet 4 Drive" "Valiant" ...
#> .. ..$ : chr [1:11] "mpg" "cyl" "disp" "hp" ...
#> $ : num [1:14, 1:11] 18.7 14.3 16.4 17.3 15.2 10.4 10.4 14.7 15.5 15.2 ...
#> ..- attr(*, "dimnames")=List of 2
#> .. ..$ : chr [1:14] "Hornet Sportabout" "Duster 360" "Merc 450SE" "Merc 450SL" ...
#> .. ..$ : chr [1:11] "mpg" "cyl" "disp" "hp" ...
Created on 2022-09-13 by the reprex package (v2.0.1)
Also note that 'GRP' objects are more efficient inputs for all of these functions (factors will be converted to 'GRP' objects). So if you can, use g = GRP(mtcars$cyl)
instead of qF(mtcars$cyl)
.
A matrix method would be convenient! Shall I file an issue in collpase's repository?
A minor update of collapse just hit CRAN, which includes rsplit.matrix. I had to update earlier than planned due to issues with newer C compilers now used on R-devel checks. So you can move ahead with this.
Hope to have the few remnants of base R's split()
substituted before long and will then release.
Btw: Since plm 2.6-2, we make already use of collapse::qtable
(with commit 82092f6).
Great, happy to see things moving forward.
Hi Kevin, having {collapse} as a hard dependency provides many opportunities for further drop-in performance improvements, one particularly significant one that came to mind would probably be to use the faster split functions:
rsplit()
being a recursive version ofsplit()
, that, with argumentsdrop = FALSE
andflatten = TRUE
works just likesplit()
. There is also a barebones function calledgsplit()
, which is even faster as it only works for vectors and by default does not save the names.Created on 2022-09-13 by the reprex package (v0.3.0)