SCArray

[zhengxwen]

Update the following URL to point to the GitHub repository of the package you wish to submit to Bioconductor

• Repository: https://github.com/zhengxwen/SCArray

Confirm the following by editing each check box to '[x]'

• [x] I understand that by submitting my package to Bioconductor, the package source and all review commentary are visible to the general public.

• [x] I have read the Bioconductor Package Submission instructions. My package is consistent with the Bioconductor Package Guidelines.

• [x] I understand that a minimum requirement for package acceptance is to pass R CMD check and R CMD BiocCheck with no ERROR or WARNINGS. Passing these checks does not result in automatic acceptance. The package will then undergo a formal review and recommendations for acceptance regarding other Bioconductor standards will be addressed.

• [x] My package addresses statistical or bioinformatic issues related to the analysis and comprehension of high throughput genomic data.

• [x] I am committed to the long-term maintenance of my package. This includes monitoring the support site for issues that users may have, subscribing to the bioc-devel mailing list to stay aware of developments in the Bioconductor community, responding promptly to requests for updates from the Core team in response to changes in R or underlying software.

• [x] I am familiar with the Bioconductor code of conduct and agree to abide by it.

I am familiar with the essential aspects of Bioconductor software management, including:

• [x] The 'devel' branch for new packages and features.
• [x] The stable 'release' branch, made available every six months, for bug fixes.
• [x] Bioconductor version control using Git (optionally via GitHub).

For help with submitting your package, please subscribe and post questions to the bioc-devel mailing list.

[bioc-issue-bot]

Hi @zhengxwen

Thanks for submitting your package. We are taking a quick look at it and you will hear back from us soon.

The DESCRIPTION file for this package is:

Package: SCArray
Type: Package
Title: Large-scale single-cell RNA-seq data manipulation with GDS files
Version: 0.99.0
Date: 2020-12-12
Depends: R (>= 4.0.0), gdsfmt (>= 1.24.0), methods, DelayedArray
Imports: BiocGenerics, S4Vectors, IRanges, utils, SummarizedExperiment,
        SingleCellExperiment
Suggests: parallel, Matrix, DelayedMatrixStats, scater, crayon, RUnit, knitr
Authors@R: c(person("Xiuwen", "Zheng", role=c("aut", "cre"),
        email="xiuwen.zheng@abbvie.com",
        comment=c(ORCID="0000-0002-1390-0708")))
Description: Provides large-scale single-cell RNA-seq data manipulation using
        Genomic Data Structure (GDS) files. It combines dense and sparse
        matrices stored in GDS files and the Bioconductor infrastructure
        framework (SingleCellExperiment and DelayedArray) to provide
        out-of-memory data storage and large-scale manipulation using the R
        programming language.
License: GPL-3
VignetteBuilder: knitr
ByteCompile: TRUE
URL: https://github.com/AbbVie-ComputationalGenomics/SCArray
biocViews: Infrastructure, DataRepresentation, SingleCell, RNASeq

[LTLA]

Some comments in passing:

• Seems like you'll want to implement extract_sparse_array() as well, if you want to take advantage of sparsity.
• We already have a GDSArray class; perhaps there is some potential for consolidation, to avoid duplication of effort.

[bioc-issue-bot]

A reviewer has been assigned to your package. Learn what to expect during the review process.

IMPORTANT: Please read this documentation for setting up remotes to push to git.bioconductor.org. It is required to push a version bump to git.bioconductor.org to trigger a new build.

Bioconductor utilized your github ssh-keys for git.bioconductor.org access. To manage keys and future access you may want to active your Bioconductor Git Credentials Account

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/SCArray to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

[mtmorgan]

@zhengxwen will you implement the sparsity method mentioned in https://github.com/Bioconductor/Contributions/issues/1809#issuecomment-744166371 ?

[LTLA]

What I want to know is: what value does SCArray provide over GDSArray? It seems like the DelayedArray-related work in this package could be consolidated into GDSArray so that all applications using GDS files can benefit, even those that aren't single-cell-related. Then SCArray could provide single-cell-specific functionality on top of the more general GDSArray.

[zhengxwen]

1) I will implement the sparsity method extract_sparse_array(), and submit a new one.

2) The current GDSArray heavily relies on the SNPRelate and SeqArray packages (the two package I wrote) (e.g., see the function .get_gds_dimnames in the GDSArray. It exclusively supports the GWAS SNP and Sequencing variant GDS files, however GDS is more general for a data container. I have contacted with Qian earlier this year about the generalization of GDSArray (but no improvement so far).

The current SCArray only contains a lightweight wrapper of DelayedArray, and uses the latest feature of the gdsfmt package (i.e., sparse matrix read/write). SCArray defines the data variables and structure needed for SingleCellExperiment. In addition, GDSArray still needs to be improved according to its efficiency.

[bioc-issue-bot]

Received a valid push on git.bioconductor.org; starting a build for commit id: 6d5db6129df7aa392048f9425397ed00c0432dac

[zhengxwen]

A new submission SCArray_0.99.1 (commit id 6d5db6129df7aa392048f9425397ed00c0432dac):

1. add extract_sparse_array()
2. fix warnings

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

Congratulations! The package built without errors or warnings on all platforms.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/SCArray to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

[mtmorgan]

This seems like a very useful integration of the GDS and DelayedArray frameworks. I have a few minor comments.

DESCRIPTION, etc

• Review the Suggests: field; are these packages all used in the vignettes / examples?

vignettes

• SCArray.Rmd:41 please add {R, eval = FALSE} to the installation code chunk.

• please provide text to introduce and interpret the output of each code chunk.

R (comments on specific lines may require changes throughout)

• AllClasses.R:58 consider developing simple (internal, if appropriate) accessors to always avoid slot access, including in package code

  .filename <- function(x) x@filename
  ...
      "File: ", .filename(x), "\n",

• AllClasses.R:68 for robust code, validate all user inputs, e.g.,

  stopifnot(is.???(index), !anyNA(index))

• scarray_main.r:44 .cfunction and .cfunction2 are not used, delete?

• scarray_main.R:112 the use of stop(sprintf(... seems unnecessary;

  stop(
      "FileVersion '", as.character(version),
      "' (should be v1.0), consider updating SCArray"
  )

[bioc-issue-bot]

Received a valid push on git.bioconductor.org; starting a build for commit id: 42116685a90c3322093398e4eb46b39151c3cafb

[zhengxwen]

• Review the Suggests: field: update the package list
• SCArray.Rmd:41 please add {R, eval = FALSE}: fixed
• please provide text to introduce and interpret the output of each code chunk: added more description
• R (comments on specific lines may require changes throughout): add more comments
• AllClasses.R:58 consider developing simple (internal, if appropriate) accessors to always avoid slot access: add .filename() and others
• AllClasses.R:68 for robust code, validate all user inputs: add stopifnot(is.list(index), !anyNA(index))
• scarray_main.r:44 .cfunction and .cfunction2 are not used: removed
• scarray_main.R:112 the use of stop(sprintf(... seems unnecessary: removed

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

Congratulations! The package built without errors or warnings on all platforms.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/SCArray to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

[bioc-issue-bot]

Your package has been accepted. It will be added to the Bioconductor nightly builds.

Thank you for contributing to Bioconductor!

[mtmorgan]

The master branch of your GitHub repository has been added to Bioconductor's git repository.

To use the git.bioconductor.org repository, we need an 'ssh' key to associate with your github user name. If your GitHub account already has ssh public keys (https://github.com/zhengxwen.keys is not empty), then no further steps are required. Otherwise, do the following:

1. Add an SSH key to your github account
2. Submit your SSH key to Bioconductor

See further instructions at

https://bioconductor.org/developers/how-to/git/

for working with this repository. See especially

https://bioconductor.org/developers/how-to/git/new-package-workflow/ https://bioconductor.org/developers/how-to/git/sync-existing-repositories/

to keep your GitHub and Bioconductor repositories in sync.

Your package will be included in the next nigthly 'devel' build (check-out from git at about 6 pm Eastern; build completion around 2pm Eastern the next day) at

https://bioconductor.org/checkResults/

(Builds sometimes fail, so ensure that the date stamps on the main landing page are consistent with the addition of your package). Once the package builds successfully, you package will be available for download in the 'Devel' version of Bioconductor using BiocManager::install("SCArray"). The package 'landing page' will be created at

https://bioconductor.org/packages/SCArray

If you have any questions, please contact the bioc-devel mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel); this issue will not be monitored further.

[Liubuntu]

1. I will implement the sparsity method extract_sparse_array(), and submit a new one.
2. The current GDSArray heavily relies on the SNPRelate and SeqArray packages (the two package I wrote) (e.g., see the function .get_gds_dimnames in the GDSArray. It exclusively supports the GWAS SNP and Sequencing variant GDS files, however GDS is more general for a data container. I have contacted with Qian earlier this year about the generalization of GDSArray (but no improvement so far).

The current SCArray only contains a lightweight wrapper of DelayedArray, and uses the latest feature of the gdsfmt package (i.e., sparse matrix read/write). SCArray defines the data variables and structure needed for SingleCellExperiment. In addition, GDSArray still needs to be improved according to its efficiency.

Hi @zhengxwen Good work! I am actually up now for updating the GDSArray to accommodate more general form of gds files. Please feel free to make PR as to the efficiency and other factors. I am happy to add you to the package authors (or possibly maintainer later to accommodate with any future gdsfmt updates).