netSmooth: A Network smoothing based method for Single Cell RNA-seq

[jonathanronen]

Update the following URL to point to the GitHub repository of the package you wish to submit to Bioconductor

• Repository: https://github.com/BIMSBbioinfo/netSmooth

Confirm the following by editing each check box to '[x]'

• [x] I understand that by submitting my package to Bioconductor, the package source and all review commentary are visible to the general public.

• [x] I have read the Bioconductor Package Submission instructions. My package is consistent with the Bioconductor Package Guidelines.

• [x] I understand that a minimum requirement for package acceptance is to pass R CMD check and R CMD BiocCheck with no ERROR or WARNINGS. Passing these checks does not result in automatic acceptance. The package will then undergo a formal review and recommendations for acceptance regarding other Bioconductor standards will be addressed.

• [x] My package addresses statistical or bioinformatic issues related to the analysis and comprehension of high throughput genomic data.

• [x] I am committed to the long-term maintenance of my package. This includes monitoring the support site for issues that users may have, subscribing to the bioc-devel mailing list to stay aware of developments in the Bioconductor community, responding promptly to requests for updates from the Core team in response to changes in R or underlying software.

I am familiar with the essential aspects of Bioconductor software management, including:

• [x] The 'devel' branch for new packages and features.
• [x] The stable 'release' branch, made available every six months, for bug fixes.
• [x] Bioconductor version control using Git (optionally via GitHub).

For help with submitting your package, please subscribe and post questions to the bioc-devel mailing list.

[bioc-issue-bot]

Hi @jonathanronen

Thanks for submitting your package. We are taking a quick look at it and you will hear back from us soon.

The DESCRIPTION file for this package is:

Package: netSmooth
Type: Package
Title: Network smoothing for scRNAseq
Version: 0.1.0
Author: Jonathan Ronen [aut, cre], Altuna Akalin [aut, cre]
Maintainer: Jonathan Ronen <yablee@gmail.com>
Description: netSmooth is an R package for network smoothing of single cell RNA
    sequencing data. Using bio networks such as protein-protein interactions as
    priors for gene co-expression, netsmooth improves cell type identification
    from noisy, sparse scRNAseq data.
biocViews: Network, GraphAndNetwork, SingleCell, RNASeq, GeneExpression
License: Artistic-2.0
Encoding: UTF-8
LazyData: true
Depends:
    R (>= 3.4.2),
    scater (>= 1.6.0),
    clusterExperiment (>= 1.4.0)
Imports:
    entropy,
    SummarizedExperiment,
    SingleCellExperiment,
    Matrix,
    cluster,
    data.table,
    entropy
Suggests: knitr,
    testthat,
    Rtsne,
    biomaRt,
    igraph,
    STRINGdb,
    NMI,
    pheatmap,
    ggplot2,
    BiocStyle, 
    knitr, 
    rmarkdown
VignetteBuilder: knitr
RoxygenNote: 6.0.1

Consider adding SSH keys to your GitHub account. SSH keys will are used to control access to accepted Bioconductor packages. See these instructions to add SSH keys to your GitHub account.

[mtmorgan]

Before your package can be added to the review queue, the vignette code chunks need to be 'live' (i.e., no eval = FALSE). Please update your repository, bump the package version (to 0.99.1) and confirm that the updated package conforms to Bioconductor guidelines for time and space. Please post a comment here when your package has been updated.

[jonathanronen]

Thanks for the quick look, @mtmorgan. We made the vignette like this in order to speed up the check. This must be somewhat common in the bioconductor universe, as many packages have extensive vignettes which I find hard to imagine run in under 5 minutes. For instance, the DESeq2 vignette has a bunch of them 1, as does the methylKit vignette 2.

In this package as well, dealing with single cell rnaseq means the data matrices will be of certain size. Our vignette already uses a reduced size dataset, and reducing it further might make demonstrating some of the concepts of the package more challenging.

Would it be possible to reconsider this request?

[mtmorgan]

Your package must have the majority of code chunks (e.g., 80%) evaluated. Usually this involves identifying a realistic but trimmed data set that illustrates functionality in a timely fashion. That other packages no longer meet this requirement isn't justification for starting off on the wrong foot.

[jonathanronen]

@mtmorgan Appologies for the hiatus. Requested changes have been made. Please have a look. Thanks! J.

[bioc-issue-bot]

Your package has been approved for building. Your package is now submitted to our queue.

IMPORTANT: Please read the instructions for setting up a push hook on your repository, or further changes to your repository will NOT trigger a new build.

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "skipped, ERROR". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

d17740a netSmooth to accept sparse matrix input 3cd6ae1 Support scater 1.7 (for next bioc release)

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "ERROR". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

4f8bf08 Bump version numbers for bioc submission

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "ERROR, WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

6020206 bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "skipped, ERROR". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[mtmorgan]

Please add a version bump and I'll review the package promptly.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

240e5e4 Chase after the evolving clusterExperiment API 18fcd00 Add runnable example to robustClusters doc f1b05db Add stats to imports in DESCRIPTION (requested by ... b5ba959 1:k -> seq_len(k) (requested by BiocCheck) cae9765 plotPCA with draw_plot=FALSE --> runPCA (changes i... c115f77 bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "ERROR". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

9b0785d Fix imports from scater (ommission from cae976515f... 7be7393 bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

cd702b4 Add return values to documentation (requested by B... 653f6d2 Bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

Congratulations! The package built without errors or warnings on all platforms.

Please see the build report for more details.

[mtmorgan]

DESCRIPTION

• Use Authors@R, rather than Author and Maintainer fields

NAMESPACE

• looks good

vignettes

• remove buildingPPIsFromStringDB.html and netSmoothIntro.html; these are created during the build process.
• Please spell-check, e.g., buildingPPIsFromStringDB.Rmd:21 'databse' --> database

R

• To make your code more robust, use stopifnot() or similar to check that each incoming argument matches expectation, e.g., netSmooth.R assert that adjMatrix is a matrix, of numeric value, with appropriate dimensions.

• use message() rather than cat() to communicate with users (everywhere but show() methods). No need to message(paste(...)), usually message(...) is sufficient (also for stop(), warning()).

• use BiocParallel::bplapply() rather the mclapply(), making sure your code works when using SnowParam() (separate processes, rather than processes sharing memory).

• Adopt a code formatting style that allows for longer lines and greater readability, e.g., using a consistent indent of 4 spaces

  scores <- unlist(mclapply(seq_len(length(smoothed.expression.matrices)),
                                    function(i) {
                                        x.sm <-
                                            smoothed.expression.matrices[[i]]
                scoreSmoothing(x=x.sm,
                               method=autoAlphaMethod,
                               is.counts=is.counts,
                               dimReduceFlavor=autoAlphaDimReduceFlavor, ...)
                                    },
                                    mc.cores=numcores))

  might become

  scores <- unlist(mclapply(
      seq_len(length(smoothed.expression.matrices)),
      function(i) {
          x.sm <- smoothed.expression.matrices[[i]]
          scoreSmoothing(
              x=x.sm, method=autoAlphaMethod, is.counts=is.counts,
              dimReduceFlavor=autoAlphaDimReduceFlavor, ...
          )
      }, mc.cores=numcores
  ))

• structure your code to test and fail early, reducing the need for complicated if () statements deeper into the code, e.g.,

  stopifnot(is.numeric(alpha) || alpha == "auto")

• use the @keyword internal to mark functions that are not ment for end users.

• prefer vectorized functions over iterations wherever possible, e.g., clusterExperimentWorkflow.R:58 can be constructed in vectorized calls outside the loop.

man

• provide more detail on how each data set (e.g., human.ppi) is obtained -- enough so that the curious user could reproduce the data themselves. I realize there are scripts in ./data-raw; the Bioconductor convention would have these as scripts in the ./inst/data-raw directory, so that the scripts are available to the end user in the installed package; refer to these scripts in the man page, e.g., system.file(package="netSmooth", "data-raw")

[mtmorgan]

Please respond within two weeks.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

14ff080 use Authors@R instead of Author and Maintainer fie... e2ddc5e Remove generated vignette htmls 301be12 Spellcheck vignettes 76c60ba Use message() instead of cat() to communicate with... 9628381 add a couple of stopifnot()'s cf06090 Add @keywords internal to clusterExperiment workfl... 2b083cd Use bplapplyl instead of mclapply 226ea45 Change indentations to multiples of 4 to comply wi... 3b13811 Use seq_len instead of 1:maxiter here 40b1d9d Add inst/data-raw folder for script that shows how... 8e2bee0 Bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "ERROR". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

592c5bf forgot to update the documentation 196470e Maybe this will fix the build on bioconductor's wi... 3550256 Bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "ABNORMAL". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[jonathanronen]

@mtmorgan thanks for the input! i've done the changes you suggested.

right now it seems the windows build is failing - this is following your request that I use BiocParallel::bplapply instead of mclapply. Is this a problem with the windows build machine, or is there some other windows-specific configuration which is not well documented in BiocParallel?

[mtmorgan]

Windows uses separate, completely independent, processes that do not share memory etc., whereas Linux / Mac do share memory. Likely what happens (I have not investigated closely) is that your function references symbols or data that are not available in the new process. You can likely see the problem on Linux / Mac by using a 'SnowParam()', e.g., register(SnowParam()). The solution is to write the bplapply functions so that they are provided, through arguments, with all the relevant data. If you point me to a simple reproducible example I can provide more concrete help.

[jonathanronen]

That can't be it right now. The test that fails on the windows machine passes the linux/mac machines, and uses a SnowParam. Here's the default parameter: https://github.com/BIMSBbioinfo/netSmooth/blob/master/R/netSmooth.R#L59

And the test which fails calls this function.

Any ideas?

[mtmorgan]

I looked at your code a little bit.

Three things about BiocParallel. You need to either register() the BPPARAM or pass BPPARAM as an argument to bplapply. For single thread execution, its more efficient to use SerialParam(), otherwise, you'll pay the cost of loading your package on each worker, and of transmitting large data via socket connections to the workers. If you're going to run multiple bplappy() in a single function call, then it's better to start the cluster at the top of the function, instead of (implicitly) on each call to bplappy(). So for your code

bpparam = BiocParallel::SerialParam()
...
if (!BiocParallel::bpisup(bpparam)) {
    bpstart(bpparam)
    on.exit(bpstop(bpparam), add=TRUE
}
...
smoothed.expression.matrices <- BiocParallel::bplapply(
                autoAlphaRange,
                function(a) {
                    smoothAndRecombine(x, adjMatrix, a,
                        normalizeAdjMatrix=normalizeAdjMatrix)
                }
            , BPPARAM = bpparam)

(oops, sorry, didn't mean to close the issue).

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

76b2b0f Use SerialParam as default instead of SnowParam; m... a2d3da6 Bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[jonathanronen]

Thanks for the tip, @mtmorgan ! Are we all set?

[mtmorgan]

Looks like there are addressable warnings

* checking dependencies in R code ... NOTE
package 'methods' is used but not declared

Use Imports: methods in the DESCRIPTION file, importing all or part of it in the NAMESPACE.

* checking R code for possible problems ... [17s] NOTE
netSmooth,matrix: no visible global function definition for 'bpstart'
netSmooth,matrix: no visible global function definition for 'bpstop'
Undefined global functions or variables:
  bpstart bpstop

These should be resolved BiocParallel::bpstart() or importFrom(BiocParallel, bpstart, bpstop) in the NAMESPACE. BiocParallel should be in the Suggests: field of the DESCRIPTION file (for BiocParallel:: solution) or Imports: (for importFrom(...) soluion).

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

5216077 Add BiocParallel to Imports: in DESCRIPTION; expli... ce81a99 Bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

bd4d6a6 Add methods to Imports: field; bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[mtmorgan]

Still seem like there are warnings that can be addressed

* checking DESCRIPTION meta-information ... NOTE
Package listed in more than one of Depends, Imports, Suggests, Enhances:
  knitr
A package should be listed in only one of these fields.

List it only in Suggests:

* checking top-level files ... NOTE
File
  LICENSE
is not mentioned in the DESCRIPTION file.

The DESCRIPTION file says that the package is licensed under Artistic-2.0. The included LICENSE file is GPL-3. Please choose a single license. Either way, R contains copies of the Artistic-2.0 and GPL-3 license, so a LICENSE file is not required.

* checking dependencies in R code ... NOTE
Namespace in Imports field not imported from: methods
  All declared Imports should be used.

The DESCRIPTION file has Imports: methods but the NAMESPACE does not mention the methods package. If you use functions from the methods package, add import(methods) to the NAMESPACE file.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

682ee06 remove duplicate knitr in Suggests: ; correct lice... e3118ad Bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[bioc-issue-bot]

Received a valid push; starting a build. Commits are:

ff922f8 Add recommended biocviews 80ce8be Remove LICENSE file as licence is specified in DES... b8d6caa Bump version number

[bioc-issue-bot]

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details.

[jonathanronen]

thanks again, @mtmorgan . hope that was OK to meet the deadline!

[bioc-issue-bot]

Your package has been accepted. It will be added to the Bioconductor Git repository and nightly builds. Additional information will be posed to this issue in the next several days.

Thank you for contributing to Bioconductor!

[mtmorgan]

The master branch of your GitHub repository has been added to Bioconductor's git repository.

To use the git.bioconductor.org repository, we need an 'ssh' key to associate with your github user name. If your GitHub account already has ssh public keys (https://github.com/jonathanronen.keys is not empty), then no further steps are required. Otherwise, do the following:

1. Add an SSH key to your github account
2. Submit your SSH key to Bioconductor

See further instructions at

https://bioconductor.org/developers/how-to/git/

for working with this repository. See especially

https://bioconductor.org/developers/how-to/git/new-package-workflow/ https://bioconductor.org/developers/how-to/git/sync-existing-repositories/

to keep your GitHub and Bioconductor repositories in sync.

Your package will be included in the next nigthly 'devel' build (check-out from git at about 6 pm Eastern; build completion around 2pm Eastern the next day) at

https://bioconductor.org/checkResults/

(Builds sometimes fail, so ensure that the date stamps on the main landing page are consistent with the addition of your package). Once the package builds successfully, you package will be available for download in the 'Devel' version of Bioconductor using biocLite(\"netSmooth\"). The package 'landing page' will be created at

https://bioconductor.org/packages/netSmooth

If you have any questions, please contact the bioc-devel mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel); this issue will not be monitored further.