Bioconductor / Contributions

Contribute Packages to Bioconductor
134 stars 33 forks source link

HiCExperiment #2879

Closed js2264 closed 1 year ago

js2264 commented 1 year ago

Update the following URL to point to the GitHub repository of the package you wish to submit to Bioconductor

Confirm the following by editing each check box to '[x]'

I am familiar with the essential aspects of Bioconductor software management, including:

For questions/help about the submission process, including questions about the output of the automatic reports generated by the SPB (Single Package Builder), please use the #package-submission channel of our Community Slack. Follow the link on the home page of the Bioconductor website to sign up.

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "ERROR". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/HiCool to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

bioc-issue-bot commented 1 year ago

Received a valid push on git.bioconductor.org; starting a build for commit id: 6c1943eb35532ac3660a5cde0dc188b9ff8f1371

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/HiCool to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

bioc-issue-bot commented 1 year ago

Received a valid push on git.bioconductor.org; starting a build for commit id: 8caa5d336a258131c823830d69d7863b654e3e77

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/HiCool to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

js2264 commented 1 year ago

@lshep,

Thanks for letting my 2 extra packages run on the builder. I have now fixed things and:

Also, should @DarioS review these packages, or someone else take over? Cheers, J

DarioS commented 1 year ago

I will continue the review process. One of the Bioconductor employees might know how to solve the Python-related timeout.

vjcitn commented 1 year ago

With every version bump, all python infrastructure is re-downloaded and installed. I think there is no desirable workaround at the basilisk/client end. However if @lshep would manually trigger a rebuild, the installed python infrastructure should be available -- it goes to .cache/R, as long as that is not purged with every build it should be reusable.

lshep commented 1 year ago

I'll try a manual trigger now to see how it changes --

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

Congratulations! The package built without errors or warnings on all platforms.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/HiCool to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

js2264 commented 1 year ago

@DarioS gentle ping: is there anything you would need for the review of HiCool and fourDNData? Cheers, J

DarioS commented 1 year ago

Just more free time. I have got seven packages to review and university administrative work, too. I should do it next week.

js2264 commented 1 year ago

AdditionalPackage: https://github.com/js2264/DNAZooData

bioc-issue-bot commented 1 year ago

Hi @js2264, Thanks for submitting your additional package: https://github.com/js2264/DNAZooData. We are taking a quick look at it and you will hear back from us soon.

js2264 commented 1 year ago

Hi @DarioS @lshep 👋,

Just to let you know I have added (yet) another AdditionalPackage (DNAZooData). It is also a ExperimentData package, providing programmatic access to HiC and genome references generated by the DNA Zoo consortium. It operates the same way than the fourDNData package, by fetching metadata from the consortium database, caching the HiC contact matrix file and creating a HicFile (BiocFile variant) to be imported in R with HiCExperiment::import(). Sorry for the extra burden, and please let me know if I should rather submit it as a separate issue. Cheers, J

DarioS commented 1 year ago

Additional Packages Review

HiCool

The parameter descriptions are good and not single words like previously in HiCExperiment. Checks don't have any warnings or errors. Parameters could use more explanation. For example

#' @param balancing_args Balancing arguments for cooler (Default: " --cis-only --min-nnz 3 --mad-max 7 ")

What are all of the valid parameter names? Are there more than just --cis-only, --min-nnz, --mad-max acceptable? A link to some list of valid options somehere would be desirable.

The package now has a vignette but it only has one code chunk and it is a template rather than executable code. Although this package does some intensive processing, I think it could be demonstrated on real data, perhaps subset to chr22, a small chromosome.

One of the outputs produced is "several diagnostic plots". What quality control does each one show and how should the package user interpret the graphs?

fourDNDdata

R CMD check ERROR on all operating systems. Vignette is well-written and shows how to programatically download a data set from the consortium.

DNAZooData

Not yet accepted for review, but it looks similar in spirit to fourDNDdata. The parameters are not similar, however, but I wonder if downloading from both consortiums could be merged into a single package. Currently, both packages consist of just three R files, so even after merging, the package would not be big.

bioc-issue-bot commented 1 year ago

Received a valid push on git.bioconductor.org; starting a build for commit id: 8d7f8de30136b40d34faf6a83033ad1b9e774cc6

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/HiCExperiment to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

bioc-issue-bot commented 1 year ago

Received a valid push on git.bioconductor.org; starting a build for commit id: b492acdec1af80b2f5500620fedb8e3058500b3d

bioc-issue-bot commented 1 year ago

Received a valid push on git.bioconductor.org; starting a build for commit id: 386d05faf8e982266c5e611741ff04d8f3b54f19

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

Congratulations! The package built without errors or warnings on all platforms.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/fourDNData to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

js2264 commented 1 year ago

Thanks for this extra round of reviews @DarioS.

I have made some internal changes and contribution changes in HiCExperiment package and forgot to push it to Bioc, so perhaps the fourDNData-related CHECK errors you are seeing are due to a mismatch in versions. I have now bumped HiCExperiment to 0.99.10 and pushed it to Bioc and build/check was ok (except for the previously reported check > 10min only in mac builder). I have also bumped fourDNData to 0.99.3, pushed it to Bioc and build/check is ok. I have also bumped HiCool to 0.99.4 and pushed it to Bioc and build/check was ok (except for time limit due to building the basilisk env. only in mac builder as previously reported).

Regarding documentation changes:

I have now clarified this in HiCool doc and in the vignette.

I have added an executable chunk in the vignette, to demonstrate real use (on a subset of yeast fastq files)

I have added a paragraph re: this in the vignette.

Regarding fourDNData:

Could you please provide the check results? Bioc SPB builds/checks fourDNData without errors, perhaps yours are due to a mismatch in versions. I've bumped all packages up, would be worth updating all of them.

Regarding DNAZoo:

This is because the two consortia have very different strategies to label data, different associated metadata and different types of files. E.g. 4DN consortium can provide contact matrices (.cool files), but also automatically processed .bigwig files, or .bed files, ... matching a single dataset. One may wish to recover all of these files together or only a subset of them, hence the added type argument in fourDNData. On the other hand, DNA Zoo only provides a contact map per species.

It is feasible, but I don't think it would be a valuable improvement. The 2 consortia do provide contact matrices, but processing workflows are vastly different. 4DN processes data from published experiments (from a 20+ different labs) on known genome references (90% human or mouse). DNA Zoo is a project led by a single lab (working in collaboration with others of course), and aims to correct/improve genome assemblies of non-model organisms. So the contact map is the result of a large amount of upstream work to re-assemble an original genome reference into an improved reference. IMO, providing a gateway to the two types of contact matrices in the same package would feel unnatural. Let me know what you think based on this answer.

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

On one or more platforms, the build results were: "WARNINGS". This may mean there is a problem with the package that you need to fix. Or it may mean that there is a problem with the build system itself.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/HiCool to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

js2264 commented 1 year ago

Hi @DarioS @lshep 👋,

Just to let you know I have added (yet) another AdditionalPackage (DNAZooData). It is also a ExperimentData package, providing programmatic access to HiC and genome references generated by the DNA Zoo consortium. It operates the same way than the fourDNData package, by fetching metadata from the consortium database, caching the HiC contact matrix file and creating a HicFile (BiocFile variant) to be imported in R with HiCExperiment::import(). Sorry for the extra burden, and please let me know if I should rather submit it as a separate issue.

Gentle nudge following my replies to @DarioS review, to ask whether the AdditionalPackage DNAZooData could be added to the the build report? As @DarioS said, it follows the exact same structure than fourDNData which has already be added to the builds. Thanks! J

js2264 commented 1 year ago

Hi @lshep, sorry for being pushy, but would it be possible to add DNAZooData to the build system? Many thanks for your help, J

bioc-issue-bot commented 1 year ago

Additional Package has been approved for building.

IMPORTANT: Please read this documentation for setting up remotes to push to git.bioconductor.org. It is required to push a version bump to git.bioconductor.org to trigger a new build.

bioc-issue-bot commented 1 year ago

Dear Package contributor,

This is the automated single package builder at bioconductor.org.

Your package has been built on Linux, Mac, and Windows.

Congratulations! The package built without errors or warnings on all platforms.

Please see the build report for more details. This link will be active for 21 days.

Remember: if you submitted your package after July 7th, 2020, when making changes to your repository push to git@git.bioconductor.org:packages/DNAZooData to trigger a new build. A quick tutorial for setting up remotes and pushing to upstream can be found here.

js2264 commented 1 year ago

Hi @DarioS 👋 Sorry for coming back to it, I know this has been a heavy reviewing assignment, but would it be possible to wrap this up soonish? Let me know if I can provide any help. May thanks! J

DarioS commented 1 year ago

I have no further requests and the packages seem to be building well together. Thanks for your comprehensive infrastructure.

bioc-issue-bot commented 1 year ago

Your package has been accepted. It will be added to the Bioconductor nightly builds.

Thank you for contributing to Bioconductor!

Reviewers for Bioconductor packages are volunteers from the Bioconductor community. If you are interested in becoming a Bioconductor package reviewer, please see Reviewers Expectations.

lshep commented 1 year ago

The master branch of your GitHub repository has been added to Bioconductor's git repository.

To use the git.bioconductor.org repository, we need an 'ssh' key to associate with your github user name. If your GitHub account already has ssh public keys (https://github.com/js2264.keys is not empty), then no further steps are required. Otherwise, do the following:

  1. Add an SSH key to your github account
  2. Submit your SSH key to Bioconductor

See further instructions at

https://bioconductor.org/developers/how-to/git/

for working with this repository. See especially

https://bioconductor.org/developers/how-to/git/new-package-workflow/ https://bioconductor.org/developers/how-to/git/sync-existing-repositories/

to keep your GitHub and Bioconductor repositories in sync.

Your package will be included in the next nigthly 'devel' build (check-out from git at about 6 pm Eastern; build completion around 2pm Eastern the next day) at

https://bioconductor.org/checkResults/

(Builds sometimes fail, so ensure that the date stamps on the main landing page are consistent with the addition of your package). Once the package builds successfully, you package will be available for download in the 'Devel' version of Bioconductor using BiocManager::install("HiCExperiment"). The package 'landing page' will be created at

https://bioconductor.org/packages/HiCExperiment

If you have any questions, please contact the bioc-devel mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel); this issue will not be monitored further.

lshep commented 1 year ago

The master branch of your GitHub repository has been added to Bioconductor's git repository.

To use the git.bioconductor.org repository, we need an 'ssh' key to associate with your github user name. If your GitHub account already has ssh public keys (https://github.com/js2264.keys is not empty), then no further steps are required. Otherwise, do the following:

  1. Add an SSH key to your github account
  2. Submit your SSH key to Bioconductor

See further instructions at

https://bioconductor.org/developers/how-to/git/

for working with this repository. See especially

https://bioconductor.org/developers/how-to/git/new-package-workflow/ https://bioconductor.org/developers/how-to/git/sync-existing-repositories/

to keep your GitHub and Bioconductor repositories in sync.

Your package will be included in the next nigthly 'devel' build (check-out from git at about 6 pm Eastern; build completion around 2pm Eastern the next day) at

https://bioconductor.org/checkResults/

(Builds sometimes fail, so ensure that the date stamps on the main landing page are consistent with the addition of your package). Once the package builds successfully, you package will be available for download in the 'Devel' version of Bioconductor using BiocManager::install("HiCool"). The package 'landing page' will be created at

https://bioconductor.org/packages/HiCool

If you have any questions, please contact the bioc-devel mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel); this issue will not be monitored further.

lshep commented 1 year ago

The master branch of your GitHub repository has been added to Bioconductor's git repository.

To use the git.bioconductor.org repository, we need an 'ssh' key to associate with your github user name. If your GitHub account already has ssh public keys (https://github.com/js2264.keys is not empty), then no further steps are required. Otherwise, do the following:

  1. Add an SSH key to your github account
  2. Submit your SSH key to Bioconductor

See further instructions at

https://bioconductor.org/developers/how-to/git/

for working with this repository. See especially

https://bioconductor.org/developers/how-to/git/new-package-workflow/ https://bioconductor.org/developers/how-to/git/sync-existing-repositories/

to keep your GitHub and Bioconductor repositories in sync.

Your package will be included in the next nigthly 'devel' build (check-out from git at about 6 pm Eastern; build completion around 2pm Eastern the next day) at

https://bioconductor.org/checkResults/

(Builds sometimes fail, so ensure that the date stamps on the main landing page are consistent with the addition of your package). Once the package builds successfully, you package will be available for download in the 'Devel' version of Bioconductor using BiocManager::install("fourDNData"). The package 'landing page' will be created at

https://bioconductor.org/packages/fourDNData

If you have any questions, please contact the bioc-devel mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel); this issue will not be monitored further.

lshep commented 1 year ago

The master branch of your GitHub repository has been added to Bioconductor's git repository.

To use the git.bioconductor.org repository, we need an 'ssh' key to associate with your github user name. If your GitHub account already has ssh public keys (https://github.com/js2264.keys is not empty), then no further steps are required. Otherwise, do the following:

  1. Add an SSH key to your github account
  2. Submit your SSH key to Bioconductor

See further instructions at

https://bioconductor.org/developers/how-to/git/

for working with this repository. See especially

https://bioconductor.org/developers/how-to/git/new-package-workflow/ https://bioconductor.org/developers/how-to/git/sync-existing-repositories/

to keep your GitHub and Bioconductor repositories in sync.

Your package will be included in the next nigthly 'devel' build (check-out from git at about 6 pm Eastern; build completion around 2pm Eastern the next day) at

https://bioconductor.org/checkResults/

(Builds sometimes fail, so ensure that the date stamps on the main landing page are consistent with the addition of your package). Once the package builds successfully, you package will be available for download in the 'Devel' version of Bioconductor using BiocManager::install("DNAZooData"). The package 'landing page' will be created at

https://bioconductor.org/packages/DNAZooData

If you have any questions, please contact the bioc-devel mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel); this issue will not be monitored further.