search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
696 stars 36 forks source link

[REVIEW]: UltraNest - a robust, general purpose Bayesian inference engine #3001

Closed whedon closed 3 years ago

whedon commented 3 years ago

Submitting author: @JohannesBuchner (Johannes Buchner) Repository: https://github.com/JohannesBuchner/UltraNest/ Version: v3.2.0 Editor: @fboehm Reviewer: @mattpitkin, @ziatdinovmax Archive: 10.5281/zenodo.4653423

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/85b49ded8945434bdc1343d14cad7b6e"><img src="https://joss.theoj.org/papers/85b49ded8945434bdc1343d14cad7b6e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/85b49ded8945434bdc1343d14cad7b6e/status.svg)](https://joss.theoj.org/papers/85b49ded8945434bdc1343d14cad7b6e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@mattpitkin & @ziatdinovmax, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @fboehm know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @mattpitkin

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @ziatdinovmax

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @mattpitkin, @ziatdinovmax it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago

PDF failed to compile for issue #3001 with the following error:

Can't find any papers to compile :-(

mattpitkin commented 3 years ago

@JohannesBuchner for the conda installation instructions in the documentation can you add that you need to explicitly specify the conda-forge channel, e.g., either:

conda install -c conda-forge ultranest

or 

conda config --add channels conda-forge
conda install ultranest
mattpitkin commented 3 years ago

@johannesbuchner can you merge the joss-paper branch of the ultranest repo into master so that the paper can be compiled?

JohannesBuchner commented 3 years ago

I'd rather not, I understood I can use a branch https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements

Your paper (paper.md and BibTeX files, plus any figures) must be hosted in a Git-based repository together with your software (although they may be in a short-lived branch which is never merged with the default).

JohannesBuchner commented 3 years ago

@whedon generate pdf from branch joss-paper

whedon commented 3 years ago 
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mattpitkin commented 3 years ago

@johannesbuchner thanks for regenerating. I hadn't realised there was a method to generate it from a different branch!

mattpitkin commented 3 years ago

@johannesbuchner can you add the DOI https://doi.org/10.1021/acs.jctc.9b00567 for the Griffiths & Wales paper? Can you also update the Higson, Handley, Hobson & Lasenby paper to the published version as given here?

mattpitkin commented 3 years ago

Minor paper comments:

fboehm commented 3 years ago

Hello, @ziatdinovmax - How is the review going? Please feel free to make comments in this thread. Also, please feel free to "@" mention me. I can try to help with any issues or clarifications.

thanks again!

ziatdinovmax commented 3 years ago

@fboehm My apologies for the delay on my end. I will add my comments shortly.

whedon commented 3 years ago

:wave: @ziatdinovmax, please update us on how your review is going.

whedon commented 3 years ago

:wave: @mattpitkin, please update us on how your review is going.

mattpitkin commented 3 years ago

@whedon @fboehm I'm just waiting on @JohannesBuchner to address my above minor comments on the paper and then I can sign-off the review.

whedon commented 3 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands
ziatdinovmax commented 3 years ago

@fboehm

Here you go:

In their package/paper, J. Buchner introduces a general-purpose Bayesian-inference approach for parameter estimation and model comparison. I was able to easily install the package and reproduce the examples in tutorials. The paper itself is written in a clear and concise manner. My only suggestion is to improve documentation of individual functions according to one of the accepted styles (reST, Google-style, Numpydoc, etc.). For example, currently, most of the package's functions just have this

"""This function does this and that"""

But it should be expanded to something like this:

"""
This function does this and that.

Args:
    param1: This is the first param.
    param2: This is a second param.

Returns:
    This is a description of what is returned.
"""

After this I will be happy to recommend the acceptance of this paper.

JohannesBuchner commented 3 years ago

I haven't had time yet to address Matt's comment, but just as a quick reply: This convention is from PEP 257, section "One-line Docstrings". I use it for simple functions where I believed the function name, the argument names and the one-line docstring give sufficient documentation. If you have a list of functions which are unclear, I am happy to expand the documentation.

fboehm commented 3 years ago

@JohannesBuchner - Would it be feasible to update the documentation as suggested by @ziatdinovmax? I get the impression that the suggestion is more comprehensive than what is currently written. THanks!

mattpitkin commented 3 years ago

@JohannesBuchner thanks for the updates. If you could just address https://github.com/openjournals/joss-reviews/issues/3001#issuecomment-771025395 I'll sign-off my final tick box for the review.

JohannesBuchner commented 3 years ago

@JohannesBuchner thanks for the updates. If you could just address #3001 (comment) I'll sign-off my final tick box for the review.

I think this is fixed now at https://johannesbuchner.github.io/UltraNest/installation.html

mattpitkin commented 3 years ago

@JohannesBuchner Great, thanks.

@fboehm I've ticked off all my check boxes, so am happy with the review.

JohannesBuchner commented 3 years ago

@JohannesBuchner - Would it be feasible to update the documentation as suggested by @ziatdinovmax? I get the impression that the suggestion is more comprehensive than what is currently written. THanks!

I believe I addressed this now. Please let me know of any remaining issues.

The following modules should not be considered, as they are in development and not referenced in the remaining documentation or paper: dychmc, dyhmc, flatnuts, pathsampler.

JohannesBuchner commented 3 years ago

I sent this a few minutes too early ... just uploading the docs now.

JohannesBuchner commented 3 years ago

Should be online now.

JohannesBuchner commented 3 years ago

@ziatdinovmax Please check if you can tick the "Functionality documentation" box with the updated docs, or give further comments if something is missing.

ziatdinovmax commented 3 years ago

@JohannesBuchner Looks great, thanks for addressing it!

fboehm commented 3 years ago

@whedon generate pdf

fboehm commented 3 years ago

@whedon check references

whedon commented 3 years ago

PDF failed to compile for issue #3001 with the following error:

Can't find any papers to compile :-(

fboehm commented 3 years ago

@whedon generate pdf from branch joss-paper

whedon commented 3 years ago 
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fboehm commented 3 years ago

@whedon check references

fboehm commented 3 years ago

@whedon check references from branch joss-paper

whedon commented 3 years ago 
Attempting to check references... from custom branch joss-paper
whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1063/1.1835238 is OK
- 10.1214/aoms/1177730491 is OK
- 10.1137/1116083 is OK
- 10.1126/science.220.4598.671 is OK
- 10.1103/PhysRevLett.57.2607 is OK
- 10.1080/01621459.1995.10476572 is OK
- 10.1214/aos/1056562461 is OK
- 10.1111/j.1468-4004.2006.47430.x is OK
- 10.1086/501068 is OK
- 10.1103/PhysRevD.73.123523 is OK
- 10.1063/1.2821265 is OK
- 10.1111/j.1365-2966.2007.11871.x is OK
- 10.1111/j.1365-2966.2007.12353.x is OK
- 10.1007/978-0-387-71887-3 is OK
- 10.1088/0264-9381/25/18/184010 is OK
- 10.1088/0264-9381/26/11/114011 is OK
- 10.1088/0264-9381/26/11/114008 is OK
- 10.1088/0264-9381/26/21/215003 is OK
- 10.1111/j.1365-2966.2009.14548.x is OK
- 10.1088/0264-9381/26/22/225004 is OK
- 10.1103/PhysRevD.80.095013 is OK
- 10.1111/j.1365-2966.2008.14005.x is OK
- 10.1007/JHEP04(2010)057 is OK
- 10.1093/biomet/asq021 is OK
- 10.2140/camcos.2010.5.65 is OK
- 10.1103/PhysRevD.81.062003 is OK
- 10.1088/1475-7516/2011/04/012 is OK
- 10.1088/1475-7516/2011/07/002 is OK
- 10.1063/1.3573613 is OK
- 10.1007/JHEP06(2011)042 is OK
- 10.1186/1752-0509-5-97 is OK
- 10.1111/j.1365-2966.2011.18474.x is OK
- 10.1145/1921598.1921600 is OK
- 10.1063/1.3573612 is OK
- 10.1111/j.1365-2966.2011.20288.x is OK
- 10.1111/j.1365-2966.2012.20540.x is OK
- 10.1103/PhysRevD.85.104045 is OK
- 10.1088/0004-637X/778/2/153 is OK
- 10.1371/journal.pone.0082317 is OK
- 10.1063/1.4819989 is OK
- 10.1051/0004-6361/201220063 is OK
- 10.1088/0004-637X/778/1/32 is OK
- 10.1093/mnras/stt2190 is OK
- 10.1007/s11222-014-9512-y is OK
- 10.1051/0004-6361/201322971 is OK
- 10.1051/0004-6361/201424181 is OK
- 10.1088/0264-9381/31/20/205006 is OK
- 10.1063/1.4903724 is OK
- 10.1103/PhysRevX.4.031034 is OK
- 10.1371/journal.pone.0088419 is OK
- 10.1093/mnras/stu109 is OK
- 10.1093/mnras/stv199 is OK
- 10.1098/rsos.150030 is OK
- 10.1063/1.4905971 is OK
- 10.1093/mnrasl/slv047 is OK
- 10.1093/mnras/stv1911 is OK
- 10.1088/0004-637X/802/2/107 is OK
- 10.1063/1.4933309 is OK
- 10.1103/PhysRevB.93.174108 is OK
- 10.1093/mnras/stv2370 is OK
- 10.1051/0004-6361/201527522 is OK
- 10.1093/mnras/stv2217 is OK
- 10.3847/0004-6256/152/6/203 is OK
- 10.1002/2016RS005965 is OK
- 10.1063/1.4985355 is OK
- 10.1007/s11222-018-9844-0 is OK
- 10.1088/1538-3873/aae7fc is OK
- 10.1021/acs.jctc.9b00567 is OK
- 10.1093/mnras/sty3090 is OK
- 10.1093/mnras/sty3307 is OK
- 10.21105/joss.01809 is OK
- 10.1093/mnras/staa1469 is OK
- 10.3847/1538-3881/ab5190 is OK
- 10.1093/mnras/staa278 is OK
- 10.3390/e22020185 is OK
- 10.21105/joss.00024 is OK
- 10.1086/670067 is OK

MISSING DOIs

- 10.1214/aoms/1177732360 may be a valid DOI for title: The large-sample distribution of the likelihood ratio for testing composite hypotheses
- 10.1287/opre.32.6.1296 may be a valid DOI for title: Efficient Monte Carlo procedures for generating points uniformly distributed over bounded regions
- 10.1016/s0951-8320(02)00056-x may be a valid DOI for title: An approximate epistemic uncertainty analysis approach in the presence of epistemic and aleatory uncertainties
- 10.1093/nar/gki282 may be a valid DOI for title: NestedMICA: sensitive inference of over-represented motifs in nucleic acid sequence
- 10.1063/1.2149795 may be a valid DOI for title: Using Nested Sampling in the Analysis of Multi-Rate Sound Energy Decay in Acoustically Coupled Rooms
- 10.1214/06-ba127 may be a valid DOI for title: Nested sampling for general Bayesian computation
- 10.1021/ci7000633.s004 may be a valid DOI for title: Gaussian processes: a method for automatic QSAR modeling of ADME properties
- 10.1186/1471-2105-9-19 may be a valid DOI for title: NestedMICA as an ab initio protein motif discovery tool
- 10.1016/j.marpetgeo.2009.01.003 may be a valid DOI for title: Markov chain Monte Carlo (MCMC) sampling methods to determine optimal models, model resolution and model choice for Earth Science problems
- 10.1063/1.3275622 may be a valid DOI for title: Computational methods for Bayesian model choice
- 10.1063/1.3275625 may be a valid DOI for title: Nested sampling’s convergence
- 10.1142/s0217751x11052323 may be a valid DOI for title: Bayesian model selection for electromagnetic kaon production on the nucleon
- 10.1021/jp1012973 may be a valid DOI for title: Efficient sampling of atomic configurational spaces
- 10.1007/s11222-010-9198-8 may be a valid DOI for title: Diffusive nested sampling
- 10.1111/j.1365-2966.2011.18962.x may be a valid DOI for title: Detecting extrasolar planets from stellar radial velocities using Bayesian evidence
- 10.1201/b10905-6 may be a valid DOI for title: MCMC using Hamiltonian dynamics
- 10.1109/tim.2011.2105030 may be a valid DOI for title: A model-based probabilistic inversion framework for characterizing wire fault detection using TDR
- 10.1016/j.bpj.2011.12.053 may be a valid DOI for title: Exploring the energy landscapes of protein folding simulations with Bayesian computation
- 10.1121/1.4754550 may be a valid DOI for title: Nested sampling applied in Bayesian room-acoustics decay analysis a
- 10.1111/j.1365-2966.2012.22124.x may be a valid DOI for title: An analytic model for rotational modulations in the photometry of spotted stars
- 10.1186/1752-0509-7-72 may be a valid DOI for title: Nested sampling for parameter inference in systems biology: application to an exemplar circadian model
- 10.1201/b18502-11 may be a valid DOI for title: Hamiltonian Monte Carlo for Hierarchical Models
- 10.1002/2012wr013406 may be a valid DOI for title: Nested sampling algorithm for subsurface flow model selection, uncertainty quantification, and nonlinear calibration
- 10.21105/astro.1306.2144 may be a valid DOI for title: Importance Nested Sampling and the MultiNest Algorithm
- 10.1093/mnras/stt2122 may be a valid DOI for title: TEMPONEST: a Bayesian approach to pulsar timing analysis
- 10.1063/1.4821761 may be a valid DOI for title: Nested sampling in the canonical ensemble: Direct calculation of the partition function from NVT trajectories
- 10.1016/j.cma.2013.11.001 may be a valid DOI for title: Efficient Bayesian inference of subsurface flow models using nested sampling and sparse polynomial chaos surrogates
- 10.1002/qj.2245 may be a valid DOI for title: A nested sampling particle filter for nonlinear data assimilation
- 10.1016/j.jcp.2013.10.001 may be a valid DOI for title: Hybrid nested sampling algorithm for Bayesian model selection applied to inverse subsurface flow problems
- 10.1063/1.4903717 may be a valid DOI for title: Parallelized nested sampling
- 10.1093/bioinformatics/btu675 may be a valid DOI for title: SYSBIONS: nested sampling for systems biology
- 10.1103/physreve.89.022302 may be a valid DOI for title: Nested sampling for materials: The case of hard spheres
- 10.1002/2014wr016062 may be a valid DOI for title: Model selection on solid ground: Rigorous comparison of nine ways to evaluate Bayesian model evidence
- 10.1007/s11222-015-9617-y may be a valid DOI for title: Point Process-based Monte Carlo estimation
- 10.1016/j.strusafe.2015.02.002 may be a valid DOI for title: Moving Particles: a parallel optimal Multilevel Splitting method with application in quantiles estimation and meta-model based algorithms
- 10.1016/j.advwatres.2014.10.006 may be a valid DOI for title: Calibration of channelized subsurface flow models using nested sampling and soft probabilities
- 10.1088/1742-6596/626/1/012070 may be a valid DOI for title: Ellipsoidal nested sampling, expression of the model uncertainty and measurement
- 10.1088/0264-9381/32/21/215012 may be a valid DOI for title: Classification methods for noise transients in advanced gravitational-wave detectors
- 10.1051/ps/2015017 may be a valid DOI for title: Rare Event Simulation and Splitting for Discontinuous Random Variables
- 10.1002/2014wr016718 may be a valid DOI for title: Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods
- 10.1063/1.4959055 may be a valid DOI for title: Equidistribution testing with Bayes factors and the ECT
- 10.3390/e19080422 may be a valid DOI for title: Computing Entropies With Nested Sampling
- 10.3847/1538-3881/aa7ed8 may be a valid DOI for title: HELIOS–RETRIEVAL: An Open-source, Nested Sampling Atmospheric Retrieval Code; Application to the HR 8799 Exoplanets and Inferred Constraints for Planet Formation
- 10.1093/sysbio/syy050 may be a valid DOI for title: Model selection and parameter inference in phylogenetics using Nested Sampling
- 10.1063/1.4985378 may be a valid DOI for title: New prior sampling methods for nested sampling-Development and testing
- 10.1093/mnras/staa2345 may be a valid DOI for title: Nested sampling cross-checks using order statistics

INVALID DOIs

- https://doi.org/10.1016/j.bpj.2011.12.053 is INVALID because of 'https://doi.org/' prefix
- http://dx.doi.org/10.1063/1.3703630 is INVALID because of 'https://doi.org/' prefix
- http://dx.doi.org/10.1016/j.cpc.2015.12.005 is INVALID because of 'https://doi.org/' prefix
- http://dx.doi.org/10.1016/j.dsp.2017.07.021 is INVALID because of 'https://doi.org/' prefix
fboehm commented 3 years ago

The DOIs look good. I checked all of them manually after downloading the pdf of the paper.

fboehm commented 3 years ago

@JohannesBuchner - the reviewers have recommended your submission for publication. Would you please make a tagged release and archive and report the version number and archive DOI in the review thread?

You'll need to be sure that the author and title of the archive match exactly those of the paper. You can use zenodo or figshare for the archiving.

JohannesBuchner commented 3 years ago

OK, I think I did it:

v3.1.0

released at https://github.com/JohannesBuchner/UltraNest/releases/tag/v3.1.0

zenodo record: https://zenodo.org/record/4636924

doi: 10.5281/zenodo.4636924

fboehm commented 3 years ago

@wehdon set version v3.1.0

fboehm commented 3 years ago

@whedon set v3.1.0 as version

whedon commented 3 years ago

OK. v3.1.0 is the version.

fboehm commented 3 years ago

@whedon set 10.5281/zenodo.4636924 as archive

whedon commented 3 years ago

OK. 10.5281/zenodo.4636924 is the archive.

fboehm commented 3 years ago

@whedon accept

whedon commented 3 years ago 
Attempting dry run of processing paper acceptance...
whedon commented 3 years ago

PDF failed to compile for issue #3001 with the following error:

Can't find any papers to compile :-(

fboehm commented 3 years ago

@whedon accept from branch joss-paper