[PRE REVIEW]: BayesFlow: Amortized Bayesian Workflows With Neural Networks

[editorialbot]

Submitting author: !--author-handle-->@marvinschmitt<!--end-author-handle-- (Marvin Schmitt) Repository: https://github.com/stefanradev93/BayesFlow Branch with paper.md (empty if default branch): joss-submission Version: v1.1.1 Editor: !--editor-->@osorensen<!--end-editor-- Reviewers: @sandeshkatakam, @LoryPack Managing EiC: Kevin M. Moerman

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/4e59f48f4a948bc62b0e31fc04b05a89"><img src="https://joss.theoj.org/papers/4e59f48f4a948bc62b0e31fc04b05a89/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/4e59f48f4a948bc62b0e31fc04b05a89/status.svg)](https://joss.theoj.org/papers/4e59f48f4a948bc62b0e31fc04b05a89)

Author instructions

Thanks for submitting your paper to JOSS @marvinschmitt. Currently, there isn't a JOSS editor assigned to your paper.

@marvinschmitt if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.18 s (436.4 files/s, 184394.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          49           2767           6017           6373
Jupyter Notebook                10              0          12927           2428
TeX                              1             16              0           1328
Markdown                         3            126              0            307
reStructuredText                 6            112            157            143
YAML                             5             15             21             98
INI                              1              5              0             30
TOML                             1              4              0             28
DOS Batch                        1              8              1             26
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            78           3057          19130          10770
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 1495

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/mprv.2014.42 is OK
- 10.18637/jss.v067.i01 is OK
- 10.1109/TNNLS.2020.3042395 is OK
- 10.1109/CVPR.2015.7298700 is OK
- 10.1016/j.neucom.2020.01.028 is OK
- 10.1111/insr.12107 is OK
- 10.1038/nature14541 is OK
- 10.1088/1361-6420/abe928 is OK
- 10.7717/peerj-cs.55 is OK
- 10.21105/joss.04304 is OK

MISSING DOIs

- 10.32614/rj-2018-017 may be a valid DOI for title: Advanced Bayesian Multilevel Modeling with the R Package brms
- 10.1080/00949655.2020.1783262 may be a valid DOI for title: Approximate leave-future-out cross-validation for Bayesian time series models
- 10.1006/ijhc.2001.0469 may be a valid DOI for title: ACT-R/PM and menu selection: Applying a cognitive architecture to HCI
- 10.20982/tqmp.16.2.p120 may be a valid DOI for title: Jumping to conclusion? a Lévy flight model of decision making
- 10.1088/1475-7516/2015/08/043 may be a valid DOI for title: Approximate Bayesian computation for forward modeling in cosmology
- 10.1037/0003-066x.51.4.355 may be a valid DOI for title: ACT: A simple theory of complex cognition
- 10.1207/s15327051hci1204_5 may be a valid DOI for title: ACT-R: A theory of higher level cognition and its relation to visual attention
- 10.1101/465955 may be a valid DOI for title: Distance-based protein folding powered by deep learning
- 10.21468/scipostphys.10.6.126 may be a valid DOI for title: Measuring QCD splittings with invertible networks
- 10.1080/01621459.2017.1285773 may be a valid DOI for title: Variational inference: A review for statisticians
- 10.1190/geo2022-0472.1 may be a valid DOI for title: Reliable amortized variational inference with physics-based latent distribution correction
- 10.1103/physrevlett.127.241103 may be a valid DOI for title: Real-Time Gravitational Wave Science with Neural Posterior Estimation
- 10.1101/2020.11.20.392274 may be a valid DOI for title: Likelihood Approximation Networks (LANs) for Fast Inference of Simulation Models in Cognitive Neuroscience
- 10.1016/j.jmp.2020.102416 may be a valid DOI for title: Developing memory-based models of ACT-R within a statistical framework
- 10.1080/19345747.2011.618213 may be a valid DOI for title: Why we (usually) don’t have to worry about multiple comparisons
- 10.1109/tvcg.2021.3067779 may be a valid DOI for title: FixationNet: Forecasting Eye Fixations in Task-Oriented Virtual Environments
- 10.1111/cogs.12738 may be a valid DOI for title: Parameter inference for computational cognitive models with approximate Bayesian Computation
- 10.1109/tpami.2020.2992934 may be a valid DOI for title: Normalizing flows: An introduction and review of current methods
- 10.1007/jhep01(2019)057 may be a valid DOI for title: QCD-aware recursive neural networks for jet physics
- 10.1038/s41562-021-01282-7 may be a valid DOI for title: Mental speed is high until age 60 as revealed by analysis of over a million participants
- 10.1016/j.jml.2017.08.004 may be a valid DOI for title: Models of retrieval in sentence comprehension: A computational evaluation using Bayesian hierarchical modeling
- 10.1007/s11222-020-09982-2 may be a valid DOI for title: Implicitly Adaptive Importance Sampling
- 10.1109/tnnls.2021.3124052 may be a valid DOI for title: Amortized Bayesian model comparison with evidential deep learning
- 10.1371/journal.pcbi.1009472 may be a valid DOI for title: OutbreakFlow: Model-based Bayesian inference of disease outbreak dynamics with invertible neural networks and its application to the COVID-19 pandemics in Germany
- 10.23943/princeton/9780691159287.003.0006 may be a valid DOI for title: Hierarchical Bayesian models
- 10.1371/journal.pone.0158832 may be a valid DOI for title: Applying mathematical optimization methods to an ACT-R instance-based learning model
- 10.1017/s0272263100012870 may be a valid DOI for title: Attention in Cognitive Science and Second Language Acquisition
- 10.5898/jhri.2.1.trafton may be a valid DOI for title: ACT-R/E: An embodied cognitive architecture for human-robot interaction
- 10.3758/s13423-013-0530-0 may be a valid DOI for title: A generalized, likelihood-free method for posterior estimation
- 10.1007/s11336-013-9381-x may be a valid DOI for title: Hierarchical approximate Bayesian computation
- 10.1016/j.mex.2020.100850 may be a valid DOI for title: Using Approximate Bayesian Computation for estimating parameters in the cue-based retrieval model of sentence processing
- 10.1214/12-ss102 may be a valid DOI for title: A survey of Bayesian predictive methods for model assessment, selection and comparison
- 10.1609/aaai.v33i01.33015885 may be a valid DOI for title: Infovae: Balancing learning and inference in variational autoencoders
- 10.2139/ssrn.3640351 may be a valid DOI for title: Estimation of agent-based models using Bayesian deep learning approach of BayesFlow
- 10.31234/osf.io/gpkfn may be a valid DOI for title: How to ask twenty questions and win: Machine learning tools for assessing preferences from small samples of willingness-to-pay prices
- 10.1103/physreve.106.055311 may be a valid DOI for title: Variational inference of fractional Brownian motion with linear computational complexity
- 10.2514/6.2022-0631 may be a valid DOI for title: Inverse design under uncertainty using conditional Normalizing Flows
- 10.3390/s22145408 may be a valid DOI for title: Towards reliable parameter extraction in MEMS final module testing using bayesian inference
- 10.1038/s41562-021-01282-7 may be a valid DOI for title: Mental speed is high until age 60 as revealed by analysis of over a million participants
- 10.1214/ss/1177009939 may be a valid DOI for title: Bayesian Experimental Design: A Review
- 10.1007/978-0-387-09612-4_9 may be a valid DOI for title: Bayesian versus frequentist inference
- 10.1007/s11222-022-10090-6 may be a valid DOI for title: Graphical test for discrete uniformity and its applications in goodness-of-fit evaluation and multiple sample comparison
- 10.1007/s13198-014-0242-5 may be a valid DOI for title: Bug prediction modeling using complexity of code changes
- 10.1101/162552 may be a valid DOI for title: pyABC: distributed, likelihood-free inference
- 10.21105/joss.04304 may be a valid DOI for title: pyABC: Efficient and robust easy-to-use approximate Bayesian computation
- 10.21105/joss.02505 may be a valid DOI for title: SBI–A toolkit for simulation-based inference
- 10.1038/s41569-018-0104-y may be a valid DOI for title: Computational models in cardiology
- 10.1371/journal.pone.0238559 may be a valid DOI for title: Modeling the spread of COVID-19 in Germany: Early assessment and possible scenarios
- 10.1101/2020.04.14.20063750 may be a valid DOI for title: Modeling exit strategies from COVID-19 lockdown with a focus on antibody tests
- 10.31234/osf.io/pqv2c may be a valid DOI for title: A general integrative neurocognitive modeling framework to jointly describe EEG and decision-making on single trials
- 10.21468/scipostphys.13.1.003 may be a valid DOI for title: Understanding event-generation networks via uncertainties
- 10.1201/9781003019169-8 may be a valid DOI for title: Approximate bayesian computation
- 10.1038/s41467-021-24252-z may be a valid DOI for title: Approximate Bayesian Computation of radiocarbon and paleoenvironmental record shows population resilience on Rapa Nui (Easter Island)
- 10.1111/j.2041-210x.2011.00179.x may be a valid DOI for title: ABC: an R package for approximate Bayesian computation (ABC)
- 10.1007/s13349-022-00638-5 may be a valid DOI for title: Probabilistic damage detection using a new likelihood-free Bayesian inference method

INVALID DOIs

- 10.18653/v1/P17 is INVALID

[Kevin-Mattheus-Moerman]

@editorialbot invite @fabian-s as editor

[editorialbot]

Invitation to edit this submission sent!

[Kevin-Mattheus-Moerman]

@marvinschmitt thanks for this submission. I am one of the AEiCs for JOSS and here to help process initial steps. I have started looking for a handling editor. For the moment, can you please see if you can trim down your .bib file to only include the references you use (perhaps you do use all, in which case no change is needed). In addition, can you check those potentially missing/invalid DOIs :point_up: ? FYI you can use @editorialbot check references to check those DOIs again, and you can use @editorialbot generate pdf to update the draft.

[fabian-s]

Sorry, I have to decline -- I don't really know Python & I'll be traveling most of the time until the end of August.

[marvinschmitt]

@Kevin-Mattheus-Moerman Thank you for initiating the pre-review phase! I will trim down and update the .bib file and check the DOIs. Would you like us to provide a list of suggested reviewers?

[marvinschmitt]

@editorialbot generate pdf

[marvinschmitt]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/TNNLS.2020.3042395 is OK
- 10.7717/peerj-cs.55 is OK
- 10.21105/joss.04304 is OK

MISSING DOIs

- 10.21468/scipostphys.10.6.126 may be a valid DOI for title: Measuring QCD splittings with invertible networks
- 10.31234/osf.io/pqv2c may be a valid DOI for title: A general integrative neurocognitive modeling framework to jointly describe EEG and decision-making on single trials
- 10.3390/s22145408 may be a valid DOI for title: Towards reliable parameter extraction in MEMS final module testing using bayesian inference
- 10.1109/tnnls.2021.3124052 may be a valid DOI for title: Amortized Bayesian model comparison with evidential deep learning
- 10.1371/journal.pcbi.1009472 may be a valid DOI for title: OutbreakFlow: Model-based Bayesian inference of disease outbreak dynamics with invertible neural networks and its application to the COVID-19 pandemics in Germany
- 10.1007/s11222-022-10090-6 may be a valid DOI for title: Graphical test for discrete uniformity and its applications in goodness-of-fit evaluation and multiple sample comparison
- 10.2139/ssrn.3640351 may be a valid DOI for title: Estimation of agent-based models using Bayesian deep learning approach of BayesFlow
- 10.1190/geo2022-0472.1 may be a valid DOI for title: Reliable amortized variational inference with physics-based latent distribution correction
- 10.31234/osf.io/gpkfn may be a valid DOI for title: How to ask twenty questions and win: Machine learning tools for assessing preferences from small samples of willingness-to-pay prices
- 10.2514/6.2022-0631 may be a valid DOI for title: Inverse design under uncertainty using conditional Normalizing Flows
- 10.1103/physreve.106.055311 may be a valid DOI for title: Variational inference of fractional Brownian motion with linear computational complexity
- 10.1038/s41562-021-01282-7 may be a valid DOI for title: Mental speed is high until age 60 as revealed by analysis of over a million participants
- 10.20982/tqmp.16.2.p120 may be a valid DOI for title: Jumping to conclusion? a Lévy flight model of decision making
- 10.1007/s13349-022-00638-5 may be a valid DOI for title: Probabilistic damage detection using a new likelihood-free Bayesian inference method

INVALID DOIs

- None

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[marvinschmitt]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.21468/SciPostPhys.10.6.126 is OK
- 10.1007/s42113-023-00167-4 is OK
- 10.3390/s22145408 is OK
- 10.1109/TNNLS.2021.3124052 is OK
- 10.1371/journal.pcbi.1009472 is OK
- 10.1109/TNNLS.2020.3042395 is OK
- 10.7717/peerj-cs.55 is OK
- 10.21105/joss.04304 is OK
- 10.1016/j.jedc.2021.104082 is OK
- 10.2514/6.2022-0631 is OK
- PhysRevE.106.055311 is OK
- 10.1038/s41562-021-01282-7 is OK
- 10.20982/tqmp.16.2.p120 is OK
- 10.1007/s13349-022-00638-5 is OK

MISSING DOIs

- None

INVALID DOIs

- doi.org/10.1007/s11222-022-10090-6 is INVALID because of 'doi.org/' prefix
- doi.org/10.1190/geo2022-0472.1 is INVALID because of 'doi.org/' prefix
- doi.org/10.1016/j.jocm.2023.100418 is INVALID because of 'doi.org/' prefix

[marvinschmitt]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.21468/SciPostPhys.10.6.126 is OK
- 10.1007/s42113-023-00167-4 is OK
- 10.3390/s22145408 is OK
- 10.1109/TNNLS.2021.3124052 is OK
- 10.1371/journal.pcbi.1009472 is OK
- 10.1109/TNNLS.2020.3042395 is OK
- 10.1007/s11222-022-10090-6 is OK
- 10.7717/peerj-cs.55 is OK
- 10.21105/joss.04304 is OK
- 10.1016/j.jedc.2021.104082 is OK
- 10.1190/geo2022-0472.1 is OK
- 10.1016/j.jocm.2023.100418 is OK
- 10.2514/6.2022-0631 is OK
- PhysRevE.106.055311 is OK
- 10.1038/s41562-021-01282-7 is OK
- 10.20982/tqmp.16.2.p120 is OK
- 10.1007/s13349-022-00638-5 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[osorensen]

@gkthiruvathukal, I'll edit this one.

[osorensen]

@editorialbot add @osorensen as editor

[editorialbot]

Assigned! @osorensen is now the editor

[osorensen]

👋 @rtbs-dev @Daniel-Dodd @EugeneHao, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[osorensen]

👋 @idoby, @sandeshkatakam, @jorgedch, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[idoby]

@osorensen Thanks for inviting me! This is a little bit outside my usual topics, so I would rather let someone more qualified conduct this review. If no other reviewer is found, please ping me again.

[sandeshkatakam]

👋 @idoby, @sandeshkatakam, @jorgedch, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

I want to take up this. I had some previous experience with Bayesian Neural networks, which might help me review this better. Let me know the procedure for reviewing this(I just joined as reviewer for JOSS recently)

[jorgedch]

Hi @sandeshkatakam. The topic is outside of my area of expertise, so unfortunately I'm not able to contribute as a reviewer. Best, Jorge

[osorensen]

Thanks to all of you for responding!

[osorensen]

@editorialbot add @sandeshkatakam as reviewer

[editorialbot]

@sandeshkatakam added to the reviewers list!

[osorensen]

Thanks for agreeing to review, @sandeshkatakam! The review is checklist based, and once I start the review issue, you will get instructions on how to proceed. You are also encouraged to look at the full reviewer guidelines here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html

I will start the review when I find one more reviewer.

[osorensen]

👋 @cranmer, @cthoyt, @LoryPack, given your expertise in the area, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[cthoyt]

Sorry, I don't have time, but maybe @yannikschaelte might be a good fit.

[osorensen]

Thanks for the quick response @cthoyt!

[yannikschaelte]

Hi @cthoyt , I guess I might be an good fit, but I'm indeed also a co-author, so slightly biased :rofl:

[cthoyt]

😅 I didn't carefully check this. Well, I guess I wasn't too far off! I hope you guys get a quick review, I am always a fan of your work!

[LoryPack]

Hi @osorensen . I'd be happy to help with this, but I won't be able to complete the review before the 2nd week of September. Please let me know if this works.

[osorensen]

Thanks @LoryPack, that works very well.

[osorensen]

@editorialbot add @LoryPack as reviewer

[editorialbot]

@LoryPack added to the reviewers list!

[osorensen]

@editorialbot start review

[editorialbot]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/5702.