openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
718 stars 38 forks source link

[REVIEW]: 21cmFAST v3: A Python-integrated C code for generating 3D realizations of the cosmic 21cm signal. #2582

Closed whedon closed 3 years ago

whedon commented 4 years ago

Submitting author: @steven-murray (Steven Murray) Repository: https://github.com/21cmFAST/21cmFAST Version: v3.0.3 Editor: @dfm Reviewer: @sambit-giri, @sultan-hassan Archive: 10.5281/zenodo.4107189

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/56b5f5228d04377a9d612b30ce82b447"><img src="https://joss.theoj.org/papers/56b5f5228d04377a9d612b30ce82b447/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/56b5f5228d04377a9d612b30ce82b447/status.svg)](https://joss.theoj.org/papers/56b5f5228d04377a9d612b30ce82b447)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@sambit-giri & @sultanier, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @dfm know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

Review checklist for @sambit-giri

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @sultan-hassan

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @sambit-giri, @sultanier it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 4 years ago

PDF failed to compile for issue #2582 with the following error:

Can't find any papers to compile :-(

dfm commented 4 years ago

@whedon generate pdf from branch joss-paper

whedon commented 4 years ago
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

steven-murray commented 4 years ago

Thanks again @sambit-giri and @sultanier for reviewing! And @dfm for editing.

sambit-giri commented 4 years ago

Hi @dfm I am unable to edit the reviewer's checklist. I don't remember if I had accepted the invitation. I must apologise for it. When I try to do it now, I get the following message: "Sorry, we couldn't find that repository invitation. It is possible that the invitation was revoked or that you are not logged into the invited account."

danielskatz commented 4 years ago

@whedon re-invite @sambit-giri as reviewer

whedon commented 4 years ago

OK, the reviewer has been re-invited.

@sambit-giri please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

sambit-giri commented 4 years ago

Hi @steven-murray Here are a few comments on the software paper:

  1. In paragraph 1, you say “Between these extremes lies an especially versatile middle-ground: fast semi-numerical models that approximate the full 3D evolution of the relevant fields: density, velocity, temperature, ionization, and radiation.” It is not clear which radiation you are talking about.
  2. In paragraph 1, you say “...so much so that they can be used to produce thousands of realizations on scales comparable to those observable by upcoming telescopes, in order....”. Can you mention the observables and few upcoming telescopes? Do you just mean the 21-cm signal and the upcoming radio telescopes? 21cmFast has also been used to put constraints based on other observations, such as quasar spectra and CMB observations. What about them?
  3. The caption in Figure 1 does not describe the figure completely. The “component fields” are labelled with symbols in the figure. They have not been defined in the text and therefore are difficult to understand.
  4. In paragraph 4, you say “This paper presents 21cmFAST v3+, which enables these essential new features. While keeping the same core functionality of previous versions of 21cmFAST, it…”. It is unclear which are the “new features” and which are the “same core functionality”.
  5. There is an error in your citation in the fourth line of the list in paragraph 6.
  6. In paragraph 7, I suggest you cite some example papers about milli-charged dark matter models and forward-modelled CMB auxiliary data.
  7. Please cite the 21CMMC paper.
  8. Please add a citation for “CLASS Boltzmann code”.
  9. In Figure 3, please mention that ‘xH’ is the mass/volume average neutral fraction.
dfm commented 4 years ago

@sultanier: I just wanted to ping you to make sure that this is still on your radar. Let me know if you have any questions as you go through the checklist!

sultan-hassan commented 4 years ago

thanks @dfm will do this asap.

qyx268 commented 4 years ago

Hey @sambit-giri, thanks for reviewing this! I could answer the following points. @steven-murray Could you please add these in the paper?

  1. The caption in Figure 1 does not describe the figure completely. The “component fields” are labelled with symbols in the figure. They have not been defined in the text and therefore are difficult to understand.

Lightcones for the reference model including, from left to right: (i) overdensity ($\Delta{\rm cell}$); (ii) {\lya} ($J{\alpha, {\rm eff}}{\times}\rm s^{-1}Hz^{-1}cm^{-2}sr^{-1}$); (iii) LW ($J^{\rm 21}{\rm LW,eff}{\times}{\rm 10^{{-}21}{\rm erg\ s^{-1}\ Hz^{-1}\ cm^{-2}\ sr^{-1}}}$); (iv) X-ray heating ($\varepsilon{X}$ in units of $k\mathrm{B} {\rm K Gyr}^{-1}$); (v) locally averaged UVB ($\bar{\Gamma}{\rm ion}$ in units of $10^{-12}\rm{s}^{-1}$); (vi) critical halo mass for star formation in ACGs ($M{\rm crit}^{\rm atom}$/${\msol}$); (vii) critical halo mass for star formation in MCGs ($M{\rm crit}^{\rm mol}$/${\msol}$); (viii) cumulative number of recombinations per baryon ($n{\rm rec}$); (ix) neutral hydrogen fraction ($x{\hone}$); and (x) the 21-cm brightness temperature ($\delta T_{\rm b}$ in units of ${\rm mK}$).\newline {(A high-resolution version of this figure is available at \url{http://homepage.sns.it/mesinger/Media/light-cones_minihalo.png}.})

  1. There is an error in your citation in the fourth line of the list in paragraph 6.
@ARTICLE{2020MNRAS.495..123Q,
       author = {{Qin}, Yuxiang and {Mesinger}, Andrei and {Park}, Jaehong and
         {Greig}, Bradley and {Mu{\~n}oz}, Julian B.},
        title = "{A tale of two sites - I. Inferring the properties of minihalo-hosted galaxies from current observations}",
      journal = {\mnras},
     keywords = {galaxies: high-redshift, intergalactic medium, dark ages, reionization, first stars, diffuse radiation, early Universe, cosmology: theory, Astrophysics - Cosmology and Nongalactic Astrophysics, Astrophysics - Astrophysics of Galaxies},
         year = 2020,
        month = apr,
       volume = {495},
       number = {1},
        pages = {123-140},
          doi = {10.1093/mnras/staa1131},
archivePrefix = {arXiv},
       eprint = {2003.04442},
 primaryClass = {astro-ph.CO},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2020MNRAS.495..123Q},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}
  1. Please add a citation for “CLASS Boltzmann code”.
@ARTICLE{Lesgourgues2011arXiv1104.2932L,
    author = {{Lesgourgues}, Julien},
    title = "{The Cosmic Linear Anisotropy Solving System (CLASS) I: Overview}",
    journal = {arXiv e-prints},
    keywords = {Astrophysics - Instrumentation and Methods for Astrophysics, Astrophysics - Cosmology and Extragalactic Astrophysics},
    year = 2011,
    month = apr,
    eid = {arXiv:1104.2932},
    pages = {arXiv:1104.2932},
    archivePrefix = {arXiv},
    eprint = {1104.2932},
    primaryClass = {astro-ph.IM},
    adsurl = {https://ui.adsabs.harvard.edu/abs/2011arXiv1104.2932L},
    adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}
steven-murray commented 4 years ago

@whedon generate pdf from branch joss-paper

whedon commented 4 years ago
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #2582 with the following error:

ORCID looks to be the wrong length /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/lib/whedon.rb:153:in block in check_orcids': Problem with ORCID (https://orcid.org/0000-0003-1443-3483) for Catherine A. Watkinson (RuntimeError) from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/lib/whedon.rb:151:ineach' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/lib/whedon.rb:151:in check_orcids' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/lib/whedon.rb:88:ininitialize' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/lib/whedon/processor.rb:36:in new' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/lib/whedon/processor.rb:36:inset_paper' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/bin/whedon:55:in prepare' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/command.rb:27:inrun' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in invoke_command' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor.rb:387:indispatch' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/base.rb:466:in start' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-364ded062842/bin/whedon:119:in<top (required)>' from /app/vendor/bundle/ruby/2.4.0/bin/whedon:23:in load' from /app/vendor/bundle/ruby/2.4.0/bin/whedon:23:in

'

steven-murray commented 4 years ago

@sambit-giri: I've updated the paper trying to address your points (see https://github.com/21cmfast/21cmFAST/pull/105/commits/371e1d8d502cec94408e4c96a423f4e1930ecf40 for the diff).

In brief:

  1. In paragraph 1, you say “Between these extremes lies an especially versatile middle-ground: fast semi-numerical models that approximate the full 3D evolution of the relevant fields: density, velocity, temperature, ionization, and radiation.” It is not clear which radiation you are talking about.

I've added that it is Lyman-alpha and 21-cm radiation.

  1. In paragraph 1, you say “...so much so that they can be used to produce thousands of realizations on scales comparable to those observable by upcoming telescopes, in order....”. Can you mention the observables and few upcoming telescopes? Do you just mean the 21-cm signal and the upcoming radio telescopes? 21cmFast has also been used to put constraints based on other observations, such as quasar spectra and CMB observations. What about them?

I've restricted it to low-frequency radio telescopes. While other observables are also useful in constraining astrophysics using 21cmFAST, I wanted to focus on the primary motivation.

  1. The caption in Figure 1 does not describe the figure completely. The “component fields” are labelled with symbols in the figure. They have not been defined in the text and therefore are difficult to understand.

Thanks for picking this up, I've added captions to all the figures, and this one describes the components.

  1. In paragraph 4, you say “This paper presents 21cmFAST v3+, which enables these essential new features. While keeping the same core functionality of previous versions of 21cmFAST, it…”. It is unclear which are the “new features” and which are the “same core functionality”.

I've changed the wording here to be the "essential guiding principals", and it refers back to the end of the previous paragraph. It then goes on to describe the new features.

  1. There is an error in your citation in the fourth line of the list in paragraph 6.

Fixed, thanks!

  1. In paragraph 7, I suggest you cite some example papers about milli-charged dark matter models and forward-modelled CMB auxiliary data.

Added, thanks!

  1. Please cite the 21CMMC paper.
  2. Please add a citation for “CLASS Boltzmann code”.
  3. In Figure 3, please mention that ‘xH’ is the mass/volume average neutral fraction.

All done.

steven-murray commented 4 years ago

@whedon generate pdf from branch joss-paper

whedon commented 4 years ago
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

steven-murray commented 4 years ago

@whedon generate pdf from branch joss-paper

whedon commented 4 years ago
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

sultan-hassan commented 4 years ago

@whedon I can't update the checklist, could you re-invite me again? thanks.

whedon commented 4 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands
sultan-hassan commented 4 years ago

@whedon re-invite @sultanier as reviewer

whedon commented 4 years ago

I'm sorry @sultanier, I'm afraid I can't do that. That's something only editors are allowed to do.

dfm commented 4 years ago

@whedon re-invite @sultanier as reviewer

whedon commented 4 years ago

OK, the reviewer has been re-invited.

@sultanier please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

steven-murray commented 4 years ago

@whedon generate pdf from branch joss-paper

whedon commented 4 years ago
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

sambit-giri commented 4 years ago

Hi @steven-murray

Can you make the GitHub repository self-sustained? You can add a short summary of what 21cmFast can do. It will be helpful for new users. You should also add some of the easiest installation method (for more details you provide a link to your readthedocs installation page), instructions about automated tests post-installation and community guidelines (contribution, raising issue etc).

I installed the package in MacOSX. The instructions in the readthedocs page was not enough. I had to refer to the GitHub issue you mention. Is it possible to improve the instructions by summarising a few of the important information discussed in the issue. Also, conda install -c conda-forge 21cmFAST worked better for me. Please mention this method in the readthedocs page and GitHub readme.

sambit-giri commented 4 years ago

@steven-murray

My automated tests failed. I am attaching a file containing the output I received while running python -m pytest.

pytest_output.txt

sambit-giri commented 4 years ago

@steven-murray Thanks for updating the software paper.

Just a final comment on that. The link given for Figure 1 is wrong. There is a hyphen in light-cones in the link given, but I see the high-resolution image at http://homepage.sns.it/mesinger/Media/lightcones_minihalo.png

steven-murray commented 4 years ago

Hi @sambit-giri, thanks for the very useful suggestions!

Can you make the GitHub repository self-sustained? You can add a short summary of what 21cmFast can do. It will be helpful for new users.

I've made these changes in https://github.com/21cmfast/21cmFAST/pull/172 (hopefully merged soon). The description is short but hopefully helpful.

You should also add some of the easiest installation method (for more details you provide a link to your readthedocs installation page),

Done in the above PR as well.

instructions about automated tests post-installation and community guidelines (contribution, raising issue etc).

Running automated tests post-installation is only possible if one clones the repo itself and installs that way. We expect that to happen only for those who want to be developers, in which case the developer documentation outlines how to run the tests. Also, we already have community guidelines in the organization's .github repo, which should show up when a new user tries to create an issue or PR. Agreed that it is less easy to find those instructions manually this way, but it means we have the same instructions for 21cmMC without redundant files.

I installed the package in MacOSX. The instructions in the readthedocs page was not enough. I had to refer to the GitHub issue you mention. Is it possible to improve the instructions by summarising a few of the important information discussed in the issue. Also, conda install -c conda-forge 21cmFAST worked better for me. Please mention this method in the readthedocs page and GitHub readme.

The installation page on RTD has been updated to include the simpler conda install (however, I've just noticed that the RTD build is failing, hence you didn't see it -- will fix that ASAP). Installing on MacOS without conda is just hard, and it would be difficult to summarise the different cases succinctly. I feel like referencing the issue is the best we can do for now.

Concerning your failed tests -- can you inform me of which version you're testing (master branch or otherwise?). CI is passing all tests on master currently (on MacOS as well), so it is strange that they fail for you.

I've also updated the link in the paper, thanks!

sultan-hassan commented 4 years ago

Hi @steven-murray, seems like a very nice package, congrats!

sultan-hassan commented 4 years ago

Hi @steven-murray again just few suggestions

sultan-hassan commented 4 years ago

@steven-murray, how do we see all options for the 'kind' argument in the coeval_sliceplot, I couldn't find this in the API. the actual spelling might be different, also they aren't in the flag_options: coeval8.flag_options: FlagOptions(INHOMO_RECO:False, M_MIN_in_Mass:False, PHOTON_CONS:False, SUBCELL_RSD:False, USE_HALO_FIELD:False, USE_MASS_DEPENDENT_ZETA:False, USE_MINI_HALOS:False, USE_TS_FLUCT:False)

sultan-hassan commented 4 years ago

@steven-murray the paper reads very nice, I think it misses how fast is this package compared to just running 21cmFAST directly? how the linking between python and C is achieved (OS module, Ctypes, Cython...etc)? The community would be interested to know the time spent for few examples: e.g. to to run a lightcone from z=20 to z=6 for box size 100 Mpc the code spends ...etc this is also relevant for users to have a rough estimate if they want to run MCMC with that. Otherwise looking good congrats again.

steven-murray commented 4 years ago

Thanks @sultanier for the very useful comments on the code! I've tried to address most of your comments in this PR: https://github.com/21cmfast/21cmFAST/pull/174

The installation went great as described, however just small suggestion to add might help new users to computing clusters that these dependencies like gsl, fftw..etc might already be installed so you might add: use e.g. module load gsl ....

Thanks, I added a HPC section to the installation docs.

I like the CLI interactive mode with the 21cmfast command but I was unable to run it. Could you give a complete example of how to use it? Also would it be possible to add the whole parameter list here and their meanings? because the config file isn't complete ~/.21cmfast/runconfig_example.yml, and should one copies the config file to another location or has to be there ..etc? Would be nice to include an example of a complete config file with all parameters and their definitions so a new user would know how to specify them, and where to store the generated simulation output. I see in the following page you have tutorial of how to use the package within python, but some users, like me, might prefer the interactive mode with the 21cmfast command.

Firstly, could you let me know what went wrong for you with the CLI command?

I've updated the documentation in the README to have a quick guide on using the CLI. It references the user to https://21cmfast.readthedocs.io/en/latest/reference/_autosummary/py21cmfast.inputs.html to get a list of all the possible inputs and their documentation. I've also put this URL into the runconfig_example.yml. The config file location is specified with --config=FILE on the CLI, which is also now documented in the README. I would argue against including a COMPLETE config, as it would be doubling-up the documentation of the code itself (and may get out of date). As for storing the generated output, the README now has some description of this, and you are able to save the (coeval or lightcone) file explicitly. I hope these things make it easier to use!

could you add in the beginning of section: Running and Plotting Coeval Cubes after "Clear the cache so that we get the same results for the notebook every time (don’t worry about this for now Also, set the default output directory to _cache/:" create dir, mkdir '_cache' where you want to run this jupyternotebook, otherwise it says _cache not found.

Done, thanks!

when you save files, is there anyway to make the saved files names more meaningful, for instance in this saved box name "BrightnessTemp_8da85e6e1d0e198f6a7f0f6b3af0a9a2_r12345.h5", I recognize that the last part in the files names is the seed number, is there any way for the user to specify the name? or the name can be changed to reflect other cosmo params?

The short answer is no, the automatically-saved cache files have a particular naming structure that is based on a hash of all the parameters that went into the simulation. We have tried to keep the most useful information in the filename (i.e. the kind of box it is, and the seed). Naming the file based on some subset of its parameters is not scalable, and would not do well as a cache. Of course, you can always copy whatever file you want to somewhere else on your filesystem and call it whatever you want! Also, we have made the 21cmfast query tool, which lists each box and all the parameters that went into it, to make it a bit easier to know what each simulation really is.

Finally, the higher level outputs (Coeval and Lightcone) do have a .save() method which allows you to save the box with any filename you want. They are not part of the caching system, so this is OK.

@steven-murray, how do we see all options for the 'kind' argument in the coeval_sliceplot, I couldn't find this in the API. the actual spelling might be different, also they aren't in the flag_options: coeval8.flag_options: FlagOptions(INHOMO_RECO:False, M_MIN_in_Mass:False, PHOTON_CONS:False, SUBCELL_RSD:False, USE_HALO_FIELD:False, USE_MASS_DEPENDENT_ZETA:False, USE_MINI_HALOS:False, USE_TS_FLUCT:False)

This is actually an open issue (https://github.com/21cmfast/21cmFAST/issues/81). I've tried to partially address it in the PR I linked above. There is now a method that can be called on the Coeval class (you can do it directly, before having run a simulation): Coeval.get_fields(), that will list these kinds for you. I've also added this information to the coeval_sliceplot function docstring. It's not a full solution, as the output fields are not actually documented online, but I will leave that issue open to remind us to do this in the future.

@steven-murray the paper reads very nice, I think it misses how fast is this package compared to just running 21cmFAST directly? how the linking between python and C is achieved (OS module, Ctypes, Cython...etc)? The community would be interested to know the time spent for few examples: e.g. to to run a lightcone from z=20 to z=6 for box size 100 Mpc the code spends ...etc this is also relevant for users to have a rough estimate if they want to run MCMC with that. Otherwise looking good congrats again.

This is a great suggestion. I will add another small section into the paper to describe this.

sultan-hassan commented 4 years ago

thanks @steven-murray

for the CLI command, I was't sure how to run it because there was no detailed config file and the required parameter list was not mentioned. I will try again once you add a detailed config file. Let me know once you finish adding/implementing these suggestions/comments, so I can double check them for the last time before finishing up my checklist. thanks.

steven-murray commented 4 years ago

@sultanier all the comments have been address in the current master (there are some formatting issues with the hyperlinks in the readme which are being addressed in a PR, but these shouldn't hamper your checks). The performance information to go into the paper itself is still a work in progress.

sultan-hassan commented 4 years ago

thanks @steven-murray,

I have finished checking all comments and everything looking good and working nicely! congrats!

One small fix is that these hyperlinks below for the config_example and inputs don't work (clicking on them doesn't directly take you to the files), but one can go to the repo and see these files. Just make sure these hyperlinks do actually work (if meant). I have completed all of my checklist but I am very curious to know the performance and speed for different examples, once you finish please generate the new pdf and let me know. Package is very beautifully made, congrats again!.

hyperlinks.....

There is an example configuration file here<user_data/runconfig_example.yml>_ that you can build from. All input parameters are documented here<https://21cmfast.readthedocs.io/en/latest/reference/_autosummary/py21cmfast.inputs.html>_.

steven-murray commented 4 years ago

Thanks @sultanier! There is a PR currently to fix those hyperlinks (should be merged in the next couple of days). We'll have those performance metrics into the paper very soon as well.

steven-murray commented 4 years ago

@whedon generate pdf from branch joss-paper

whedon commented 4 years ago
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #2582 with the following error:

Can't find any papers to compile :-(

steven-murray commented 4 years ago

@whedon generate pdf from branch joss-paper

whedon commented 4 years ago
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...