openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[PRE REVIEW]: PDSim: A Shiny App for Polynomial Diffusion Model Simulation and Estimation #5359

Closed editorialbot closed 1 year ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@peilun-he<!--end-author-handle-- (Peilun He) Repository: https://github.com/peilun-he/polynomial-diffusion-model-simulation-and-estimation Branch with paper.md (empty if default branch): Version: v2.0 Editor: !--editor-->@samhforbes<!--end-editor-- Reviewers: @taqtiqa-mark, @bkrayfield Managing EiC: Olivia Guest

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/01ca1963062f7aa2ac70867648ad9548"><img src="https://joss.theoj.org/papers/01ca1963062f7aa2ac70867648ad9548/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/01ca1963062f7aa2ac70867648ad9548/status.svg)](https://joss.theoj.org/papers/01ca1963062f7aa2ac70867648ad9548)

Author instructions

Thanks for submitting your paper to JOSS @peilun-he. Currently, there isn't a JOSS editor assigned to your paper.

The AEiC suggestion for the handling editor is @samhforbes.

@peilun-he if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands
editorialbot commented 1 year ago

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago
Software report:

github.com/AlDanial/cloc v 1.88  T=0.01 s (1936.7 files/s, 162039.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                               19            151            190           1164
Markdown                         3             60              0            348
TeX                              1              7              0             65
YAML                             1              1              4             18
-------------------------------------------------------------------------------
SUM:                            24            219            194           1595
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago

Wordcount for paper.md is 1949

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 1 year ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.1287/mnsc.46.7.893.12034 may be a valid DOI for title: Short-Term Variations and Long-Term Dynamics in Commodity Prices
- 10.2139/ssrn.2479826 may be a valid DOI for title: Polynomial Diffusions and Applications in Finance
- 10.1137/19m1283264 may be a valid DOI for title: A multifactor polynomial framework for long-term electricity forwards with delivery period
- 10.1109/jproc.2003.823141 may be a valid DOI for title: Unscented filtering and nonlinear estimation

INVALID DOIs

- None
oliviaguest commented 1 year ago

@samhforbes do you have capacity for this?

oliviaguest commented 1 year ago

@peilun-he thanks for choosing JOSS. Can you elaborate on what has changed since this: https://github.com/openjournals/joss-reviews/issues/5172? Thanks!

peilun-he commented 1 year ago

@oliviaguest Dear Olivia, we would like you to kindly consider this submission of publication to JOSS. We would like to point out the contributions of this application in R and the associated paper:

  1. There are no existing polynomial diffusion simulation and estimation packages. The first contribution is therefore to build a library in R that both simulates the discrete time polynomial diffusion models (multivariate), as well as forms estimate technique for both the likelihood and state estimation in this class of non-linear non-Gaussian models. In our opinion, this constitutes a non-trivial technique implementation of recently created diffusion models of the relevance of financial mathematics.
    1. The second contribution was to develop stochastic pricing model for futures term structures using this class of polynomial diffusion models. The generalises classical existing stochastic models widely used in commodity futures modelling. This was also implemented as a part of the R implementation.
    2. The third contribution was to develop a user-friendly GUI so that users would visualise the outputs of the simulation and estimation results obtained from the application created.
    3. Lastly, for user efficiency, we also provided the ability to export the results to a csv for additional interrogation should that be desirable.

Please note that the development of the code to perform this estimation task took one and a half years to develop and test. There are approximately over 1000 lines of code in R. Many of these are written by the research team and not relying on simply calling other packages. Therefore, we would like to point out that whilst the Github repository does not demonstrate a long history of user commits, this is representing the amount of effort required to code this package. The majority of the development was done outside the Github repository. We mention this since we submitted this project to JOSS and we received no review due to the superficial assessment that the commit history is too short and the application had a GUI and that perhaps therefore there was no substantial contribution. We would like to dispute this assessment and we kindly request your reconsideration of this work, as we believe the work presented is both substantial, novel, and timely for the computational finance community. We hope you will therefore please consider this work, and we look forward to receiving your feedback.

oliviaguest commented 1 year ago

@peilun-he I/we will share more soon, but for the moment please bear with us; thank you so much. ☺️

peilun-he commented 1 year ago

@oliviaguest Can you please give us an update about the status of the paper? We would like to confirm that it is now under a review.

oliviaguest commented 1 year ago

@peilun-he sorry, I'm on holiday until Monday. @samhforbes and I will discuss this next week.

samhforbes commented 1 year ago

Hi @peilun-he I'm currently looking at your submission, as well as the reasons for the previous rejection. At the current stage, this submission falls short of JOSS requirements, but I also recognise the contributions the software makes. My concerns are as follows:

  1. Documentation. The README does not address how to launch the app, and the documentation is scarce as to what changing certain values in the app mean. We expect software to be useable by people who are not the developers, but some documentation is necessary for this. Given it is an R package you could consider creating a vignette. There should also be relevant citations of publications in that documentation.
  2. Can you consider (or justify why not) including a web interface containing the app so that each user does not need to build it locally? The shiny apps webpage has limited plans for example, and this would improve accessibility.
  3. Would you consider packaging this into an R package rather than a script with helpers? This is standard in R development, and while not necessarily a deal-breaker, could reasonably be expected by users, unless there is a good reason not to.

Depending on if / how these concerns are addressed I can then consider sending this software out for review, but addressing these concerns does not at this stage guarantee a positive outcome.

peilun-he commented 1 year ago

@samhforbes Thank you for your comments. I will discuss with co-authors and get back to you as soon as possible.

samhforbes commented 1 year ago

@editorialbot assign me as editor

editorialbot commented 1 year ago

Assigned! @samhforbes is now the editor

peilun-he commented 1 year ago

Hi @samhforbes, after discussing with all co-authors, we have made the following changes:

  1. The app is deployed on a Shiny server. Users can run the app by going to https://peilunhe.shinyapps.io/pdsim/ (the link is also provided in README file).
  2. This app is packaged into an R package so users can also download it by using the "devtools::install_github()" function and run it on their local machine.
  3. User guide and how to install the app are added into the README file. All these changes have been merged into the main branch. Thank you very much for your patience.
samhforbes commented 1 year ago

Hi @peilun-he Thanks for making these changes. I do think this has substantially improved the work you've submitted. I still want to push on the first point I raised. Specifically: I see much improved documentation, but there needs to be documentation on what the values in the app represent and what a change in those values represent. So something like a worked example where you show how to input it, or really clear vignettes (I noticed that's not populated yet so it might be on the way?) would be really helpful. Other than this I'm happy with the changes you've made, and if you are willing to add this level of documentation, then I would be happy to send it out for peer review. Is that something you would be willing to do?

peilun-he commented 1 year ago

Hi @samhforbes Thank you for your comments and suggestions. I'm going to make a vignette this week.

peilun-he commented 1 year ago

Hi @samhforbes I have added a vignette to the package, which includes some GIFs about how to use this app. The meanings of some values are explained by the GIFs, and others are defined in the user guide (which is available in both the README file and the app).

samhforbes commented 1 year ago

Hi @peilun-he I've had a look at this now and I think these changes make a big improvement to the software, and am willing to send it out for review. I have started contacting reviewers, but if there are any you can recommend from our reviewer spreadsheet in the initial post in this thread, please do so without tagging them in.

samhforbes commented 1 year ago

:wave: @AvianaGlobal, @taqtiqa-mark would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

taqtiqa-mark commented 1 year ago

@samhforbes, I'm available to review this.

samhforbes commented 1 year ago

Thanks @taqtiqa-mark!

samhforbes commented 1 year ago

@editorialbot add @taqtiqa-mark as reviewer

editorialbot commented 1 year ago

@taqtiqa-mark added to the reviewers list!

peilun-he commented 1 year ago

Hi @samhforbes, may I suggest Dr Patrick Laub (https://www.unsw.edu.au/staff/patrick-laub & https://pat-laub.github.io/) from UNSW, Australia as the second reviewer? Although Patrick is not on the list, we believe that he would be a good candidate.

samhforbes commented 1 year ago

Thanks @peilun-he I've reached out.

samhforbes commented 1 year ago

👋 @seyhunsaral @emilyriederer would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

samhforbes commented 1 year ago

👋 @bkrayfield @y1my1 @dnzoktay would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

bkrayfield commented 1 year ago

@samhforbes I am available to review this.

samhforbes commented 1 year ago

@editorialbot add @bkrayfield as reviewer

editorialbot commented 1 year ago

@bkrayfield added to the reviewers list!

samhforbes commented 1 year ago

Fab, thanks @taqtiqa-mark and @bkrayfield for agreeing to review, @peilun-he I will now open a review thread and close this one.

samhforbes commented 1 year ago

@editorialbot start review

editorialbot commented 1 year ago

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/5762.