JOSS submission review

[MichalisPanayides]

This issue is part of the JOSS review. Here's a list of some comments:

• The installation process listed on the README.md along with the tests worked fine
• There's numerous bugs in The Basics section of the README.md file. All the suggested changes have been outlined in this PR
  • Would be nice to have a bit more text that describes what is happening in that section
  • Also possibly add a graph there similar to the one in the final example
• In the README.md file it would also be nice to mention the jupyter notebooks in examples/ and benchmark/ directories
  • It would be also nice to mention that some additional imports are required to run these (e.g. seaborn, yfinance)
• Would also be great to have a How to contribute section

• There seem to be some bugs when attempting to run the jupyter notebooks. e.g:

  • In examples/demo.ipynb the line sim = x.simulate(initial_state="Rainy", n_steps=500) seems to get the following error (which seems to be fixed if you run sim = x.simulate("Rainy", 500) instead). I believe there is a bug in the exceptions.py file:

    
    [38](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/examples/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/exceptions.py:38) elif func.__name__ == "simulate":
    ---> [39](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/examples/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/exceptions.py:39)     n_steps = args[2]
     [40](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/examples/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/exceptions.py:40)     initial_state = args[1]
     [41](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/examples/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/exceptions.py:41)     if not isinstance(n_steps, int) or n_steps <= 0:

  IndexError: tuple index out of range

  - In `examples/demo.ipynb` the line `new_object1.load_model("first-save.json")` also gets an error
  - In `benchmarks/benchmarks_fit.ipynb` the line `chainopy_mc.fit(words)`, under `Length = 50` also produces an error:
  ```python
  ---> [95](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/benchmarks/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/markov_chain.py:95)     raise ValueError(f"Invalid TPM {p}")
       [97](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/benchmarks/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/markov_chain.py:97) if not np.allclose(np.sum(p, axis=1), 1, atol=epsilon * len(states)):
       [98](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/benchmarks/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/markov_chain.py:98)     raise ValueError(
       [99](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/benchmarks/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/markov_chain.py:99)         f"Rows of the Transition Probability Matrix \
      [100](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/benchmarks/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/markov_chain.py:100)                             (TPM) {p} must sum to 1."
      [101](https://file+.vscode-resource.vscode-cdn.net/Users/Desktop/code/chainopy/benchmarks/~/anaconda3/envs/joss-chainopy/lib/python3.11/site-packages/chainopy/markov_chain.py:101)     )
  
  ValueError: Invalid TPM [[1.e-16 1.e-16 1.e-16 ... 1.e-16 1.e-16 1.e-16]
   [1.e-16 1.e-16 1.e-16 ... 1.e-16 1.e-16 1.e-16]
   [1.e-16 1.e-16 1.e-16 ... 1.e-16 1.e-16 1.e-16]
   ...
   [1.e+00 1.e-16 1.e-16 ... 1.e-16 1.e-16 1.e-16]
   [1.e-16 1.e-16 1.e-16 ... 1.e-16 1.e-16 1.e-16]
   [1.e-16 1.e-16 1.e-16 ... 1.e-16 1.e-16 1.e-16]]

[aadya940]

@MichalisPanayides Hello, Thanks for the detailed review, It was really helpful. The changes have been made. :))

[MichalisPanayides]

@aadya940 Thanks for addressing all my comments. I can confirm that the contents of all notebooks run fine now. Here's a couple more comments:

• Running the notebooks in the benchmarks/ directory I found that the fit method seems to not satisfy the claims presented in the paper. More precisely when running the benchmarks_fit.ipynb notebook it seems like for Length = 500, Length = 1000, Length = 5000 and Length = 10000 the fit mehod of chanopy seems to be slower than the fit_sequence method of pydtmc. Please let me know if this makes sense and/or if I have misunderstood something. (Picture of the results of Length = 500 running in a local jupyter notebook)

• Regarding the software paper I think some of the tables might need some editing. I am slightly confused about the alignment of the Number of Words/Transition-Matrix size row and the Mean/St. dev. columns. Apologies if I'm missing something there. This comment relates to the following tables:
  • is_absorbing Methods
  • stationary_dist vs pi Methods
  • fit vs fit_sequence Method

[aadya940]

Hello, it was quite strange to know this happened in the fit function. However, upon careful inspection I realized the caching decorator was missing in the fit method for some reason which was present earlier while I was writing benchmarks in the paper. Could just be a silly mistake. I assume it will work fine once this PR is merged and am attaching a screenshot below. Really sorry though.

[aadya940]

Merged the PR.

Regarding the paper, I see the issue, I'll rewrite the tables a bit. Thanks :)) @MichalisPanayides

[MichalisPanayides]

Hello, it was quite strange to know this happened in the fit function. However, upon careful inspection I realized the caching decorator was missing in the fit method for some reason which was present earlier while I was writing benchmarks in the paper. Could just be a silly mistake. I assume it will work fine once this PR is merged and am attaching a screenshot below. Really sorry though.

I just checked the changes and it looks like the fit method is now faster than the fit_sequence method 👍

Thanks for addressing the comment

[aadya940]

@MichalisPanayides Do the tables look better now? https://github.com/aadya940/chainopy/blob/master/JOSS/paper.md

[MichalisPanayides]

Yes. That looks a lot better now. Will check off the final item in the checklist now 👍