Submission for AMS

[jinningwang]

Submitting Author: Jinning Wang (@jinningwang)
All current maintainers: @jinningwang Package Name: AMS One-Line Description of Package: Power system dispatch modeling and dispatch-dynamic co-simulation. Repository Link: https://github.com/CURENT/ams Version submitted: v0.9.6 EIC: @Batalex Editor: @NimaSarajpoor
Reviewer 1: Alireza Gholamzadeh Khoee @Alizak-Mech
Reviewer 2: Alireza Alamgir Tehrani @AlirezaAlamgir Reviewer 3: Kavian Mashayekhi @Kawians Archive: TBD JOSS DOI: TBD Version accepted: TBD Date accepted (month/day/year): TBD

---

Code of Conduct & Commitment to Maintain Package

• [x] I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package after should it be accepted.
• [x] I have read and will commit to package maintenance after the review as per the pyOpenSci Policies Guidelines.

Description

As part of CURENT Large-scale Testbed platform, AMS serves as power system production cost modeling. Our framework offers a modularized approach that seamlessly incorporates dynamics, enhancing traditional dispatch modeling methods. We create a versatile solution that bridges the gap between device-level and system-level models. The tool is developed to be extensible, scalable, compatible, and interoperable.

Scope

• Please indicate which category or categories. Check out our package scope page to learn more about our scope. (If you are unsure of which category you fit, we suggest you make a pre-submission inquiry):

  • [ ] Data retrieval
  • [ ] Data extraction
  • [x] Data processing/munging
  • [ ] Data deposition
  • [ ] Data validation and testing
  • [ ] Data visualization[^1]
  • [ ] Workflow automation
  • [ ] Citation management and bibliometrics
  • [ ] Scientific software wrappers
  • [ ] Database interoperability

Domain Specific

• [ ] Geospatial
• [ ] Education

Community Partnerships

If your package is associated with an existing community please check below:

• [ ] Astropy:My package adheres to Astropy community standards
• [ ] Pangeo: My package adheres to the Pangeo standards listed in the pyOpenSci peer review guidebook

[^1]: Please fill out a pre-submission inquiry before submitting a data visualization package.

• For all submissions, explain how the and why the package falls under the categories you indicated above. In your explanation, please address the following points (briefly, 1-2 sentences for each):

  • Who is the target audience and what are scientific applications of this package?
    Power system researchers and engineers. Power system steady state modeling and analysis.

  • Are there other Python packages that accomplish the same thing? If so, how does yours differ? There are some Python packages cover part of our functions: PYPOWER, pandapower, and PyPSA. Compared to existing tools that focus on fixed power system optimization problems, our package AMS enables customizing formulations thus enable rapid prototyping for renewables integration. Additionally, with the built-in interface with dynamic simulator ANDES, AMS allows native interoperation between dynamics and dispatch, which significantly relieves the researchers manual efforts when conducting power system simulations.

  • If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted: https://github.com/pyOpenSci/software-submission/issues/169 @Batalex

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

• [x] does not violate the Terms of Service of any service it interacts with.
• [x] uses an OSI approved license.
• [x] contains a README with instructions for installing the development version.
• [x] includes documentation with examples for all functions.
• [x] contains a tutorial with examples of its essential functions and uses.
• [x] has a test suite.
• [x] has continuous integration setup, such as GitHub Actions CircleCI, and/or others.

Publication Options

• [ ] Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements][JossSubmissionRequirements]. Be aware that completing the pyOpenSci review process **does not** guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS. - [ ] The package is not a "minor utility" as defined by JOSS's [submission requirements][JossSubmissionRequirements]: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria. - [ ] The package contains a `paper.md` matching [JOSS's requirements][JossPaperRequirements] with a high-level description in the package root or in `inst/`. - [ ] The package is deposited in a long-term repository with the DOI: *Note: JOSS accepts our review as theirs. You will NOT need to go through another full review. JOSS will only review your paper.md file. Be sure to link to this pyOpenSci issue when a JOSS issue is opened for your package. Also be sure to tell the JOSS editor that this is a pyOpenSci reviewed package once you reach this step.*

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

• [x] Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Confirm each of the following by checking the box.

• [x] I have read the author guide.
• [x] I expect to maintain this package for at least 2 years and can help find a replacement for the maintainer (team) if needed.

Please fill out our survey

• [x] Last but not least please fill out our pre-review survey. This helps us track submission and improve our peer review process. We will also ask our reviewers and editors to fill this out.

P.S. Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

The editor template can be found here.

The review template can be found here.

[Batalex]

Editor in Chief checks

Hi there! Thank you for submitting your package for pyOpenSci review. Below are the basic checks that your package needs to pass to begin our review. If some of these are missing, we will ask you to work on them before the review process begins.

Please check our Python packaging guide for more information on the elements below.

• [x] Installation The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
  • [x] The package imports properly into a standard Python environment import package.
    Just so you know, I was unable to import the package on OS X with the following error: symbol not found in flat namespace '_klu_l_analyze'. I'll report it properly later.
• [x] Fit The package meets criteria for fit and overlap.
• [x] Documentation The package has sufficient online documentation to allow us to evaluate package function and scope without installing the package. This includes:
  • [x] User-facing documentation that overviews how to install and start using the package.
  • [x] Short tutorials that help a user understand how to use the package and what it can do for them.
  • [x] API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
• [x] Core GitHub repository Files
  • [x] README The package has a README.md file with clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
  • [x] Contributing File The package has a CONTRIBUTING.md file that details how to install and contribute to the package.
  • [x] Code of Conduct The package has a CODE_OF_CONDUCT.md file.
  • [x] License The package has an OSI approved license. NOTE: We prefer that you have development instructions in your documentation too.
• [x] Issue Submission Documentation All of the information is filled out in the YAML header of the issue (located at the top of the issue template).
• [x] Automated tests Package has a testing suite and is tested via a Continuous Integration service.
• [x] Repository The repository link resolves correctly.
• [x] Package overlap The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
• [ ] Archive (JOSS only, may be post-review): The repository DOI resolves correctly.
• [ ] Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

• [ ] Initial onboarding survey was filled out We appreciate each maintainer of the package filling out this survey individually. :raised_hands: Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:

[jinningwang]

Hi @Batalex,

Thanks for your response.

Just so you know, I was unable to import the package on OS X with the following error: symbol not found in flat namespace '_klu_l_analyze'. I'll report it properly later.

We run into it before and have addressed it, https://github.com/CURENT/andes/issues/508

Another possible solution is to add the used environment to path:

export PATH="/path/to/conda/envs/ams/bin:$PATH"

[Batalex]

Hey there @jinningwang! We have an editor 🙌

@NimaSarajpoor will take over this submission, and will most likely handle ANDES after that. Happy reviewing y'all

[NimaSarajpoor]

Editor response to review:

---

Editor comments

:wave: Hi (reviewer1 @Alizak-Mech and reviewer2 @AlirezaAlamgir) and reviewer3 @kawians. Thank you for volunteering to review for pyOpenSci!

[Note] Reviewer 1 and 2 (@Alizak-Mech and @AlirezaAlamgir) will collaborate together and submit one review from their side.

Please fill out our pre-review survey

Before beginning your review, please fill out our pre-review survey. This helps us improve all aspects of our review and better understand our community. No personal data will be shared from this survey - it will only be used in an aggregated format by our Executive Director to improve our processes and programs.

• [ ] reviewer 1 survey completed.
• [ ] reviewer 2 survey completed.
• [ ] reviewer 3 survey completed.

The following resources will help you complete your review:

1. Here is the reviewers guide. This guide contains all of the steps and information needed to complete your review.
2. Here is the review template that you will need to fill out and submit here as a comment, once your review is complete.

Please get in touch with any questions or concerns! Your review is due:

Reviewers: @Alizak-Mech @AlirezaAlamgir @kawians Due date: Regular deadline 2024-08-15 (see note below)

[Note] deadline extension Yellow deadline: 2024-08-30 Red deadline: 2024-09-07

[NimaSarajpoor]

@jinningwang Still looking for reviewers. Not easy to find people with domain knowledge interested in reviewing software. I found two reviewers who seem to have domain knowledge and experience in python programming. They, however, are looking for 4-6 weeks (rather than 3 weeks). It might make sense as it is summer.

I was wondering if you are okay with that?

[jinningwang]

@jinningwang Still looking for reviewers. Not easy to find people with domain knowledge interested in reviewing software. I found two reviewers who seem to have domain knowledge and experience in python programming. They, however, are looking for 4-6 weeks (rather than 3 weeks). It might make sense as it is summer.

I was wondering if you are okay with that?

Yes, it works for me. Thanks for your efforts, especially considering it is a small crowded domain-software.

[NimaSarajpoor]

Top comment and Editor's Response are updated.

[NimaSarajpoor]

@Alizak-Mech @AlirezaAlamgir @Kawians Please gives a thumbs up (👍) on this comment so that I know if anyone is missing.

[lwasser]

hi there editorial team! i'm checking in on reviews to see what the status is. I noticed that this review hasn't had any movement. Is there anything that i can do to support? Any questions that folks have?

[NimaSarajpoor]

@lwasser I got a response from one of the reviewers. Currently, I am doing my own review. Will provide both by end of this week.

[lwasser]

@NimaSarajpoor oh gosh - my intention wasn't to make you feel the need to do a review as editor. If i can support you in another way - maybe help find a second person please let me know? Thank you so much for the speedy reply!

[NimaSarajpoor]

Here is the review provided by one group of the reviewers, @Alizak-Mech @AlirezaAlamgir, that collaborated together. The reviewers (one mechanical engineer and one energy engineer) wanted to use AMS to simulate their case study that involves a smart [power distribution] grid with several Electrical Vehicle (EV) stations, where each station has its own energy storage system. To review, the reviewers decided to use AMS for their case study. I assume the reviewers have the mathematical form of the objective function, and the constraints, and wanted to use the AMS library to find the optimal solution for their problem.

The following challenges were reported:

(1) The authors did a good job in providing examples on how this library accepts different input / output formats. However, some technical details seem to be unclear, which makes it hard for [energy / mechanical] engineers who wants to use this library.

(2) The abbreviations seem to be a bit unclear and it is not easy to understand. If users need to know the naming conventions in this library, it might be a good idea to have a table to explain them, and maybe add a link to that in the README page. It is recommended, if possible, to improve naming for the sake of readability and ease of use.

(3) The reviewers cannot understand how they should define their objective function in AMS.

(4) To motivate users to use this library, the advantages need to be indicated.

• Is it simpler?
• Is it faster?
• Is it more accurate?
• etc?

@Alizak-Mech @AlirezaAlamgir Feel free to add/correct if needed.

---

Note1: My own review, using the review template with some extra notes, will be provided in my next comment.

Note2: Since the accuracy and running time depend (highly) on the "optimization solver", this editor believes the authors of library has focused on the "simplicity" by abstracting away some complex layers one may face in building a case study using other libraries. [More on this in my own, upcoming, comment]

[NimaSarajpoor]

To fill the gap, I decided to do a review myself. Kindly find my review below. If you notice something is off, please let me know. Before that, I first want to acknowledge the fact that the case studies in power system can become very complicated, and the author(s) of library did a great job in creating a software that can handle different kinds of studies (referred to as "routines" in the library IIUC).

The package review is provided below. The parts extra notes are my additional comments..

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide*

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [ ] A statement of need clearly stating problems the software is designed to solve and its target audience in README.
• [x] Installation instructions: for the development version of the package and any non-standard dependencies in README.
• [x] Vignette(s) demonstrating major functionality that runs successfully locally.
• [x] Function Documentation: for all user-facing functions.
• [x] Examples for all user-facing functions.
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• [x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a pyproject.toml file or elsewhere.

Extra notes:

• I can see a table and figure in the README that try to provide some evidence. The information there is to show two things: (1) The result is accurate, and (2) the running time is reasonable. These two items highly depend on the optimization solver. So, I believe the purpose of the evidence was to show that it is comparable to other libraries. But, the "advantage" of using this library needs to be stated as well. If the goal of library is to provide a "simple" interface for users to handle power system case studies, then README should try to bring user's attention to it.

• The figure is a bit confusing. The legend shows one colour for MATPOWER but several colours for AMS. User cannot easily understand at what they should look there. It is better to remove unnecessary information. For instance, is the running time for AMS init necessary? IIUC, AMS init gives initial answer which is not optimal. So, is there any point in reporting that?

Readme file requirements

The package meets the readme requirements below:

• [x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

• [x] The package name
• [x] Badges for:
  • [x] Continuous integration and test coverage,
  • [x] Docs building (if you have a documentation website),
  • [x] A repostatus.org badge,
  • [x] Python versions supported,
  • [x] Current package version (on PyPI / Conda).
  • [x] License

NOTE: If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the a badge for pyOpenSci peer-review will be provided upon acceptance.)

• [x] Short description of package goals.
• [x] Package installation instructions
• [x] Any additional setup required to use the package (authentication tokens, etc.)
• [x] Descriptive links to all vignettes. If the package is small, there may only be a need for one vignette which could be placed in the README.md file.
  • [ ] Brief demonstration of package usage (as it makes sense - links to vignettes could also suffice here if package description is clear)
• [x] Link to your documentation website.
• [ ] If applicable, how the package compares to other similar packages and/or how it relates to other packages in the scientific ecosystem.
• [ ] Citation information

Extra notes:

• I can see citation in API document. Maybe create a section in README as well (maybe just add a link to that)

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider whether:

• [ ] Package documentation is clear and easy to find and use.
• [ ] The need for the package is clear
• [x] All functions have documentation and associated examples for use
• [x] The package is easy to install

Extra note:

• There is this nice table that contains different types of routines. It might be a good idea to provide a corresponding example per each routine.

• The API documentation page does not show AMS 0.9.10 properly on top left of the page.

• Regarding AMS xlsx input format, I noticed that a link is provided that takes user to a page that contains information about ANDES xlsx. I understand that the expectation from user is to click on that and go through the doc. Still, I think some common names can be defined in AMS too. For instance, maybe just copy the definition of common parameters and provide it in AMS API doc as well.

• On API doc, I started from installation and click on next... till I get to the example simulate. However, there are some gaps that should have been filled first. For instance, the example talks about Device, Model, and Routine. For instance, it says: "model refers to the device-level models". However, the definition of device, and device-level model themselves are not clear.

• Each routine is a particular study in power system that tries to find an optimal solution for an optimization problem. In my opinion, one of the most important parts of API doc is this table of routines. Because, each routine is linked to a page that clearly shows params, variables, constraints, etc. One thing that can improve it further is to add a link to an example notebook as well. Sharing that link in the README might be helpful for users.

• In API referemce, the first item is ams.system.System, and the first argument is "case". However, the definition of "case" may not be clear for all. I understand that it is an input path to the data of a case study but I believe it should have been mentioned in the description of the argument.

• Again, in ams.system, I noticed the description of system.add says: "Add a device instance for an existing model." However, I believe the definition of device / model has not yet been revealed to the user by that point. So, in the very beginning of the document, letting user know the common terminologies and their definition can be very helpful.

Functionality

• [x] Installation: Installation succeeds as documented.
• [?] Functionality: Any functional claims of the software been confirmed.
• [N/A] Performance: Any performance claims of the software been confirmed.
• [x] Automated tests:
  • [x] All tests pass on the reviewer's local machine for the package version submitted by the author. Ideally this should be a tagged version making it easy for reviewers to install.
  • [x] Tests cover essential functions of the package and a reasonable range of inputs and conditions.
• [x] Continuous Integration: Has continuous integration setup (We suggest using Github actions but any CI platform is acceptable for review)
• [x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines. A few notable highlights to look at:
  • [x] Package supports modern versions of Python and not End of life versions.
  • [x] Code format is standard throughout package and follows PEP 8 guidelines (CI tests for linting pass)

Extra note:

• Re: "Any functional claims of the software been confirmed": Too complicated to be checked. But, I ran the examples and the classes seem to be working as expected.

• Re: "Any performance claims of the software been confirmed.": By performance, if it means accuracy, I can say it is confirmed by running the examples myself. If it means the running time, then I can't confirm it since our hardwares do not match.

• I ran tests on colab using the following script:

  
  !pip install -q condacolab
  import condacolab
  condacolab.install()

!conda install -q -c conda-forge cvxopt !conda install -q conda-forge::ltbams

import ams !git clone https://github.com/CURENT/ams.git !cd ams && python -m unittest ./tests/*

### For packages also submitting to JOSS

- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements](http://joss.theoj.org/about#submission_requirements).

*Note:* Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a `paper.md` matching [JOSS's requirements](http://joss.theoj.org/about#paper_structure) with:

- [ ] **A short summary** describing the high-level functionality of the software
- [ ] **Authors:** A list of authors with their affiliations
- [ ] **A statement of need** clearly stating problems the software is designed to solve and its target audience.
- [ ] **References:** With DOIs for all those that have one (e.g. papers, datasets, software).

### Final approval (post-review)

- [ ] **The author has responded to my review and made changes to my satisfaction. I recommend approving this package.**

Estimated hours spent reviewing: 

[NimaSarajpoor]

@lwasser

Maybe help find a second person please let me know? Sure! That should be very helpful!

[Just a suggestion] According to my experience with this library, I think the focus of the new person can be more on the documentation part.

[jinningwang]

Hello, the editorial team @NimaSarajpoor @lwasser, thanks for your efforts!

Feel free to drop me a message when I can start preparing revision and response.

[NimaSarajpoor]

@lwasser
Should we wait for another reviewer? What do you think?

[lwasser]

@NimaSarajpoor why don't you post in our editorial channel. Let's bring our eic @SimonMolinsky into this conversation . It's unclear to me where this review stands! Thank you!

[SimonMolinsky]

@NimaSarajpoor Let's discuss it on our Slack, I'll try to help you, and we will move the review forward :)

[NimaSarajpoor]

@Alizak-Mech @AlirezaAlamgir Are you able to use this template to reflect the review you conducted, and share it here?

@Kawians Did you review the package? If no, can you please use this template and share it here by 2024-Nov-10?

[Kawians]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [ ] A statement of need clearly stating problems the software is designed to solve and its target audience in README.
• [x] Installation instructions: for the development version of the package and any non-standard dependencies in README.
• [x] Vignette(s) demonstrating major functionality that runs successfully locally.
• [x] Function Documentation: for all user-facing functions.
• [x] Examples for all user-facing functions.
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• [x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a pyproject.toml file or elsewhere.

Peer Review Notes

• Problem Definition: While it mentions AMS's functionality (stability-constrained modeling and credible scheduling), the README lacks a clear, concise statement on the specific problems AMS solves. For example, defining challenges in power system stability, cost optimization, or integration with existing simulation engines would clarify the exact problems it addresses.
• Target Audience: Although implied, the README does not directly specify the target audience (e.g., researchers in power systems, energy market analysts, or grid operators). Explicitly stating who would benefit from AMS could strengthen this section.

Readme file requirements The package meets the readme requirements below:

• [x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

• [x] The package name
• [x] Badges for:
  • [x] Continuous integration and test coverage,
  • [x] Docs building (if you have a documentation website),
  • [ ] A repostatus.org badge,
  • [x] Python versions supported,
  • [x] Current package version (on PyPI / Conda).

Peer Review Note:

• Repostatus badge could be added to the readme file.

NOTE: If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the a badge for pyOpenSci peer-review will be provided upon acceptance.)

• [x] Short description of package goals.
• [x] Package installation instructions
• [ ] Any additional setup required to use the package (authentication tokens, etc.)
• [x] Descriptive links to all vignettes. If the package is small, there may only be a need for one vignette which could be placed in the README.md file.
  • [ ] Brief demonstration of package usage (as it makes sense - links to vignettes could also suffice here if package description is clear)
• [x] Link to your documentation website.
• [x] If applicable, how the package compares to other similar packages and/or how it relates to other packages in the scientific ecosystem.
• [ ] Citation information

Peer Review Notes:

• There is no demonstration of package usage in the README. Adding a simple code snippet or example would make it clearer for users.
• The Citation to your work could be found on your Documentation web page. Just as a suggestion, it could be added to your Readme file if needed as well.

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider whether:

• [x] Package documentation is clear and easy to find and use.
• [x] The need for the package is clear
• [x] All functions have documentation and associated examples for use
• [ ] The package is easy to install

Peer Review Note:

• The documentation is well-structured, featuring a clear table of contents that includes sections such as "Getting started," "Examples," "Development," "Release notes," "Routine reference," "Model reference," and "API reference." This organization facilitates easy navigation and access to relevant information. The "Getting Started" section provides concise installation instructions and links to further resources, enhancing user accessibility.

Functionality

• [ ] Installation: Installation succeeds as documented.
• [ ] Functionality: Any functional claims of the software been confirmed.
• [ ] Performance: Any performance claims of the software been confirmed.
• [x] Automated tests:
  • [x] All tests pass on the reviewer's local machine for the package version submitted by the author. Ideally this should be a tagged version making it easy for reviewers to install.
  • [x] Tests cover essential functions of the package and a reasonable range of inputs and conditions.
• [x] Continuous Integration: Has continuous integration setup (We suggest using Github actions but any CI platform is acceptable for review)
• [x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines. A few notable highlights to look at:
  • [x] Package supports modern versions of Python and not End of life versions.
  • [x] Code format is standard throughout package and follows PEP 8 guidelines (CI tests for linting pass)

Peer Review Note:

• Installation Challenge: I had difficulties installing the package using pip install on the Mac intel series in Late 2015. It took forever, caused the laptop to get very hot, and frozen, and the package never got installed. However, the conda install worked properly (but again for a bit of a long time about 40 minutes if I remember right). To make sure it is not my computer's problem I tried it on a newer version of the Mac intel series and it worked properly. However it seems it could be only my computer issue, I decided to bring this up with you to further investigate it if needed.

• Functionality and Performance: I second what Nima mentioned. The examples work as expected, but it is hard to generalize the functionality and performance.

For packages also submitting to JOSS

• [ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

• [ ] A short summary describing the high-level functionality of the software
• [ ] Authors: A list of authors with their affiliations
• [ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
• [ ] References: With DOIs for all those that have one (e.g. papers, datasets, software).

[Kawians]

@NimaSarajpoor Thanks for the reminder and sorry for being late in getting back to you on this. My review is ready and could be seen above.