search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
707 stars 37 forks source link

[REVIEW]: FaaSr: R Package for Function-as-a-Service Cloud Computing #7027

Open editorialbot opened 1 month ago

editorialbot commented 1 month ago

Submitting author: !--author-handle-->@spark0510<!--end-author-handle-- (Sungjae Park) Repository: https://github.com/FaaSr/FaaSr-package Branch with paper.md (empty if default branch): Version: v1.3.0 Editor: !--editor-->@danielskatz<!--end-editor-- Reviewers: @tyson-swetnam, @yadudoc, @gvegayon Archive: Pending

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/6646d02b73aad6e2b23e25301864e16c"><img src="https://joss.theoj.org/papers/6646d02b73aad6e2b23e25301864e16c/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/6646d02b73aad6e2b23e25301864e16c/status.svg)](https://joss.theoj.org/papers/6646d02b73aad6e2b23e25301864e16c)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@tyson-swetnam & @yadudoc & @gvegayon, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @tyson-swetnam

📝 Checklist for @yadudoc

📝 Checklist for @gvegayon

editorialbot commented 1 month ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 month ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.48550/arXiv.1810.09679 is OK
- 10.1145/3127479.3128601 is OK
- 10.1145/3369583.3392683 is OK
- 10.32614/CRAN.package.httr is OK
- 10.32614/CRAN.package.paws is OK
- 10.1145/2872427.2883029 is OK
- 10.32614/CRAN.package.arrow is OK
- 10.1002/ecs2.2567 is OK
- 10.48550/arXiv.2001.10641 is OK

MISSING DOIs

- No DOI given, and none found for title: S3
- No DOI given, and none found for title: The Object Store for AI Data Infrastructure
- No DOI given, and none found for title: Lambda
- No DOI given, and none found for title: Github Actions
- No DOI given, and none found for title: Open Source Serverless Cloud Platform
- No DOI given, and none found for title: Curl: a gentle slope language for the Web.
- No DOI given, and none found for title: Using cron and crontab
- No DOI given, and none found for title: FaaSr-Docker Repository
- No DOI given, and none found for title: FaaSr JSON-Builder

INVALID DOIs

- None
editorialbot commented 1 month ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.03 s (1731.2 files/s, 231527.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                               33            695           1496           2986
JSON                             2              1              0            274
Markdown                         5             78              0            244
TeX                              1             17              0            145
YAML                             4             15              6            116
Rmd                              9            384            727             38
-------------------------------------------------------------------------------
SUM:                            54           1190           2229           3803
-------------------------------------------------------------------------------

Commit count by author:

   318  Sungjae Park
    97  github-actions[bot]
    35  Renato Figueiredo
    11  nan mu
     1  Cayelan Carey
     1  Nan Mu
     1  Yun-Jung
     1  renatof
editorialbot commented 1 month ago

Paper file info:

📄 Wordcount for paper.md is 1557

✅ The paper includes a Statement of need section

editorialbot commented 1 month ago

License info:

🟡 License found: Other (Check here for OSI approval)

editorialbot commented 1 month ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

danielskatz commented 1 month ago

@tyson-swetnam and @yadudoc and @gvegayon - Thanks for agreeing to review this submission. This is the review thread for the paper. All of our communications will happen here from now on.

As you can see above, you each should use the command @editorialbot generate my checklist to create your review checklist. @editorialbot commands need to be the first thing in a new comment.

As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#7027 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me know if either of you require some more time. We can also use editorialbot (our bot) to set automatic reminders if you know you'll be away for a known period of time.

Please feel free to ping me (@danielskatz) if you have any questions/concerns.

Also, I note that you that @yadudoc won't be able to start until the week of 5 Aug

danielskatz commented 1 month ago

@editorialbot add @gvegayon as reviewer

editorialbot commented 1 month ago

@gvegayon added to the reviewers list!

danielskatz commented 1 month ago

👋 @gvegayon - please see the instructions above to get started.

danielskatz commented 1 month ago

👋 @tyson-swetnam and @yadudoc and @gvegayon - you all should use the command @editorialbot generate my checklist to create your review checklist to start your work on the review. @editorialbot commands need to be the first thing in a new comment.

yadudoc commented 1 month ago

Review checklist for @yadudoc

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

tyson-swetnam commented 1 month ago

Review checklist for @tyson-swetnam

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

tyson-swetnam commented 1 month ago

the repository LICENSE file is not compliant yet, it only states:

YEAR: 2024
COPYRIGHT HOLDER: FaaSr authors
yadudoc commented 1 month ago

@tyson-swetnam Interesting, the package has a README.md file which does get picked up by github.

danielskatz commented 1 month ago

I think this is partly related to CRAN rules, where the full license is in LICENSE.md

gvegayon commented 1 month ago

Review checklist for @gvegayon

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

danielskatz commented 1 month ago

I think this is partly related to CRAN rules, where the full license is in LICENSE.md

I can now confirm that this is acceptable.

danielskatz commented 3 weeks ago

👋 @tyson-swetnam and @yadudoc and @gvegayon - How are your reviews coming along?

tyson-swetnam commented 3 weeks ago

Thanks for the reminder. I'll finish it this AM.

yadudoc commented 3 weeks ago

@danielskatz The FaaSr repo points to a tutorial repo for instructions on running locally. I followed the instructions for local docker deploy with Rocker and got stuck. I'm going to give the Posit cloud alternative a try, hopefully by the weekend.

gvegayon commented 3 weeks ago

Will finalize tomorrow.

tyson-swetnam commented 3 weeks ago

In the FaaSr/FaaSr-tutorial there are user's "My_Minio_Bucket_ACCESS_KEY" and "My_MinioBucket_SECRET_KEY" committed publicly. Should remove these and put "REPLACE_WITH_YOUR_MINIO_SECRET_KEY".

But I guess, this is part of the tutorial? https://github.com/FaaSr/FaaSr-tutorial?tab=readme-ov-file#configure-the-faasr-secrets-file-with-your-github-token

https://github.com/FaaSr/FaaSr-tutorial/blob/main/faasr_env

tyson-swetnam commented 3 weeks ago

Open Storage Network is mentioned as potential storage, but are no tutorial instructions for setting this up (it would be highly similar to S3, I'm guessing).

danielskatz commented 3 weeks ago

👋 @tyson-swetnam - thanks for putting in comments here. One way to track them so they don't get lost would to create issues in the source repo, and to mention openjournals/joss-reviews#7027 in the, so that a little breadcrumb is created here that tracks if they are open or closed.

tyson-swetnam commented 3 weeks ago

Understood. I was also wondering where to put my overall comments for the author(s)?

danielskatz commented 3 weeks ago

Either here or there would work.

danielskatz commented 1 week ago

👋 @yadudoc - I notice a few unchecked items in your review. Are there things @spark0510 needs to do beyond the two issues you opened, or just things you haven't yet checked?

danielskatz commented 1 week ago

👋 @gvegayon - Will you be able to start on your review soon?

danielskatz commented 1 week ago

@spark0510 - I see a couple of comments above from @tyson-swetnam that need to be addressed. Alternatively, @tyson-swetnam could create issues in the source repo as @yadudoc has done, tagging them with this review, which can make them easier to track...

spark0510 commented 1 week ago

Hi! @tyson-swetnam, thank you so much for your precious review! The tutorial is using Minio "playground", which its keys are open to public and has no need to set things(permissions and so on) up. Therefore, as a part of the tutorial, we let users to use Minio playground.

Setting up S3, Minio, and Open storage network is introduced on the manual: https://faasr.io/_pages/credentials.html Manuals are also included in the CRAN package.

We expected that users will be taking a look at manuals once they successfully complete the tutorial. Though, we can put the link on the tutorial!

yadudoc commented 1 week ago

@danielskatz The repo and the project appear to be in great shape. Kudos to @spark0510 and I wanted to express my appreciation for all the help I've been getting from @renatof with debugging some issues with the tutorial 👍

I'm comfortable signing off on the documentation and the article. The last hold up is getting a minimal example/tutorial working on my end, and my debugging efforts are documented in this issue: https://github.com/FaaSr/FaaSr-package/issues/108. I'm waiting for this to be resolved to cross off Functionality and Automated tests from the checklist.

danielskatz commented 1 week ago

👋 @gvegayon - Will you be able to start on your review soon?

gvegayon commented 6 days ago

Hi @danielskatz, thanks for the reminder. I am working on it as I'm writing. Thank you for your patience.

gvegayon commented 6 days ago

Hi @spark0510, just finished my review here. The package is looking great! Documentation looks great and I love you have GHA for the CI up and running. Here are some points in which the package+submission could improve (details in the GitHub issue):

  1. It is unclear what the contribution is of all authors. I can understand that sometimes coauthors don't make direct commits to the code, or they made other types of contributions (which is OK!) but I would like some clarification on that.
  2. There are two other R packages on CRAN that can be used with AWS lambdas: aws.lambda and lambdr. I don't see those mentioned in the package. How do those compare to your project?
  3. There is no enough code coverage of the project. This is a complicated issue as writing tests can take significant time. Currently, the package shows about 9% coverage. I would encourage you to increase that and at least ensure the core function(s) work as expected. I would recommend writing a GHA workflow in which you make a simple call using the core functionality of the package. You don't need 100%, but at least make sure the core functionality is tested properly.

There are other technical issues that I describe in my review (including a minor PR addressing some quick fixes). Happy to continue the discussion in the associated issue.

danielskatz commented 4 days ago

@tyson-swetnam - it looks like you've checked off everything but performance. What do you need in order to check that one too?

danielskatz commented 4 days ago

@yadudoc - I see a couple of unchecked items in your list. What do you need in order to check them off too?

tyson-swetnam commented 4 days ago

It can be checked off -- i wasn't able to assess performance.

danielskatz commented 4 days ago

It can be checked off -- i wasn't able to assess performance.

So you are ok with this being published now?

yadudoc commented 4 days ago

@danielskatz I have not been able to run the tutorial (https://github.com/FaaSr/FaaSr-tutorial) which I assumed was the easiest path to confirm the claimed functionality. That's the only hold-up at this point. I've had a few exchanges with the developers on this issue (https://github.com/FaaSr/FaaSr-package/issues/108) but I've not had much luck. I'm happy to check this off if any of the reviewers have been able to verify functionality.

danielskatz commented 4 days ago

@tyson-swetnam & @gvegayon - Can you explain what you did to check the functionality? Did you run the tutorial?