search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[REVIEW]: Synthia: multi-dimensional synthetic data generation in Python #2863

Closed whedon closed 3 years ago

whedon commented 3 years ago

Submitting author: @dmey (D. Meyer) Repository: https://github.com/dmey/synthia Version: v1.1.0 Editor: @oliviaguest Reviewer: @khinsen, @mnarayan Archive: 10.5281/zenodo.5358432

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b"><img src="https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b/status.svg)](https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@khinsen & @mnarayan, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @oliviaguest know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @khinsen

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @mnarayan

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @khinsen, @mnarayan it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago

PDF failed to compile for issue #2863 with the following error:

Can't find any papers to compile :-(

dmey commented 3 years ago

@whedon generate pdf from branch joss-paper

whedon commented 3 years ago 
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

oliviaguest commented 3 years ago

Hi all! 👋 Thank you so much, @khinsen, @mnarayan for accepting to review this. Please read the instructions above. Any questions, feedback on the paper, etc., please post here. Any very code-specific questions, suggestions, etc., please use the issues in the code repo and link to them from this thread so we can all keep track of them. 🌸

For an example of how this process plays out feel free to skim previous reviews, such as: #2285 and #2348. ☺️

oliviaguest commented 3 years ago

@whedon check references from branch joss-paper

whedon commented 3 years ago 
Attempting to check references... from custom branch joss-paper
whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1201/b17116 is OK
- 10.1109/DSAA.2016.49 is OK
- 10.13140/2.1.4476.8963 is OK
- 10.1109/ICDE.2008.4497436 is OK
- 10.5334/jors.148 is OK
- 10.5281/zenodo.4287554 is OK
- 10.5281/zenodo.4288292 is OK

MISSING DOIs

- None

INVALID DOIs

- None
khinsen commented 3 years ago

Question to @dmey, @tnagler, @letmaik: I am investigating authorship as required by JOSS reviewing guidelines. There is no doubt that @dmey is the main contributor to this project. Of the two other contributors, whose contributions look roughly equal in importance from the "size of commits" point of view, @tnagler is a co-author whereas @letmaik is merely acknowledged for "comments and contributions". If that's OK with all of you, it's fine with me as well of course, but it looks surprising.

letmaik commented 3 years ago

@khinsen That's fine, we discussed this offline in advance, no problem there.

dmey commented 3 years ago

@khinsen it's good that you picked this up as it may appear a bit confusing -- all commits during development were squashed therefore the current metadata is not very indicative of the amount of work and contributions made during development. @tnagler made substantial contributions in the conceptual and practical development of the tool. @letmaik gave us suggestions and made contributions to the project.

whedon commented 3 years ago

:wave: @khinsen, please update us on how your review is going.

whedon commented 3 years ago

:wave: @mnarayan, please update us on how your review is going.

khinsen commented 3 years ago

Thanks @dmey and @letmaik! I realize that GitHub statistics share the problem of bibliometry in being superficial.

khinsen commented 3 years ago

@whedon Going fine, thanks for asking!

whedon commented 3 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands
khinsen commented 3 years ago

OK, I am being mean in talking like that to a bot, but I really don't know what I am expected to do in reply to the request for "updating us"!

oliviaguest commented 3 years ago

@khinsen the bot is just checking to see you are indeed doing the review, exactly like a reminder from a more "traditional" journal in your email inbox. Thus, you are able to ignore as you're already doing it.

If you are curious what commands @whedon does accept, I am sure you can figure that out too — haha: https://joss.readthedocs.io/en/latest/whedon.html

khinsen commented 3 years ago

@oliviaguest The checklist asks "Do references in the text use the proper citation syntax?", referring to a description of Markdown syntax for citations. What I have for review is a PDF file. I don't quite understand what I am supposed to do. Should I be looking at some Markdown source instead? If so, where can I find it?

khinsen commented 3 years ago

@dmey After going through the checklist for the software, here comes my review of the paper itself. Overall, it looks very good, there is just one point I would like you to address:

"Do the authors describe how this software compares to other commonly-used packages?"

No. Please add a statement on this question, even if it is only "we are not aware of any other comparable software". The Wikipedia page on Synthetic Data lists one Open Source package (DataGenerator) and one publicly available package without a licence (Dataset Generator), in addition to proprietary software.

oliviaguest commented 3 years ago

@khinsen do you mean where is the paper being compiled from? Here: https://github.com/dmey/synthia/tree/joss-paper

khinsen commented 3 years ago

@oliviaguest Exactly. Thanks for the pointer, I hadn't considered looking for a specific branch. Perhaps the review instructions should contain that pointer, given that reviewers are expected to check the Markdown source.

oliviaguest commented 3 years ago

@khinsen I thought (wrongly!) it was obvious when we ask for @whedon to do this:

@whedon generate pdf from branch joss-paper

But either way — it's no problem at all and I am more than happy to explain it (there is so much to process for those who have never used JOSS before anyway). ☺️

khinsen commented 3 years ago

@oliviaguest Everything is obvious if you do it often enough! Authors and editors are a lot more familiar with @whedon than reviewers, and for now that has been my only role in JOSS. We can usually do our jobs without ever talking to @whedon. Which I think is great, reviewing for JOSS is a real pleasure because of the absence of technical boilerplate tasks.

oliviaguest commented 3 years ago

@khinsen Yes, reviewers do not need to interact with @whedon at all, indeed. But if you want to there is a list of commands you may run. In practice, it's basically just editors (and sometimes authors) who need to ask for the bot to do anything.

@mnarayan would you be able to give a rough ETA, please? 😊

dmey commented 3 years ago

@khinsen many thanks for your feedback -- apologies for getting back just now but I generally like to respond to the two reviews in one sweep. @oliviaguest would you have any updates on the status of the second review please?

oliviaguest commented 3 years ago

@dmey I will send a message to @mnarayan but (!) I also think it's important to be a bit understanding with reviewers and their time. So I am going to ask Manjari to get back to us, but given both pandemic and holidays I don't want to push too much right now. I hope this makes sense and is OK for everybody involved. 😊

oliviaguest commented 3 years ago

Oh, and of course... @dmey what you can do is work on the changes @khinsen might have highlighted, if you want to do something (that might help) to speed this along.

mnarayan commented 3 years ago

Hi @dmey and @oliviaguest, Apologies for the delayed response here. I should be able to provide some feedback by Dec 28th and be done by Jan 4th.

mnarayan commented 3 years ago

I am not able to check things off on the review checklist. Any idea what the problem could be?

danielskatz commented 3 years ago

@whedon re-invite @mnarayan as reviewer

whedon commented 3 years ago

OK, the reviewer has been re-invited.

@mnarayan please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

danielskatz commented 3 years ago

👋 @mnarayan - the original invitation to the JOSS organization may have expired - please use this link ☝️ to join, then you should be able to interact with the checklist.

dmey commented 3 years ago

@whedon generate pdf from branch joss-paper

whedon commented 3 years ago 
Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

dmey commented 3 years ago

@dmey After going through the checklist for the software, here comes my review of the paper itself. Overall, it looks very good, there is just one point I would like you to address:

"Do the authors describe how this software compares to other commonly-used packages?"

No. Please add a statement on this question, even if it is only "we are not aware of any other comparable software". The Wikipedia page on Synthetic Data lists one Open Source package (DataGenerator) and one publicly available package without a licence (Dataset Generator), in addition to proprietary software.

@khinsen in the meantime I have updated the paper and added two recent data generation tools at the end of the second paragraph -- the ones you suggested do not appear to be maintained. I have also removed the example application as I think it did not add much to the paper (it's already on the website and readme) and were issues with the layout.

dmey commented 3 years ago

@mnarayan would you have any update re this please?

dmey commented 3 years ago

@oliviaguest would you have any updates on the status of the second review please?

oliviaguest commented 3 years ago

@dmey I'm sorry this is taking a while, but we need to be aware that this is a very tough period for a lot of people. Hopefully, this will get reviewed soon enough — and I apologize that it might be causing you stress!

dmey commented 3 years ago

@oliviaguest this is not causing me any stress -- my question was simply about being able to estimate some time for completion as I need to cite this work in 2 papers. However if this is not possible, that's no problems and I will deposit the summary paper on arXiv for the meantime...

oliviaguest commented 3 years ago

@dmey thanks for understanding and good luck!

mnarayan commented 3 years ago

@dmey This looks like a wonderful package. One that I might use myself in the near future!

I've opened a few issues with respect to the following

oliviaguest commented 3 years ago

@dmey @mnarayan please (maybe when you go through them, but sooner over later) link to each issue that is opened in the software repo from here, so we can keep track of the required changes and comments.

mnarayan commented 3 years ago

@oliviaguest I've updated my review above with links to the issues.

oliviaguest commented 3 years ago

👋 @dmey have you had time to check any of the feedback from the reviewers?

dmey commented 3 years ago

Hello, thanks for the feedback @mnarayan and sorry for not getting back to you yet @mnarayan and @oliviaguest. I will not have time to look at these issue for the next couple weeks but will try to address those the week after that if that's OK!?

oliviaguest commented 3 years ago

Totally fine, let me know when you are back on this! I assume soon! 😊

dmey commented 3 years ago

@whedon generate pdf from branch joss-paper