xarray-contrib / xskillscore

Metrics for verifying forecasts
https://xskillscore.readthedocs.io/en/stable/
Apache License 2.0
225 stars 40 forks source link

Write xs JOSS paper #344

Open aaronspring opened 3 years ago

aaronspring commented 3 years ago

very easy to create a CITATION.cff if the software is on zenodo (which xs is not yet): https://github.com/citation-file-format/doi2cff

xskillscore is actually used in a few studies already:

Another option would be a JOSS paper, but that takes a bit more effort.

see https://github.com/pangeo-data/climpred

raybellwaves commented 3 years ago

You know how to upload to zendo? I got as far as here. I took a look at https://zenodo.org/record/5038065#.YQiYLVNKh6o

Screen Shot 2021-08-02 at 9 12 11 PM
aaronspring commented 3 years ago

I would use the zenodo GitHub access https://guides.github.com/activities/citable-code/

raybellwaves commented 3 years ago

cheers. Looks like it needs a release to push

aaronspring commented 3 years ago

Or would anyone else be interested in writing a JOSS paper? @raybellwaves @bradyrx @dougiesquire

bradyrx commented 3 years ago

I think that could be a good thing to do, since it's a pretty short write-up. I could be available mostly to proof-read/contribute to an existing draft.

dougiesquire commented 3 years ago

I'd also be happy to help out. Perhaps it makes sense for @raybellwaves to lead (if he wants to/has time) given that it's his baby originally?

raybellwaves commented 3 years ago

on zenedo now. +1 on JOSS paper. I'm happy for whoever writes it to be lead author.

aaronspring commented 3 years ago

Who would be a co-author? Everyone in https://github.com/xarray-contrib/xskillscore/graphs/contributors?

bradyrx commented 3 years ago

For reference, here's the climpred JOSS paper: https://joss.theoj.org/papers/10.21105/joss.02781. That is basically at the length limit, so it's effectively a pretty short paper.

raybellwaves commented 3 years ago

Who would be a co-author? Everyone in https://github.com/xarray-contrib/xskillscore/graphs/contributors?

Would say the core devs: you, me, Riley, Dougie

aaronspring commented 3 years ago

What would be the criterion? More than one/5 feature/PRs merged?

raybellwaves commented 3 years ago

What would be the criterion? More than one/5 feature/PRs merged?

I think we can filter people > 1 PR (edited). So the author list can be:

Spring (assuming you write it :) ), Bell, Brady, Squire and Huang.

aaronspring commented 3 years ago

being > 1 PR

aaronspring commented 3 years ago

I started a first draft. I took the structure from our climpred paper https://joss.theoj.org/papers/10.21105/joss.02781.

https://sharelatex.gwdg.de/6916457693cyyrcvjcmqsw

Do we meet the JOSS requirements? I think so.

This is just a very very bare-bone first sketch. In terms of structure and content, do you feel something is missing or misleading? What would you add/remove? Please feel free to comment and also modify the text. (overleaf/sharelatex is anyways under version control so we could revert anything.)

@raybellwaves @bradyrx @dougiesquire @ahuang11

raybellwaves commented 3 years ago

Thanks @aaronspring. I think it meets the requirements. Worth adding it's domain agnostic e.g. https://xskillscore.readthedocs.io/en/stable/tabular-data.html. The accessor method allows it to be dropped in a pandas/scikit-learn workflow on tabular data. I wonder if this could be a separate section? i.e. it's use in ML competitions. AFAICT no one has used it for tabular data on kaggle (https://www.kaggle.com/search?q=xskillscore) but i'm sure xss can play a part in the convos on https://www.kaggle.com/search?q=metrics. I know it's used widely in https://s2s-ai-challenge.github.io/

dougiesquire commented 3 years ago

Sorry for the long delay in responding @aaronspring, I've been on leave. What you have looks like a great start to me, but I'll take a better look over the coming days. How would you like to progress? Provide feedback here for you to add into the document? Or just add directly into your document ourselves?

aaronspring commented 3 years ago

just add directly into your document ourselves