search

pydata / xarray

N-D labeled arrays and datasets in Python
https://xarray.dev
Apache License 2.0
3.56k stars 1.07k forks source link

0.13.0 release #3257

Closed max-sixty closed 5 years ago

max-sixty commented 5 years ago

What do we think about a minor release soon?

My colleague just hit https://github.com/pydata/xarray/pull/3145 which reminded me the current release can make filing an issue confusing for folks

We've had a v good pipeline of changes in just a month: https://github.com/pydata/xarray/blob/master/doc/whats-new.rst#L18

I'm happy to help with this; ref https://github.com/pydata/xarray/issues/2998#issuecomment-516218628

shoyer commented 5 years ago

There are couple of small fixes for sparse arrays with open pull requests. I would lean toward getting those in before the release.

One thing that would be especially helpful is finalizing some deprecations that have been open for a while, e.g., the removal of the inplace argument and sel_points().

shoyer commented 5 years ago

Also it would be great if you’d like to try going through the release process. We can potentially automate and/or remove some of those steps.

max-sixty commented 5 years ago

What do you think about writing a small script and deploying with an azure pipeline, triggering only on new tags? Could include the standard running tests etc. I'd need to look into how authentication re PyPI would work if done remotely. Or is this begging the xkcd cartoon?

crusaderky commented 5 years ago

Given that's 1 hour of work about once a month... Unless there is appetite for increasing the release frequency, I would say automation is overkill

dcherian commented 5 years ago 
 4. On the master branch, commit the release in git:
      git commit -a -m 'Release v0.X.Y'
 5. Tag the release:
      git tag -a v0.X.Y -m 'v0.X.Y'
 6. Build source and binary wheels for pypi:
      git clean -xdf  # this deletes all uncommited changes!
      python setup.py bdist_wheel sdist
 7. Use twine to register and upload the release on pypi. Be careful, you can't
    take this back!
      twine upload dist/xarray-0.X.Y*
    You will need to be listed as a package owner at
    https://pypi.python.org/pypi/xarray for this to work.
 8. Push your changes to master:
      git push upstream master
      git push upstream --tags
 9. Update the stable branch (used by ReadTheDocs) and switch back to master:
      git checkout stable
      git rebase master

This bit seems easily scriptable

max-sixty commented 5 years ago

Yeah, automation is probably overkill if someone does it correctly each time – it does also mean hopefully no one makes a mistake / forgets a step

shoyer commented 5 years ago

My experience has been that the manual processes (e.g., checking release notes, finishing up deprecation cycles) take up most of the effort of issuing a new release. The commands 4-9 take about 5 minutes to run. I agree that the main value of automation there would be reducing the likelihood of manual mistakes.

It looks like there's an experimental documentation builder for pull requests with read the docs. It would be interesting to explore using it in xarray: https://blog.readthedocs.com/building-docs-for-pull-requests/

max-sixty commented 5 years ago

It looks like there's an experimental documentation builder for pull requests with read the docs. It would be interesting to explore using it in xarray: https://blog.readthedocs.com/building-docs-for-pull-requests/

Perfect - I emailed. Anything else like this, feel free to push onto us / me

dopplershift commented 5 years ago

The benefit to automation also makes it easier to distribute the workload to other people, helping with project sustainability. On my projects, I find it very nice that I make a new release on GitHub and packages appear on PyPI and the web docs are automatically updated to the new version.

dcherian commented 5 years ago

I find it very nice that I make a new release on GitHub and packages appear on PyPI and the web docs are automatically updated to the new version.

How do you do this? i.e. trigger PyPI and RTD updates with a tag on github?

dopplershift commented 5 years ago

On Travis I have some deploy hooks:

  1. Using Travis' built-in PyPI support, it uploads wheels and sdist, only on tags
  2. Execute a custom script to commit built docs to github pages (not RTD). On master builds, this updates dev docs. On a tag, it adds a new directory for that version of the docs.

Travis Config Doc Deploy Script

max-sixty commented 5 years ago

FYI: we're on the waitlist for the RTD feature

max-sixty commented 5 years ago

FYI re RTD PR feature, too many PRs! :)

We are sorry, we can't enable the pull requests builder feature for your project right now. This is because the long build time and many active PRs of your project. We have limited resources, we are working in making the builds faster. We let you know when we can enable the feature for your project.

fmaussion commented 5 years ago

RTD is struggling with resources, and I can understand that. What is really annoying (and the problem is often the same on travis) is that the conda install is taking a huge part of the build process resources. See e.g. https://github.com/readthedocs/readthedocs.org/issues/6025 , where I ended up using pip for very satisfying results (!).

crusaderky commented 5 years ago

Can't we have a nightly build of docker images with all the xarray depencencies? Then the individual CI jobs would literally become something like

FROM xarray-ci-py37
RUN pip install .
fmaussion commented 5 years ago

Can't we have a nightly build of docker images with all the xarray depencencies?

This would help on Travis yes, but not on RTD which doesn't support docker. But otherwise yes, I've also found that CI is faster and more reliable with docker. In terms of resources "in general", I wonder if it wouldn't be nice to share a common testing image base with other packages of the pydata ecosytem.

shoyer commented 5 years ago

I did some searching for "FutureWarning" and "DeprecationWarning", and turned up a bunch worth cleaning up for 0.13: https://github.com/pydata/xarray/issues/3280

Otherwise I think we're ready do a release?

max-sixty commented 5 years ago

(FYI I am on vacation — can contribute next week)

dcherian commented 5 years ago

It would be nice to get #3239 in since it fixes a longstanding issue with open_mfdataset.

dcherian commented 5 years ago

We should also merge the scipy19-docs branch since that has some nice changes (ping @rabernat).

rabernat commented 5 years ago

We should also merge the scipy19-docs branch since that has some nice changes

Ideally there are a few things we should finish:

I agree it would be nice to get these into a release. On the other hand, I feel like most doc users are just looking at the "latest" docs, so perhaps not as urgent as the other items on here.

dcherian commented 5 years ago

On the other hand, I feel like most doc users are just looking at the "latest" docs

I'm not so sure of this. The default is stable. and it isn't very obvious how to change it.

We can also merge that branch multiple times...

shoyer commented 5 years ago

Yes, we can merge/rebase changes into "stable" even after cutting a release.

On Tue, Sep 10, 2019 at 1:56 PM Deepak Cherian notifications@github.com wrote:

On the other hand, I feel like most doc users are just looking at the "latest" docs

I'm not so sure of this. The default is stable. and it isn't very obvious how to change it.

We can also merge that branch multiple times...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3257?email_source=notifications&email_token=AAJJFVV7GENIIQAJJHZ73VDQJACZPA5CNFSM4IPDKA22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6MO3VI#issuecomment-530116053, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJJFVTWYNFSRJMKFL3GOFDQJACZPANCNFSM4IPDKA2Q .

max-sixty commented 5 years ago

Anything final remaining? A couple of nice-to-haves on https://github.com/pydata/xarray/issues/3280 but otherwise I think all done?

If so, I have some time this week to help—I can coordinate

dcherian commented 5 years ago

Yes we should release ASAP. #3313 and #3314 are the last two items on the checklist.

shoyer commented 5 years ago

OK, @max-sixty or @dcherian do one of you want to take the lead on issuing the release this time? Everything should be in the HOW_TO_RELEASE checklist, but I'm sure something will come up -- nonetheless, it's good to increase the bus factor!

dcherian commented 5 years ago

OK happy to do so.

You will need to be listed as a package owner at https://pypi.python.org/pypi/xarray for this to work.

Do you want to add the rest of the core team as owners on pypi and RTD?

max-sixty commented 5 years ago

Great—happy to help @dcherian—thank you!

dcherian commented 5 years ago

OK I've done everything but

@max-sixty Can you take over emailing the users list?

  1. Issue the release announcement! For bug fix releases, I usually only email xarray@googlegroups.com. For major/feature releases, I will email a broader list (no more than once every 3-6 months): pydata@googlegroups.com, xarray@googlegroups.com, numpy-discussion@scipy.org, scipy-user@scipy.org, pyaos@lists.johnny-lin.com Google search will turn up examples of prior release announcements (look for "ANN xarray").
dcherian commented 5 years ago

@shoyer instructions in HOW_TO_RELEASE worked perfectly :)

shoyer commented 5 years ago

@dcherian (and anyone else who's interested) could you please send me your pypi and RTD account names?

max-sixty commented 5 years ago

Yes, and our Twitter account! Ping back when it's on PyPI and I'll send out

max-sixty commented 5 years ago

@shoyer I'm at maximilianroos on PyPI but maybe we lock that down tighter than the GH privileges?

shoyer commented 5 years ago

Maintainers permissions enable: "Can upload releases for a package. Cannot add collaborators. Cannot delete files, releases, or the project."

I'm pretty happy passing that out to anyone interested :)

dcherian commented 5 years ago

Hahaha, I just signed up at PyPI as dcherian

EDIT: I'm dcherian on RTD also.

shoyer commented 5 years ago

@dcherian done! you should have permissions for both RTD and pypi now

dcherian commented 5 years ago

Thanks! Done with release on PyPI: https://pypi.org/project/xarray/0.13.0/

Time to announce!

max-sixty commented 5 years ago

@dcherian did you add a tag? I don't see that yet

dcherian commented 5 years ago

https://github.com/pydata/xarray/releases/tag/v0.13.0 ?

max-sixty commented 5 years ago

Great! My mistake

jhamman commented 5 years ago

I just merged https://github.com/conda-forge/xarray-feedstock/pull/50, so the release should be available on conda-forge soon.

shoyer commented 5 years ago

Success! Thanks team!

On Tue, Sep 17, 2019 at 6:33 PM Maximilian Roos notifications@github.com wrote:

Closed #3257 https://github.com/pydata/xarray/issues/3257.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3257?email_source=notifications&email_token=AAJJFVQP33UC653KRZMYI3DQKGAQBA5CNFSM4IPDKA22YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOTV327OA#event-2641866680, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJJFVRON7ZYBJGXQN2NQDLQKGAQBANCNFSM4IPDKA2Q .

max-sixty commented 5 years ago

@pydata/xarray let me know any feedback on the email announcements