revive Pydap project

[jhamman]

Pydap is "a Python library implementing the Data Access Protocol (DAP, aka OPeNDAP or DODS)." It facilities the streaming of datasets over a network using DAP. Its development and maintenance has slowed over the past few years, despite its continued use (and support in xarray).

This proposal would do the following:

• provide a proof on concept for a pydap server backed by xarray and dask
• provide some much needed developer maintenance to the pydap project and the pydap-xarray backend (not sexy work but absolutely needed)

per: https://twitter.com/rabernat/status/1039209501482778624

NumFOCUS project: Xarray ESIP member institution: NCAR

cc @mrocklin @rabernat @shoyer

[ocefpaf]

While I'm definitely a :+1: here I wonder what is the plan to address the second bullet point:

provide some much needed developer maintenance to the pydap project and the pydap-xarray
backend (not sexy work but absolutely needed)

There is an active community around pydap, people are sending PRs, reviewing them, posting issues, etc. I guess that the main blocker would be recruiting more people with the dev privilegies, to get the issues sorted out and the PRs merged. Is someone in touch with a pydap dev?

[shoyer]

OK, maybe pydap is in better shape than I thought :). It's a been a while I followed the project closely.

On Wed, Sep 12, 2018 at 11:12 AM Filipe notifications@github.com wrote:

While I'm definitely a 👍 here I wonder what is the plan to address the second bullet point:

provide some much needed developer maintenance to the pydap project and the pydap-xarray backend (not sexy work but absolutely needed)

There is an active community around pydap, people are sending PRs, reviewing them, posting issues, etc. I guess that the main blocker would be recruiting more people with the dev privilegies, to get the issues sorted out and the PRs merged. Is someone in touch with a pydap dev?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ESIPFed/NUMfocusFallDev/issues/3#issuecomment-420745903, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1hujekuRxrapqVuYPlkXkl2mozaMks5uaU6OgaJpZM4WiT5N .

[ocefpaf]

OK, maybe pydap is in better shape than I thought :).

I did not say exactly that :smile:

At the moment there are 13 pending PRs (most from 2017!) and 31 open issues. Unfortunately the devs are unresponsive and there are some talks about forking it. That is why I asked the plan for the second item there. I would prefer to avoid forking it but we need a larger dev base to address the PRs and open issues.

[ocefpaf]

Ping @tomkralidis here who may be interested in reviving pydap.

[shoyer]

Ping @laliberte @jameshiebert

[tomkralidis]

Thanks @ocefpaf. As a dev I'm definitely motivated to bring Pydap back to life (see https://groups.google.com/forum/#!topic/pydap/p1xKzsaBaHg). Probably talking with the Pydap maintainters would be a good first step. Count me in!

[jameshiebert]

Thanks for the ping @shoyer I wholeheartedly agree that pydap needs some revival and that the devs (coughmecough) have been pretty unresponsive. Sorry, I haven't had a lot of extra bandwidth over the last year. Where should we organize ourselves?

[rsignell-usgs]

I think this is a great idea, but is it a good fit for this funding opportunity (e.g. where $6K will help)? IMHO we need to know this before we can vote it up for potential ESIP funding.

[tomkralidis]

@jameshiebert can open a Gitter channel for Pydap? Seems like a good start.

[jhamman]

pinging this issue again. @tomkralidis and @jameshiebert - do you have any thoughts on if/how a small ballast of funding could help get things moving on the pydap front again?

[tomkralidis]

@jhamman perhaps it can facilitate a code sprint of some sort, providing dedicated time to bring Pydap back to life.

[ocefpaf]

@jhamman perhaps it can facilitate a code sprint of some sort, providing dedicated time to bring Pydap back to life.

In conda-forge our plans are to facilitate code sprints by supporting the core devs, that do not have funds already, to attend conferences like SciPy and sprint there. The main advantage is that we can recruit more people than the grant alone would allow.

[tomkralidis]

@jameshiebert can open a Gitter channel for Pydap? Seems like a good start.

I can open a Pydap Gitter channel from the geopython space, however I would think it's better being opened by the Pydap project itself.

[rabernat]

however I would think it's better being opened by the Pydap project itself

Anything that requires involvement of the original pydap devs seems unlikely to happen right now. That's why we are having this conversation in the first place. Does anyone on this thread have admin rights on the pydap organization?

[tomkralidis]

I do not, but would gladly take on the role to move Pydap forward.

[shoyer]

I think only @jameshiebert has admin access. I'm a "member" which means I can merge pull requests.

[laliberte]

AFAIK, that is correct. It is not really clear how this project could go forward without a new, dedicated team. @jameshiebert, any chance you could promote someone as your replacement (not me)?

On Thu, Oct 4, 2018 at 13:43 Stephan Hoyer notifications@github.com wrote:

I think only @jameshiebert https://github.com/jameshiebert has admin access. I'm a "member" which means I can merge pull requests.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ESIPFed/NUMfocusFallDev/issues/3#issuecomment-426987575, or mute the thread https://github.com/notifications/unsubscribe-auth/ADEX_qvKoJGLQf4ELTGhv4egE7u0Ux4Jks5uhfRDgaJpZM4WiT5N .

[jameshiebert]

Hi All, I'm more-or-less AFK for personal reasons for the rest of the calendar year. I've nominated @shoyer and @tomkralidis as owners of the GitHub pydap organization in the mean time. Glad that you all are keen to move things along and hope that I can join you to help out again in the new year. All the best!

[shoyer]

James, thanks for passing on the baton!

I'm not really in a position to contribute to pydap myself (I have too many other open source commitments) but I can at least facilitate adding new maintainers.

[tomkralidis]

Thanks very much @jameshiebert. Looking forward to having you back on board soon.

I've started a Gitter room at https://gitter.im/pydap/pydap

[tomkralidis]

As well, see revival thread on ML at https://groups.google.com/forum/#!topic/pydap/ntdoUhD7KD4

Let's go!