jupyter / enhancement-proposals

Enhancement proposals for the Jupyter Ecosystem
https://jupyter.org/enhancement-proposals
BSD 3-Clause "New" or "Revised" License
115 stars 65 forks source link

Elyra Incorporation #90

Closed lresende closed 1 year ago

lresende commented 2 years ago

We propose to make Elyra an official Jupyter subproject.

fperez commented 2 years ago

First, thanks a lot for proposing this, and for the amazing work the entire Elyra team has done! It's a great contribution to the growing ecosystem of Jupyter tools for various use cases, with impressive functionality.

Now - what follows is just my personal take, not in any way an "official" stance, and I'd be happy to hear other voices. I also want to caveat this with the fact that I don't personally use Elyra, so this is a high-level perspective, and I'd be happy to be corrected on any misconceptions.

That said - my impression is that pulling Elyra in as a core Jupyter project would be a stretch: Jupyter isn't an ML-specific tool (while many people do ML with it), and Elyra is a fairly complex codebase aimed at this particular use case. It takes advantage wonderfully of various aspects of Jupyter, but for example maintaining pieces of it would probably be a tough stretch for any "regular" Jupyter developer who was not part of the Elyra team.

Having said that, as a project, we absolutely want to be a great foundation for projects and communities like Elyra, regardless of how specialized their use cases may be!

So I would ask instead:

I think all of the above, from software to community work, are fair game to improve the experience and processes for Elyra users and developers. But for the Jupyter teams, who are already stretched pretty thin, I think it would be a strategically risky stretch to absorb this maintenance and development commitment.

Again - I hope this does not come across in any way as negative towards Elyra or the team: the project is awesome, and the team has been a wonderful citizen of the Jupyter community. But we should find ways to continue building on that track record without necessarily taking on potentially unsustainable responsibilities both towards Elyra and the rest of Jupyter.

davidbrochart commented 2 years ago

I just wanted to mention that there is kind of a similar situation with IPython and IPython Parallel: while the former is a general tool that can be used to do parallel computing, the latter does it by taking advantage of IPython's architecture (more precisely the kernel protocol, if I understand correctly). IPython Parallel started before Dask, which shares the same goal, so there might be historical reasons, but still, ipyparallel is part of the ipython project and Dask is not.

westurner commented 2 years ago

The mission of NumFOCUS is to promote open practices in research, data, and scientific computing by serving as a fiscal sponsor for open source projects and organizing community-driven educational programs.

NumFOCUS is a 501(c)(3) public charity in the United States.

The mission of LF AI & Data is to build and support an open artificial intelligence (AI) and data community, and drive open source innovation in the AI and data domains by enabling collaboration and the creation of new opportunities for all the members of the community

lresende commented 1 year ago

Thank you. Elyra jas joined the Linux Foundation for open governance.