Transitioning the Blaze project

[teoliphant]

Continuum Analytics (now Anaconda) no longer has funding for Blaze. The number of Blaze developers is small and not very active. Blaze was a very ambitious project and generated some really good ideas. Some of the ideas could continue to live on perhaps as other projects. There are now other libraries or combinations of libraries that can mostly handle the use-cases Blaze envisioned.

Other projects where the ideas of Blaze live on include:

• Dask
• Numba
• Ibis
• Intake
• xnd

We should use this thread to discuss homes for sub-parts of Blaze. For example, datashape is used by the Plures (xnd) project and should likely live there.

Odo is still an interesting "generalized copy" command that could be re-factored to use Intake

Is there anyone else interested in picking up Blaze development or parts of the Blaze list of projects?

[teoliphant]

Quansight is willing to take up the charge of ensuring Blaze ideas have a good home (in collaboration with Anaconda). In particular, we will want to be sure that "odo" can live on --- likely using Intake as its data-ingest API.

Also, there are some good ideas that Anaconda open-sourced that used to be called IOPro that we will continue to find a home for.

If others have opinions on this, please jump in.

[teoliphant]

As a first step, I'm going to pair down the devs associated with @blaze/dev. If you are eager to remain a Blaze dev or assist with the transition, please let me know.

[llllllllll]

Hey, I am glad to see there is more recent interest in blaze. At Quantopian, we are still active users of blaze, but we haven't needed to make any big changes recently. We are maintaining the components that we use, but we don't really have the bandwidth to support all of the backends. The backends that we use heavily are the postgres backend, the pandas/numpy backend, and the blaze server. In my experience, the only other backends that get much use or maintenance are the other SQL backends, particularly MSSQL.

We use both odo and blaze as part of our user facing APIs, so we have a strong desire to preserve backwards compatibility in those two libraries. We have also built extensions like warp prism that use the odo registration API. Given the relative stability of odo, if there's a desire to make major backwards-incompatible changes to odo, we'd prefer for those to happen in a fork (e.g. odo2) rather than in the original repo.

If we are looking for a new home for at least blaze and odo, Quantopian would be willing to take them. If we took over these projects we would remove the blaze backends that are unused so that we could release a tested, stable version. If people wanted to add them back and actually maintain them that would be great, but we think that removing unused code would be better than leaving it there but broken.

[postelrich]

We could split the various blaze backends into separate respective packages. That would ease the maintenance/development of blaze. Unused backends could be allowed to go defunct without holding back the core of blaze.

[teoliphant]

This is a good idea. I think we are looking for what would take the least amount of work at this point. Joe, makes a good proposal that I think we can use to keep the Blaze org active for awhile --- but I'm also OK with moving the Blaze and odo projects to Quantopian repositories and removing the Blaze org.

-Travis

On Mon, Jun 4, 2018 at 11:49 PM postelrich notifications@github.com wrote:

We could split the various blaze backends into separate respective packages. That would ease the maintenance/development of blaze. Unused backends could be allowed to go defunct without holding back the core of blaze.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/blaze/blaze/issues/1669#issuecomment-394580702, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPjoBLNHHmMB0uNJA2CpAfXtlAma3Gyks5t5g3vgaJpZM4T9AT_ .

[llllllllll]

My personal preference would be to leave the Blaze organization. This group could hold repos for stable core and the split out backends. We use the Quantopian github organization for many things so it would not really be possible to make other people admins, but if we just use the blaze org we could.

[teoliphant]

Fair enough. This makes sense.

On Tue, Jun 5, 2018, 9:06 AM Joe Jevnik notifications@github.com wrote:

My personal preference would be to leave the Blaze organization. This group could hold repos for stable core and the split out backends. We use the Quantopian github organization for many things so it would not really be possible to make other people admins, but if we just use the blaze org we could.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/blaze/blaze/issues/1669#issuecomment-394722119, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPjoPSTUaJgi0dvaTPLnyr9UPpW9D1-ks5t5pBYgaJpZM4T9AT_ .

[postelrich]

https://github.com/blaze/blaze/issues/1670

[westurner]

Re: the relation between Ibis and Blaze+Dask https://github.com/ibis-project/ibis/issues/2294#issuecomment-663053479

[mujina93]

Has there been any update on the maintenance of the project? Should it be marked as archived or something similar? (At least to warn people that approach it for the first time, try spending some time on it, and then discover that it's not maintainted)

[maxall41]

I would recommend taking a look at Lance