Current status of leaf?

anp commented 8 years ago

Which is why Max and I will suspend the development of Leaf and focus on new ventures.

Is this accurate? If so, should the README mention so?

Autumn is also still listed on http://rust-lang.org/friends.html, is Autumn still a thing? If so, will the company continue to use Rust?

MichaelHirn commented 8 years ago

Hi @dikaiosune,

We are procrastinating really hard on updating the README, although we should because Autumn is in the process of shutting down.

Also, no one stepped up to take over Leaf which means Leaf is rather dead at the moment, too.

neuralvis commented 8 years ago

@MichaelHirn I can make an effort to take over leaf, and it's associated libraries. But I will be needing a lot of help and assistance in assimilating the code. I believe there is a lot of value in collenchyma framework - especially in building robust code for large scale data visualization and analysis pipelines.

kination commented 8 years ago

I also agree with @vyslexic too. Still lot of people are interested on leaf, and it seems bad idea to close it. Instead of shutting down, how about take over to someone else who wants too?

KodrAus commented 8 years ago

@vyslexic I'm a hobbyist with an expensive graphics card, so don't have a lot of GPU experience but would be happy to step up and help if someone with more experience took it on too.

I was working through the documentation book when Autumn wound down.

neuralvis commented 8 years ago

@MichaelHirn , @hobofan - I am happy to carry forward your work with autumnai. How would you guys like to proceed ? You can either add me to the autumnai org or I can start maintaining a mirror.

@KodrAus, @djKooks - we can use all the help we can to develop the community and the software.

kination commented 8 years ago

@vyslexic good to hear that! Please share the progress of carrying it on.

hobofan commented 8 years ago

Hey everyone, @MichaelHirn and me wanted to write down a well formulated statement, but I never seem to find the time for it, so I'll just write out right now what we decided on:

We both don't feel comfortable handing over the maintainership over Leaf's and the other Autumn repositories to a maintainer we don't know well, and also don't have the time to vet one. Another motivation for that is, that both Leaf and Collenchyma can develop into very different directions from here on, and we don't want to endorse one over the other. If the projects would have had more time to mature and grow a healthy community around them, that would probably be a different story, but that's not the situation we are in right now.

So we feel that the best way moving forward to ensure that the code we built continues to provide value, and that you, the community can start to build a healthy project is for anyone wanting to continue with the project to do a hard fork.

Feel free to discuss forming a hard fork and linking them in the issues of the relevant repositories. We will try our best to make them easy to find by linking them in the READMEs, so that anyone wanting to contribute can find the new projects.

MichaelHirn commented 8 years ago

Thanks, @hobofan.

I agree, but am open to being convinced otherwise. If taking over Autumn instead of forking it, is more relevant to your plans, @vyslexic, I'm happy to do a 15 min call in the next few days. (Max, happy to have you on the call if you want to.)

kination commented 8 years ago

Thanks to @MichaelHirn, @hobofan . @vyslexic please share us the result about it. Lots of people are interested on it.

neuralvis commented 8 years ago

@djKooks - my apologies for intermittent responses. It has been a busy last few days. @hobofan, @MichaelHirn - thanks for clarifying the recommended approach moving forward. I understand your concerns, and fully respect them. My intention wasn't to "take over" as maintainer, but to merely be added as a contributor to the group, so I can take an active role in handling issues, and arbitrating pull requests etc. I am content with working on a fork as well.

Explaining the motivation and plans behind my interest in leaf/collenchyma will require a long blog post, and I hope to get there some day (soonish!). But briefly - my background and contributions in the past few years (~8yrs) has been in the area of high performance visualization, in-situ analytics, and data mining for large scale (~petabytes) scientific (HPC) applications. A recurring frustration that I often encounter is the lack of robust, scalable, fault-tolerant software (for analytics and visualization) that "just works" on large scale HPC systems.

The "state-of-the-art" software libraries such as Paraview, VTK are around ~20 years old, and do not support the computing paradigms (both hardware and software) that are evolving (hybrid architectures, fine-grained concurrency, deep learning etc). Solutions like Hadoop and Spark may be engineered but do not map very well to the problem domains we are looking into (more on this on my yet-to-come blog).

This is where I believe leaf/collenchyma can really add value. First, they are based on Rust, and therefore inherit all the guarantees that come with the language. Second, a library such as leaf/collenchyma can be combined with a fault-tolerant, soft real-time system (erlang, elixir) to provide a data analysis/machine intelligence/visualization framework with the following benefits:

A highly-available run-time system that is
Massively scalable on-demand,
Is fault-tolerant and "self-reparing" to some extent,
and can scale-up and scale-out, effectively at minimal development cost.

This is a long-term goal, of course, and I would hope to achieve this with the help of many other contributors that have expressed interest here, and elsewhere in my community. I hope this answers a few questions you may have had. I come from a C/C++ family of languages, so I would say I am an intermediate Rust programmer. It will still take me a while to get up-to-speed with the Rust code in these repositories.

Happy to chat more on Slack/Skype/Hangout if needed ...

MichaelHirn commented 8 years ago

Thank you, @vyslexic. I am looking forward to the blog post, to learn more about it - sounds very cool.

I think (@hobofan, let me know if you disagree), that it would be best, given your objections, that to me, seem more complementary to Rust/Leaf/Collenchyma's general environment than to Leaf/Collenchyma-the library and the initial goals that we had, to start a fork and see from there.

neverfox commented 8 years ago

Also interested and would like to join in on any discussions.

sunface commented 8 years ago

Also interested and would like to join in on any discussions.

kination commented 8 years ago

@vyslexic do you have some next plan or schedule? It would be helpful to people who wants to support the project.

dlight commented 8 years ago

Just a thought: perhaps a software foundation like the Apache foundation could accept this project under their umbrella?

ehiggs commented 8 years ago

Apache is not a dumping ground for OSS. Apache also doesn't magically provide resources to a project.

castarco commented 7 years ago

Hi, I'm very interested on this project. I'd love to contribute to it. I'm not an expert, but I love Rust and ML, and I'm working on a PHP extension programmed in Rust which could use Leaf as a dependency.

kination commented 7 years ago

Hello~ Is there some progress on plan in this project?

ghost commented 7 years ago

Suggestion: Perhaps @alexandermorozov's fork could become a new venue for development? At a glance, his contributions appear to be the most active outside the core AutumnAI folks.

@hobofan @MichaelHirn - If Alexander is willing to be considered a "hard fork" maintainer, even temporarily, would you be willing to link to his repo from the README here?

I'd really love to see leaf live. It's an OpenCL-as-first-class learning framework written in an amazing high-performance language that makes mistakes hard. Right now, for OpenCL, the choices are pretty bleak (improving super rapidly but still..); Torch, if you like Lua and Magic, or Tensorflow, if you like proprietary compiler frameworks (required for OpenCL). Leaf could really carve out a following by approaching the hot machine-learning-learning segment. Using OpenCL, people can learn ML on the integrated graphics of their CPUs, they don't need to invest in expensive NVidia hardware!

So please, let's elect or nominate a new interim BDFL and see where Leaf can go? :)

alexandermorozov commented 7 years ago

@cathalgarvey Initially I didn't offer to take on maintainership because I wanted to explore graph-based approach mostly for anticipated performace reasons.

Currently I don't have enough time, energy and interest to move Leaf forward or to maintain the project. If situation changes I'll do the work in my own fork and @hobofan and @MichaelHirn could later decide if they like the direction and results and if they want to make it official.

With regards to OpenCL IIRC there is only a memory allocator in collenchyma. Generic tensor operations and NN specific stuff are yet to be implemented.

ghost commented 7 years ago

Well, I wasn't suggesting a full maintainer role, but rather a "willing to look at and merge PRs", followed by "delegates other maintainers to the repo".

Right now, the main repo is dead, and the Deva don't want to play an active role in it. Not trying to be unkind, I think their work has been amazing! But, there it is.

Under the idea that " anything is better than nothing", and picking your name from the list of active contribs, you made the most sense.

Of course, I'm also happy to fork the repos and give a few others on this issue access to the repo, so we have a community-maintained fork..but I'm not yet qualified to contribute anything material. :)

On 8 January 2017 21:21:53 GMT+00:00, Alexander Morozov notifications@github.com wrote:

@cathalgarvey Initially I didn't offer to take on maintainership because I wanted to explore graph-based approach mostly for anticipated performace reasons.

Currently I don't have enough time, energy and interest to move Leaf forward or to maintain the project. If situation changes I'll do the work in my own fork and @hobofan and @MichaelHirn could later decide if they like the direction and results and if they want to make it official.

With regards to OpenCL IIRC there is only a memory allocator in collenchyma. Generic tensor operations and NN specific stuff are yet to be implemented.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/autumnai/leaf/issues/108#issuecomment-271180462

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

kination commented 7 years ago

@MichaelHirn @hobofan Hello~

It looks like still lot of people are interest in this project, and some of are keep updating PR though it cannot be pulled. Could you let people know just how this project will be(like just closing or handing over maintainership to somebody) or it is still thinking about it? It will help people here to go on to next step.

It is sad to let this great project unsettled.

hobofan commented 7 years ago

@djKooks We already did that. As of now there are no plans from us to maintain this repository.

Our recommended method of moving forward for the community is still to do a hard fork, but as I the comments here read, nobody seems to be interested enough in working on the project and actively driving it forward in a maintainership role.

kination commented 7 years ago

@hobofan Thanks for quick answer. One more, do you have some criteria of a maintainer?

It will a hard job to be a maintainer of quite a massive project. Most of engineer would have their job, and don't have time to participate in full-time. How about making a community for managing this project? It would be more better if each people share there time together. Is there someone following this issue who is interest in?

alexandermorozov commented 7 years ago

@cathalgarvey One problem is that there are no code PRs since shutdown, only small documentation fixes. People will not likely contribute much effort to a dead project, so what is dead tends to stay dead. Also Leaf with other crates is quite complex in itself, so there is a rather high barrier to entry. To move forward someone should put effort in and continue development, after that others may want to join in. Another path is to fork it, then conduct a massive PR campaign to rekindle interest in Leaf but it's not very likely to succeed.

I'm currently restarting a private project that may benefit from a NN lib in rust in the future. I'm quite familiar with Leaf, so if there would be perceived advantages of developing own tools instead of using Tensorflow/... I may continue work on Leaf in my fork. That's not to dissuade anyone from forking, of course do it if you have inclination! I have quite a lot going on and may not get around to work on it again, life is quite unpredictable )

kination commented 7 years ago

@alexandermorozov thanks for good message. I'm also looking on leaf but because it's not quite long since I started studying Rust/ML, it is hard going in. Hope someone make this work again.

castarco commented 7 years ago

I'm answering from a crappy smartphone, so I'll try to be brief.

I'm not very skilled on Rust, but I'm learning and wanting to contribute. In the meantime I can contribute documentation and support on chats&forums, maybe we should list the contributor candidates to have a better view of the areas that still need someone to be covered... and then start an organized effort to revive the project.

Kind regards

alexandermorozov commented 7 years ago

I've had a little time to merge most of PRs into my "fork" manually except #68 (seems to be already fixed), #75 (author has deleted his branch), autumnai/collenchyma-nn#46 (it's a great work but after my refactoring it's unmergeble, I'll adapt it later). I've also changed cargo files to point to my forks on github. As of now leaf-examples fails to compile, everything else should.

Short-term plans:

fix and merge autumnai/collenchyma-nn#46,
consider applying collenchyma/decoupling branch in my repo. While it adds some decoupling, I want it for properly typed memory in user API of SharedTensor. There are tradeoffs though,
see what can be done about Arc<RWLock<_>> everywhere in leaf code,
IDevice, IHardware, IMemory, IFramework, IBinary traits, DeviceType, MemoryType -- read code and figure out what does what, and if there is a chance to reduce complexity,
see what can be done about feature flags and conditional compilation. Currently some combinations of flags produce warnings or compilation failures. If decoupling makes it in, things could be possibly simplified,
there are lots of unwraps in leaf code. Maybe reduce error boilerplate in collenchyma with error-chain,
ensure that NNs really work. There were some issues, they have been fixed, but I'm not sure that learning really works as it should,
try to keep docs up-to-date; fix statements about the scope of the project,

Long term plans (if there is a long term):

proper hard fork (select name, do fork, create org, releases on crates.io),
consider using a tensor lib for native memory (instead of flat slices), ndarray maybe.

ghost commented 7 years ago

Wow, bravo and thank you sir! Looking forward to seeing where it goes.

Nomenclature suggestion: Leaf was all about plants, particularly aerial plant organs.. perhaps the more grassroots revival from a dead repo closer to fungi and other detritivores? Particularly thinking of slime molds like Physarum polycephalum which demonstrate pseudointelligent behaviour.

On 15 January 2017 22:18:14 GMT+00:00, Alexander Morozov notifications@github.com wrote:

I've had a little time to merge most of PRs into my "fork" manually except #68 (seems to be already fixed), #75 (author has deleted his branch), autumnai/collenchyma-nn#46 (it's a great work but after my refactoring it's unmergeble, I'll adapt it later). I've also changed cargo files to point to my forks on github. As of now leaf-examples fails to compile, everything else should.

Short-term plans:

fix and merge autumnai/collenchyma-nn#46,

consider applying collenchyma/decoupling branch in my repo. While it adds some decoupling, I want it for properly typed memory in user API of SharedTensor. There are tradeoffs though,

see what can be done about Arc<RWLock<_>> everywhere in leaf code,

IDevice, IHardware, IMemory, IFramework, IBinary traits, DeviceType, MemoryType -- read code and figure out what does what, and if there is a chance to reduce complexity,

see what can be done about feature flags and conditional compilation. Currently some combinations of flags produce warnings or compilation failures. If decoupling makes it in, things could be possibly simplified,

there are lots of unwraps in leaf code. Maybe reduce error boilerplate in collenchyma with error-chain,

ensure that NNs really work. There were some issues, they have been fixed, but I'm not sure that learning really works as it should,

try to keep docs up-to-date; fix statements about the scope of the project,

Long term plans (if there is a long term):

proper hard fork (select name, do fork, create org, releases on crates.io),

consider using a tensor lib for native memory (instead of flat slices), ndarray maybe.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/autumnai/leaf/issues/108#issuecomment-272734377

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

kination commented 7 years ago

@alexandermorozov Seems good~Is it okay to fork on your project?

alexandermorozov commented 7 years ago

@cathalgarvey I like this suggestion ) Though it'll be a long time until hard fork makes sense. I think it'll take me about 3 months to work through the list of short-term issues at the current pace. That's if everything in life stays more or less the same.

@djKooks Yes, feel free to fork it!

botev commented 7 years ago

@cathalgarvey @alexandermorozov @djKooks @castarco @hobofan I recently posted this suggestion for a full Graph based framework/compiler with autodiff in rust on reddit https://www.reddit.com/message/inbox/. The main goal of that would be to reuse the backend/lower level compute of what Leaf already has for a start, but use a graph based framework to make it very flexible and comparable to other frameworks in the wild. I'm currently finishing writing some of the major parts of the graph with the autodiff, however I have no expertise in how to make this then translate to the low level part. We had and some discussion on reddit that might be of use to you guys.

hobofan commented 7 years ago

Correct link to the reddit discussion: https://www.reddit.com/r/rust/comments/5of7t8/is_anyone_interested_in_trying_to_build_a_graph/

botev commented 7 years ago

I've made a gitter room where potentially if we get to be in at the same time can have a slightly more legthy discussion: https://gitter.im/rust-ml/Lobby?utm_source=share-link&utm_medium=link&utm_campaign=share-link

Sorry for the bad link last time -.-

botev commented 7 years ago

@alexandermorozov I've saw you are interested in graph interface and I just want to make an update on it. I currently implemented it for a very limited number of ops. One difference with all other current ML frameworks is that it validate shapes(including dynamic) at graph construction, thus this gives you "graph construction" time guarantee that if there is no error, the resulting operations are valid (This is the spirit of rust and in contrast to Theano and Tf where you can get this cryptic error messages at runtime). Currently the interface looks like this:

    let g = gir::Graph::default();
    let x = &f_var!(g, (784, "n"), "input");
    let y = &f_var!(g, ("n"), "target");
    let y_t = &transpose(y)?;
    let w1 = &f_param!(g, (1, 784), "w1")?;
    let b1 = &f_param!(g, (1), "b1")?;
    let h1 = &tanh((mat_mul(w1, x)? + b1))?;
    let error = sum_all((h1 - y_t) * (h1 - y_t))? / dim0(y)?;
    let params = &vec![w1, b1];
    let grads = gir::derivative::gradient(error, params)?;

However, currently the project lacks any backend at all for executing the graph. Any help on that front would be most helpful.

kazimuth commented 7 years ago

@MichaelHirn @hobofan

After nearly a year, the README has still not been updated stating that this project has been completely abandoned (barring a hard fork sometime in the future). It's confusing and misleading, since the project has thousands of stars and a lot of apparent polish.

A single commit adding "this project is dead, don't expect updates or support" to the top of the README would do wonders to keep people from wasting time trying to learn it.

Similarly, it's entirely non-obvious that autumnai.com is defunct, and a header there would be helpful as well.

Enet4 commented 7 years ago

@kazimuth On that end, I agree that the project should not leave a false sense of security. Adding the "no maintenance intended" badge () should be sufficiently alarming, but we also don't want people to be steered away from a potential fork, so we should also leave a link to this issue or the project where all future progress will take place.

jonysy commented 7 years ago

Hello all!

I and a few others are considering hard-forking Leaf since we've already become familiar with it.

From an HN discussion regarding the flexibility of Leaf's current design:

[HN commenter]: I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes.

M. Goisser: I am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky.

I think there is still a lot of room for improvement for layer-based networks, but even if the graph-based approach will become more prelevant I think the Rust trait system gives us great tools to migrate and reuse a lot from our layer based approach.

@MichaelHirn @hobofan Would you two mind elaborating on this? The question came up a lot during Leaf's lifetime, but I couldn't find a clear, detailed answer..

Boscop commented 7 years ago

Is there a new maintained hard fork now? It's sad to see all this hard work go to waste..

Binero commented 7 years ago

@Boscop At least one guy seems to be maintaining it, haven't tried it though: https://github.com/ratpoison-io/leaf

jonysy commented 7 years ago

@Boscop @Binero here's a quick overview:

I have my own project, Parenchyma (started off as a Collenchyma (an HPC framework) fork). I'm currently not using it for anything related to ML, but there's nothing keeping it from being used for that!
@botev Is currently experimenting with a graph-based approach, or at least a library that'll make implementing new algorithms less of a hassle.
There was a Leaf-RustyMachine collaboration effort, but it went nowhere due to the shutdown (cc @AtheMathmo) (*informative discussion in the linked thread).

It seems that @drahnr is currently working off of @alexandermorozov's fork (last commit was 2 days ago at the time of typing this post), which hopefully means that I can abandon my leaf fork. In any case, it would make for an awesome community project.

jonysy commented 7 years ago

I'd be willing to adapt Leaf for Parenchyma if there's interest.

How can we go about having a community-maintained fork of Leaf? Any suggestions would be appreciated.

drahnr commented 7 years ago

@jonysy I would definitely be interested regarding the changes and goals of parenchyma vs today's collenchyma :) drop a mail or find me on gitter, I am open for discussion and merging efforts

jonysy commented 7 years ago

@drahnr 👍 great! I sent you a message via Gitter.

drahnr commented 7 years ago

just a quick update: I am working on cuda 5.1 support which is almost done and adaptations on the cudnn backend for collenchyma-nn (partially based on @DiamondLovesYou 's refactorings adopted to the new rw tensor API introduced by @alexandermorozov ).

Afterwards I am planning to rust-ify the leaf interface and polish the examples.

autumnai / leaf

Current status of leaf? #108