anaderi
commented
7 years ago Hey Jonas,
thanks a lot for constructive comments! we’ll consider them thoroughly. BTW, would you be interested in the further development/refactoring of the code?
-- Kindest Regards, Andrey Ustyuzhanin
On 8 Jul 2017, at 14:22, Jonas Eschle wrote:
Hey developers,
there has not been a lot of activity, it looks clearly as if the repo gets abandoned. Which I find quite sad! I think it took quite some time for people to discover your repository (to be honest, reproducibility was not the best reason to convince people to use it. I usually promote REP under: "save 80% of the time you spend coding") and 400+ stars speak for themselves. On the other hand a lot of work is needed for the upkeep of the repo (wrappers...), sure, I understand. But I think times and needs have also changed.
Let me first tell you what I like about REP:
- reports! great tool (also the metrics)
- ML-lego using folding, bagging etc (it's very nice for people who like to "play around" and not just use default algorithms)
- useful gridsearch tools
- same API for several libraries like XGBoost etc allows for simple replacement of any classifier without altering the code
This things are quite useful and so far I have not found anything similar.
So let me propose you this:
Convert REP to an "LTS" version
which means, there are some crucial things that can be removed and therefore drastically reduce the time spent on the upkeep of the repo.
Things to do:
get rid of all libraries except of XGBoost, Scikit-Learn, (may add a direct keras wrapper?) and improve their wrapper (for example, XGBoost has a lot of additional parameters I think).
which includes to drop the ROOT bindings. And that's fine. You have helped to show that opensource machine learning does not have to hide in any way from TMVA and is usually the better choice (thanks for that!). Although TMVA has drastically improved its library, I still would not recommend to have any bindings (as they are a vast majority of the upkeep I think). If the ROOT guys really come up with something genius, I would highly suggest you to port this solution, implement it in your own repo and therefore make it available for the whole ML community (think of this: if TMVA has a great tool, no one will just install root to have a great ML algorithm except of already ROOT users who will use TMVA then directly anyway). This would benefit everyone (ML community, you, even TMVA as their ideas get exported to people who would otherwise never use them).
Although the factory is a nice thing, if it generates a lot of bugs or is time-consuming for upkeep, may get rid of it as well.
may add some of the requested features (like stratified-k-folding which is waiting in a branch to be merged, stackingClassifier and so on, see issues). I think those are quite small things in general which would complete REP.
May also include you new tool hyperopt as an option to the GridSearch.
May promote REP differently. It is a very convenient wrapper around other libraries and saves a lot of work.
There are probably more things which require a loot of maintenance effort. Let us know about them and discuss things.
I know that this requires some effort. But I also think this will drastically reduce the maintenance (all libraries have more or less stable APIs, XGBoost, sklearn and keras are in a stage where changes are rare I think) and make REP survivable for the next 5-10 years or so, probably serving a niche of people (although I do not know about similar alternatives). In the current state, I don't think it will last too long, as lack of support (like reactions to issues etc.) lets people abandon the repo. If you are able to reduce the maintenance, I think you will have time for the (still quite rare) issues coming up from time to time.
On the other hand, if you don't like the extra amount of work and decide to abandon the repo, it would be nice to state that in the README, stating reasons and alternatives (-> hyperopt) so that new people do not use the repo as they otherwise would be disappointed.
Anyway, thanks for the great work you have done!
Cheers, Jonas
-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/yandex/rep/issues/104
eyadsibai
jonas-eschle
gandreassi
HosseinAfsharnia
Hey developers,
there has not been a lot of activity, it looks clearly as if the repo gets abandoned. Which I find quite sad! I think it took quite some time for people to discover your repository (to be honest, reproducibility was not the best reason to convince people to use it. I usually promote REP under: "save 80% of the time you spend coding") and 400+ stars speak for themselves. On the other hand a lot of work is needed for the upkeep of the repo (wrappers...), sure, I understand. But I think times and needs have also changed.
Let me first tell you what I like about REP:
This things are quite useful and so far I have not found anything similar.
So let me propose you this:
Convert REP to an "LTS" version
which means, there are some crucial things that can be removed and therefore drastically reduce the time spent on the upkeep of the repo.
Things to do:
get rid of all libraries except of XGBoost, Scikit-Learn, (may add a direct keras wrapper?) and improve their wrapper (for example, XGBoost has a lot of additional parameters I think).
which includes to drop the ROOT bindings. And that's fine. You have helped to show that opensource machine learning does not have to hide in any way from TMVA and is usually the better choice (thanks for that!). Although TMVA has drastically improved its library, I still would not recommend to have any bindings (as they are a vast majority of the upkeep I think). If the ROOT guys really come up with something genius, I would highly suggest you to port this solution, implement it in your own repo and therefore make it available for the whole ML community (think of this: if TMVA has a great tool, no one will just install root to have a great ML algorithm except of already ROOT users who will use TMVA then directly anyway). This would benefit everyone (ML community, you, even TMVA as their ideas get exported to people who would otherwise never use them).
Although the factory is a nice thing, if it generates a lot of bugs or is time-consuming for upkeep, may get rid of it as well.
may add some of the requested features (like stratified-k-folding which is waiting in a branch to be merged, stackingClassifier and so on, see issues). I think those are quite small things in general which would complete REP.
May also include you new tool hyperopt as an option to the GridSearch.
May promote REP differently. It is a very convenient wrapper around other libraries and saves a lot of work.
There are probably more things which require a loot of maintenance effort. Let us know about them and discuss things.
I know that this requires some effort. But I also think this will drastically reduce the maintenance (all libraries have more or less stable APIs, XGBoost, sklearn and keras are in a stage where changes are rare I think) and make REP survivable for the next 5-10 years or so, probably serving a niche of people (although I do not know about similar alternatives). In the current state, I don't think it will last too long, as lack of support (like reactions to issues etc.) lets people abandon the repo. If you are able to reduce the maintenance, I think you will have time for the (still quite rare) issues coming up from time to time.
On the other hand, if you don't like the extra amount of work and decide to abandon the repo, it would be nice to state that in the README, stating reasons and alternatives (-> hyperopt) so that new people do not use the repo as they otherwise would be disappointed. This is a reasonable choice, most of all if you may know that the functionality provided by REP will be obsolete anyway soon (because of similar libraries, or other things to come in the near future...).
Anyway, thanks for the great work you have done!
Cheers, Jonas