Open szha opened 4 years ago
@StevenJokes for usage questions, feel free to use https://mxnet.apache.org/community/contribute#forum. RFC issues like this notify all mxnet developers automatically for any discussion so let's move this elsewhere.
@StevenJokes thanks for the feedback and I'm really glad that you feel strongly about mxnet and want it to get better. I share the sentiment and as a community I think this is indeed an area where we try to improve. cc @astonzhang on the d2l support. cc @sandeep-krishnamurthy
That said, the past few comments have been off-topic, and I will mark them so once others have a chance to look at them.
I think one reason why mxnet 1.0 failed is that we don't care others and give no reply to their questions in discuss.mxnet.io .
@StevenJokes Thank you for mentioning this notable and important point! I perceive that an open source project should has an active community to grow up. The forum of MXNet is really inanimate, almost no experienced developers answer the questions on it. This situation will strongly reduce people’s desire to use this framework. Please try to improve it! Tks.
@qqaatw thank you for sharing your feedback. Again, I share the same sentiment and I myself will at least try to help as much as my bandwidth allows. That said, the proper channel to bring this up is a separate thread on dev@mxnet.apache.org.
This issue is for feedback on 2.0 features and use cases and I request that we stay on topic. I will mark these conversations as off-topic once others have a chance to look at them.
@StevenJokes you still can reach developers by joining discussion on dev@mxnet.apache.org, or mentioning this team in GitHub @apache/mxnet-committers
. They should happen in a different issue and thread, not here.
@szha I was in the dev mailing list months ago, and I’m not really a new. BUT what @StevenJokes said really make sense, it is the most important thing, even the fatal problem on MXNet. Please do not put this matter aside, or a off-topic. Thank you for understanding in advance
@StevenJokes, I am not involved in mxnet in any way. Your comments are really crossing over the line. While it is fair to point out that mxnet can be doing a better job reaching out to its intended community, this is not the place to do it. If I were you, I really would not want unprofessional comments such yours to be appearing where potential employers can find them.
@StevenJokes I think the issue about DCGAN is related to D2L and you'd better to discuss it in the D2L-related channels. (Find one in the D2L: https://github.com/d2l-ai/d2l-en/issues/1381) If you have specific issues about MXNet, you may create an issue and tag it as BUG, or other labels so we can take a look.
@qqaatw this thread is for 2.0 discussion on the technical roadmap. Community engagement is not part of this topic. As mentioned above, I shared the same goal and sentiment as you. And I also treat this issue seriously and by no means am dismissing you. But, unless the community can have a proper discussion on this, others have no way of working together with you on this.
@StevenJokes here is how we intended the discussion channels to be used: https://mxnet.apache.org/versions/1.6/community/contribute.html And here's the code of conduct of this project: http://www.mxnet.incubator.apache.org/foundation/policies/conduct.html And very much contrary to what you said, I've been on this project for the past three years and I care deeply about having a healthy community, and opinions on making it better like yours.
@qqaatw @StevenJokes I started a thread for this on your behalf here: #18963, hope that's ok.
@StevenJokes In addition, for the DCGAN issue that is related to D2L, a simple way to verify that you are correct (and also convince the others), is to write a test case that checks whether these two networks are equivalent.
For example, you have a network A implemented in MXNet and a network B implemented in PyTorch. There are several checks that you can do:
Usually, you will need to do more to convince the others that certain issues exist. There are some examples:
It will be a good practice if you can write such test cases and tell D2L people.
@StevenJokes Thanks for your effort working on MXNet project and D2L part. It's really an awesome work you have done. We should definitely raise our attention on active contributors. Before we drive this thread into cat-fight, I would recommend you open another issue like Sheng mentioned in the issue. We can discuss over there. This one is just the roadmap for 2.0 features and updates. If you have concerns to MXNet community, please feel free to reach out to dev@mxnet.apache.org. For the no-official reply part, we all just open-source and voluntarily contribute to the community, so there is absolute no gauranteed 3-day respond mechanism. We tried our best to maintain this and we will keep doing that.
If possible, I would recommend to withdraw the extreme comment to this thread and put it to the community conversation channel. Thanks for your support!
@StevenJokes Thanks for your comments! Couple of things:
What's plan of the validation and user tutorial?
Does the example still work?
What's plan of the validation and user tutorial?
I think the tutorials (in the form of markdown, shown on the website) are currently validated in the website build pipeline. Examples will be maintained in apache/incubator-mxnet-examples with CI enforcement.
@szha Just a small comment and question on the issue of the tutorials asked by @pengzhao-intel. I am starting to look at this framework and find that I need to hunt down the dependencies so that the project examples work.
Case in point the MNIST example has a link to the setup. However, we still need information on the dependencies for the ai.djl.basicdataset.Mnist
import. That important information is found here. I have not found a link to this page from the documentation.
I would also like to know how are you validate the tutorials (I haven't checked so they may be in perfect working order).
I am interested in this because I have other projects in Scala (example) wherein I generate a site. In one case I use for example Laika to process the Markdown. To ensure the code is working I use either Tut or Mdoc, which supersedes it to preprocess the Markdown sources. Note that all the code are in the Markdown files and are checked when they are compiled and execute (output can be placed in the source Markdown). Thus code and documentation are always in sink. Is there a way to do this in Java?
EDIT: just realized that the checks above won't catch any missing references to dependencies.
@hmf thanks for pointing out the issue on dependencies.
I would also like to know how are you validate the tutorials (I haven't checked so they may be in perfect working order).
For tutorials on mxnet.apache.org, they are jupyter notebooks in markdown format (processed by notedown) that are executed at the time of building the documentation. for the examples folder, we plan to move them to the new repo gradually, guarded by regular CI checks to make sure they are working.
Since the example issues you pointed out belong to DJL, @lanking520 will likely be of best help in resolving them.
@szha Thanks for the answers regarding the checks. Thanks for link also.
Since the example issues you pointed out belong to DJL, @lanking520 will likely be of best help in resolving them.
Oops, my apologies. Scratch that.
@szha Sorry for being late, but can someone please comment on R-package support in MXNet 2.0? I looked over the master branch and the 2.0.0.beta0.rc0 tagged pre-release and the whole R-package
directory is missing? Is R support supposed to be dropped with the upcoming MXNet 2.0 version?
@jens-maus while we don't explicitly plan on dropping, for R-package maintenance we will need community members who maintain R package to come up with 2.0 support plan. At the moment this hasn't happened yet.
Overview
As MXNet development approaches its 2.0 major milestone, we would like to update our community on roadmap status, and highlight new and upcoming features.
Motivation
The deep learning community has largely evolved independently from the data science and machine learning(ML) user base in NumPy. While most deep learning frameworks now implement NumPy-like math and array libraries, they differ in the definition of the APIs which creates confusion and a steeper learning curve of deep learning for ML practitioners and data scientists. This creates a barrier not only in the skillsets of the two different communities, but also hinders the knowledge sharing and code interoperability. MXNet 2.0 seeks to unify the deep learning and machine learning ecosystems.
What's new in version 2.0?
MXNet 2.0 is a major version upgrade of MXNet that provides NumPy-like programming interface, and is integrated with the new, easy-to-use Gluon 2.0 interface. Under the hood, we provide an enhanced DL implementation in NumPy. As a result, NumPy users can easily adopt MXNet. Version 2.0 incorporated accumulative learnings from MXNet 1.x and focuses on usability, extensibility, and developer experiences.
What's coming next?
We plan to make a series of beta releases of MXNet 2.0 in lockstep with downstream projects migration schedule. The first release is tracked in #19139. Also, subscribe to dev@mxnet.apache.org for additional announcements.
How do I get started?
As a developer of MXNet, you can check out our main 2.0 branch. MXNet 2.0 nightly builds are available for download.
How can I help?
There are many ways you can contribute:
Highlights
Below are the highlights of new features that are available now in the MXNet 2.0 nightly build.
NumPy-compatible Array and Math Library
NumPy has long been established as the standard array and math library in Python and the MXNet community recognizes significant benefits in bridging the existing NumPy machine learning community and the growing deep learning community. In #14253, the MXNet community reached consensus on moving towards a NumPy-compatible programming experience, and committed to a major effort on providing NumPy compatible array library and operators.
To see what the new programming experience is like, check out Dive into Deep Learning book, the most comprehensive interactive deep learning book with code+math+forum. The latest version has an MXNet implementation with the new MXNet np, the NumPy-compatible math and array interface.
Gluon 2.0
Since the introduction of the Gluon API in MXNet 1.x, it has superseded other MXNet API for model development such as symbolic, module, and model APIs. Conceptually, Gluon was the first attempt in the deep learning community to unify the flexibility of imperative programming with the performance benefits of symbolic programming, through just-in-time compilation.
In Gluon 2.0, we are extending support to MXNet np with simplified interface and new functionalities:
3rdparty Plugin Support
Extensibility is important for both academia and industry users who want to develop new, and customized capabilities. In MXNet 2.0, we added the following support for plugging in 3rdparty functionality at runtime.
Developer Experiences
In MXNet 2.0, we are making development process more efficient in MXNet.
Documentation for Developers
We are improving the documentation for MXNet and deep learning developers.
Ecosystem: GluonNLP NumPy
We are refactoring GluonNLP with NumPy interface for the next generation of GluonNLP. The initial version is available on dmlc/gluon-nlp master branch:
API Deprecation
As described in #17676, we are taking this major version upgrade as an opportunity to address the legacy issues in MXNet 1.x. Most notably, we are deprecating the following API:
Related Projects
Below is a list of project trackers for MXNet 2.0.
@apache/mxnet-committers feel free to comment or directly edit this post for updates in additional areas.