Package mxnet

[AbdealiLoKo]

@pjmdtw would you be intersted in contributing your mxnet package to conda-forge ? https://github.com/pjmtdw/mxnet-conda-builder

conda-forge already has xgboost and it would be excellent to have mxnet too.

[barry-jin]

Hi there. I'm the contributor of apache/incubator-mxnet and I'm interested in contributing mxnet packages to conda-forge. MXNet build will be based on different OSs and different cuda versions, so is there any way or examples to customize the mxnet recipes' build steps with different Dockerfiles and scripts. Thanks!

[jakirkham]

Hi Barry, thanks for helping out here 😀

We have some docs on building with CUDA. Also would take a look at a few recipes (like nccl)

Could you please clarify what you mean by different OSes? Typically we build CUDA 10.2 on CentOS 6 and CUDA 11.0+ on CentOS 7. This happens automatically. So if this is all you are looking for, then we don’t need to do anything special. Though if there’s something else needed here, please let us know

[barry-jin]

Hi @jakirkham, Thanks for your information. I find some old receipes here, mxnet build for different cpu and gpu variants and on different computation paltforms and systems. So, the receipes for mxnet could be a little complex. I have drafted a PR #14832 and I may need some instructions and help based on these old receipes. Thanks!

[ngam]

@jakirkham @barry-jin is there a reason why this hasn't been adopted yet?

I am currently building it locally (as a staged-recipe) and I have been having trouble getting rid of the deps --- it seems the upstream peeps are afraid of messing things up with onednn, etc. https://github.com/apache/incubator-mxnet/issues/19610

At any rate, I have a lengthy build going --- about 40% of it seems to be just building onednn which we have here in cf. I will submit a PR if things pass and then we can potentially customize the build. For now, I am just looking at the CPU builds.

[ 39%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_u8.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/brgemm_cell_common.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/cell_common.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/cell_gru.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/cell_gru_lbr.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_gru.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_gru_lbr.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_lstm.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_lstm_projection.cpp.o
[ 40%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_rnn.cpp.o
[ 41%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_rnn.cpp.o
[ 41%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/rnn_utils.cpp.o
[ 41%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/jit_utils/jit_utils.cpp.o
[ 41%] Building CXX object 3rdparty/onednn/src/cpu/CMakeFiles/dnnl_cpu.dir/jit_utils/linux_perf/linux_perf.cpp.o
[ 41%] Built target dnnl_cpu
[ 41%] Linking CXX static library libdnnl.a
[ 41%] Built target dnnl
[ 41%] Building CXX object 3rdparty/intgemm/CMakeFiles/intgemm.dir/intgemm/intgemm.cc.o
[ 41%] Linking CXX static library libintgemm.a
[ 41%] Built target intgemm

[ngam]

Also, we likely should start at version 2.x because of the switch of onednn naming and all that. I am working on the latest rc

[barry-jin]

Thanks @ngam. Thanks for bringing it up! Yes, we can start at version 2.x (current master branch) and if you have any issues or concerns, feel free to discuss here or submit issue to MXNet. I'm currently out of bandwidth on managing this, but I will offer any help that you need.

[ngam]

Thanks @ngam. Thanks for bringing it up! Yes, we can start at version 2.x (current master branch) and if you have any issues or concerns, feel free to discuss here or submit issue to MXNet. I'm currently out of bandwidth on managing this, but I will offer any help that you need.

Thanks a lot, @barry-jin. I will add you as a recipe maintainer once I push my PR. I will definitely need your help in ensuring everything is set up correctly, and then we can extend to include all os/arch matrix step by step. This is exciting!

[ngam]

see PR #17580