https://tianchi.aliyun.com/competition/entrance/532005/introduction
DeepRec is a recommendation engine based on TensorFlow 1.15, Intel-TensorFlow and NVIDIA-TensorFlow.
Sparse model is a type of deep learning model that accounts for a relatively high proportion of discrete feature calculation logic in the model structure. Discrete features are usually expressed as non-numeric features that cannot be directly processed by algorithms such as id, tag, text, and phrases. They are widely used in high-value businesses such as search, advertising, and recommendation.
DeepRec has been deeply cultivated since 2016, which supports core businesses such as Taobao Search, recommendation and advertising. It precipitates a list of features on basic frameworks and has excellent performance in sparse models training. Facing a wide variety of external needs and the environment of deep learning framework embracing open source, DeepeRec open source is conducive to establishing standardized interfaces, cultivating user habits, greatly reducing the cost of external customers working on cloud and establishing the brand value.
DeepRec has super large-scale distributed training capability, supporting model training of trillion samples and 100 billion Embedding Processing. For sparse model scenarios, in-depth performance optimization has been conducted across CPU and GPU platform. It contains 3 kinds of features to improve usability and performance for super-scale scenarios.
CPU Platform
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-cpu-py36-ubuntu18.04
Docker Hub repository
alideeprec/deeprec-build:deeprec-dev-cpu-py36-ubuntu18.04
GPU Platform
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-gpu-py36-cu110-ubuntu18.04
Docker Hub repository
alideeprec/deeprec-build:deeprec-dev-gpu-py36-cu110-ubuntu18.04
Configure
$ ./configure
Compile for CPU and GPU defaultly
$ bazel build -c opt --config=opt //tensorflow/tools/pip_package:build_pip_package
Compile for CPU and GPU: ABI=0
$ bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --host_cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" -c opt --config=opt //tensorflow/tools/pip_package:build_pip_package
Compile for CPU optimization: oneDNN + Unified Eigen Thread pool
$ bazel build -c opt --config=opt --config=mkl_threadpool //tensorflow/tools/pip_package:build_pip_package
Compile for CPU optimization and ABI=0
$ bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --host_cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" -c opt --config=opt --config=mkl_threadpool //tensorflow/tools/pip_package:build_pip_package
$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
$ pip3 install /tmp/tensorflow_pkg/tensorflow-1.15.5+${version}-cp36-cp36m-linux_x86_64.whl
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec2204u1-gpu-py36-cu110-ubuntu18.04
Docker Hub repository
alideeprec/deeprec-release:deeprec2204u1-gpu-py36-cu110-ubuntu18.04
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec2204u1-cpu-py36-ubuntu18.04
Docker Hub repository
alideeprec/deeprec-release:deeprec2204u1-cpu-py36-ubuntu18.04
Build Type | Status |
---|---|
Linux CPU | |
Linux GPU | |
Linux CPU Serving |
Unit Test Type | Status |
---|---|
Linux CPU C | |
Linux CPU CC | |
Linux CPU Contrib | |
Linux CPU Core | |
Linux CPU Examples | |
Linux CPU Java | |
Linux CPU JS | |
Linux CPU Python | |
Linux CPU Stream Executor | |
Linux GPU C | |
Linux GPU CC | |
Linux GPU Contrib | |
Linux GPU Core | |
Linux GPU Examples | |
Linux GPU Java | |
Linux GPU JS | |
Linux GPU Python | |
Linux GPU Stream Executor | |
Linux CPU Serving UT |
Join the Official Discussion Group on DingTalk