Closed makefile closed 4 years ago
It is a performance problem, since MobulaOP will call wait_to_read
and wait_to_write
for intputs and outputs, and MobulaOP is implemented by Python, which drops the computation performance.
I will refactor it in C++.
For ROIAlign Operator, I recommend mxnet.symbol.contrib.ROIAlign
.
Thanks for your great work, expecting your refactor. Writing c++ layer then compile the mxnet source tree is really painful. It will be good if MobulaOP runs faster than the python CustomOp using numpy.
Thanks! : )
@makefile Hi! MobulaOP supports asychronous exection in MXNet, Linux, and it may runs faster.
Thanks for your wonderful job. By the way, can the code run in multi-gpu?
@wkcn Thanks very much!
@hustzxd Sorry that I do not have the server with multi-gpu to test it now. Theoretically it works on multi-gpu.
@wkcn Thanks for your amazing work. It can run on multiple GPUs. The problem is that wait_to_read may lock the computation on other GPUs as well, which results in low performance on multiple GPUs.
@YutingZhang Currently, On Linux, MobulaOP supports asychronous exection, which does not call wait_to_read.
Hi @makefile and @YutingZhang , I have added TVM bridge into MobulaOP, and MobulaOP enables asynchronous execution for MXNet by default.
There is not any wait_to_read
lock to synchronize.
Moreover, MobulaOP works in multi-gpu context.
Thanks! @wkcn
Awesome! Thank you. @wkcn
A simple benchmark: https://github.com/wkcn/MobulaOP/blob/master/benchmark/benchmark_roi_align.py MXNet is the baseline 100%.
MobulaOP(83e10986740410aab314b29fd5ad3dfa2d3a8b47) uses the default configuration (disable openmp).
Implement | time (s) | ratio |
---|---|---|
MobulaOP | 36.303 | 96.96% |
MXNet | 35.199 | 100% |
Implement | time (s) | ratio |
---|---|---|
MobulaOP | 43.229 | 99.94% |
MXNet | 43.201 | 100% |
The issue has been addressed. Close it : )
Hi, I have tried to use the ROIAlign custom op provided by the repo, and I run faster rcnn examples, I just simplily replace the symbol code:
with
the running speed decrease from 0.1s to 1~2s, and when use multi-gpu, the code cannot run parallel, and become much more slower. My mxnet version is 1.3.0-cu92 from pip install. What might be the problem?