search

dmlc / gluon-nlp

NLP made easy
https://nlp.gluon.ai/
Apache License 2.0
2.56k stars 538 forks source link

surrogate for F.contrib.arange_like? #1199

Closed yqNLP closed 4 years ago

yqNLP commented 4 years ago

The following line in transformer encoder use F.contrib.arange_like, which depends on mxnet>=1.6.0. https://github.com/dmlc/gluon-nlp/blob/3f7465ab5d0f926c2c6f424644daaa30668570ba/src/gluonnlp/model/transformer.py#L397 And for some reason I only have installed mxnet-cu100=1.5.1, so I wrote a line to for surrogate this line: steps = nd.array(range(inputs.shape[1])). And the code can run successfully. However I am new to mxnet and I wonder is it ok? would it have some unexpected effects?

leezu commented 4 years ago

You won't be able to hybridize the model. It won't run as efficiently.

yqNLP commented 4 years ago

You won't be able to hybridize the model. It won't run as efficiently.

which means the training speed would be slower?

leezu commented 4 years ago

Yes. But you can also use GluonNLP 0.8 which supports MXNet 1.5 and has a workaround for arange_like. https://github.com/dmlc/gluon-nlp/blob/v0.8.x/src/gluonnlp/model/transformer.py

https://github.com/dmlc/gluon-nlp/blob/75b3c121ac02c1bdef25a785fda2238e256246f9/src/gluonnlp/model/transformer.py#L447-L458

yqNLP commented 4 years ago

Yes. But you can also use GluonNLP 0.8 which supports MXNet 1.5 and has a workaround for arange_like. https://github.com/dmlc/gluon-nlp/blob/v0.8.x/src/gluonnlp/model/transformer.py

https://github.com/dmlc/gluon-nlp/blob/75b3c121ac02c1bdef25a785fda2238e256246f9/src/gluonnlp/model/transformer.py#L447-L458

Thanks so much!