search

dmlc / gluon-nlp

NLP made easy
https://nlp.gluon.ai/
Apache License 2.0
2.56k stars 538 forks source link

BERT tutorial failed #984

Closed zburning closed 4 years ago

zburning commented 4 years ago

Description

I was trying to go through the BERT tutorial on http://gluon-nlp.mxnet.io/examples/sentence_embedding/bert.htmland got error in the following code. for epoch_id in range(num_epochs): metric.reset() step_loss = 0 for batch_id, (token_ids, valid_length, segment_ids, label) in enumerate(bert_dataloader): with mx.autograd.record():

Load the data to the GPU

        token_ids = token_ids.as_in_context(ctx)
        valid_length = valid_length.as_in_context(ctx)
        segment_ids = segment_ids.as_in_context(ctx)
        label = label.as_in_context(ctx)
        # Forward computation
        out = bert_classifier(token_ids, segment_ids, valid_length.astype('float32'))
        ls = loss_function(out, label).mean()
    # And backwards computation
    ls.backward()

Error Message

Traceback (most recent call last): File "/Users/zechenw/PycharmProjects/bert_playground/playground.py", line 104, in out = bert_classifier(token_ids, segment_ids, valid_length.astype('float32')) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/bert.py", line 635, in call return super(BERTClassifier, self).call(inputs, token_types, valid_length) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 548, in call out = self.forward(args) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 915, in forward return self._call_cached_op(x, args) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 805, in _call_cached_op self._build_cache(args) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 757, in _build_cache data, out = self._get_graph(args) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 749, in _get_graph out = self.hybrid_forward(symbol, *grouped_inputs, params) # pylint: disable=no-value-for-parameter File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/bert.py", line 656, in hybridforward , pooler_out = self.bert(inputs, token_types, valid_length) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/bert.py", line 425, in call valid_length, masked_positions) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 548, in call out = self.forward(args) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 932, in forward return self.hybrid_forward(symbol, x, args, params) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/bert.py", line 434, in hybrid_forward seq_out, attention_out = self._encode_sequence(inputs, token_types, valid_length) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/bert.py", line 470, in _encode_sequence outputs, additional_outputs = self.encoder(embedding, valid_length=valid_length) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/transformer.py", line 440, in call return super(BaseTransformerEncoder, self).call(inputs, states, valid_length) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/seq2seq_encoder_decoder.py", line 149, in call return super(Seq2SeqEncoder, self).call(inputs, valid_length, states) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 548, in call out = self.forward(args) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet/gluon/block.py", line 932, in forward return self.hybrid_forward(symbol, x, args, **params) File "/opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/gluonnlp/model/transformer.py", line 479, in hybrid_forward steps = F.contrib.arange_like(inputs, axis=1) AttributeError: module 'mxnet.symbol.contrib' has no attribute 'arange_like'

To Reproduce

http://gluon-nlp.mxnet.io/examples/sentence_embedding/bert.html

What have you tried to solve it?

I'm using gluonnlp installed from github. I checked mxnet.symbol.contrib and mxnet.symbol.gen_contrib, but did not find arrage_like().

Environment

----------Python Info---------- Version : 3.7.4 Compiler : Clang 4.0.1 (tags/RELEASE_401/final) Build : ('default', 'Aug 13 2019 15:17:50') Arch : ('64bit', '') ------------Pip Info----------- Version : 19.3.1 Directory : /opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/pip ----------MXNet Info----------- Version : 1.5.1 Directory : /opt/anaconda3/envs/mxnet_15/lib/python3.7/site-packages/mxnet Num GPUs : 0 Commit Hash : c9818480680f84daa6e281a974ab263691302ba8 ----------System Info---------- Platform : Darwin-18.6.0-x86_64-i386-64bit system : Darwin node : a483e7462a21.ant.amazon.com release : 18.6.0 version : Darwin Kernel Version 18.6.0: Sun Apr 28 18:06:45 PDT 2019; root:xnu-4903.261.4~6/RELEASE_X86_64 ----------Hardware Info---------- machine : x86_64 processor : i386 b'machdep.cpu.brand_string: Intel(R) Core(TM) i7-8557U CPU @ 1.70GHz' b'machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C' b'machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 AVX2 BMI2 INVPCID SMAP RDSEED ADX IPT SGX FPU_CSDS MPX CLFSOPT TSXFA IBRS STIBP L1DF SSBD' b'machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI' ----------Network Test---------- Setting timeout: 10 Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0009 sec, LOAD: 1.1944 sec. Timing for GluonNLP GitHub: https://github.com/dmlc/gluon-nlp, DNS: 0.0006 sec, LOAD: 1.1631 sec. Timing for GluonNLP: http://gluon-nlp.mxnet.io, DNS: 0.0008 sec, LOAD: 0.3726 sec. Timing for D2L: http://d2l.ai, DNS: 0.0008 sec, LOAD: 0.3690 sec. Timing for D2L (zh-cn): http://zh.d2l.ai, DNS: 0.0012 sec, LOAD: 0.3752 sec. Timing for FashionMNIST: https://repo.mxnet.io/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0017 sec, LOAD: 0.7922 sec. Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.1924 sec, LOAD: 3.0321 sec. Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0005 sec, LOAD: 0.7737 sec.

eric-haibin-lin commented 4 years ago

Hi @zburning for the master branch, we recently dropped mxnet 1.5 support. You need to install mxnet nightly version (pip install mxnet-cuxx --pre -U). We are expecting code freeze for mxnet 1.6 soon https://lists.apache.org/thread.html/dc49d2c4ed68adbde64f2e55296b1de4a9d089cfa8429d8c2ed33c51@ and will depend on mxnet 1.6 stable release once it's ready.