search

graykode / xlnet-Pytorch

Simple XLNet implementation with Pytorch Wrapper
https://arxiv.org/pdf/1906.08237.pdf
Apache License 2.0
577 stars 107 forks source link

Error and general question #13

Open jbmaxwell opened 4 years ago

jbmaxwell commented 4 years ago

First the error — I get this both when trying to run the notebook locally (ubuntu 18.04) and from Colab: 

Traceback (most recent call last):
  File "main.py", line 89, in <module>
    num_predict=args.num_predict)
  File "/content/xlnet-Pytorch/data_utils.py", line 345, in make_permute
    reuse_len)
  File "/content/xlnet-Pytorch/data_utils.py", line 292, in _local_perm
    non_mask_tokens = (~is_masked) & non_func_tokens
RuntimeError: Expected object of scalar type Byte but got scalar type Bool for argument #2 'other' in call to _th_and

Any ideas?

For the general question; I really want to pretrain from scratch with my own small corpus. Any tips on how I might go about doing that?

Thanks

cmdllx commented 4 years ago

conda install pytorch=1.0.1 torchvision=0.2.2 cudatoolkit=10