Closed BinchaoPeng closed 3 years ago
What code are you running that leads to that error?
from transformers import AutoModel, AutoTokenizer, pipeline
import torch
model_name = 'pre-model/' + 'longformer-encdec-base-16384'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
classifier = pipeline('feature-extraction', model=model, tokenizer=tokenizer)
# encoded_inputs = tokenizer(["ATGCATGCNACT"], ["ATGCATGCNACT"], return_token_type_ids=True, return_tensors='pt')
encoded_inputs = tokenizer(["ATGCATGCNACT", "ATGCATG", "ACTGGTCATGCAC"], return_tensors='pt',
padding=True)
print(encoded_inputs)
# feature = model(input_ids=encoded_inputs['input_ids'], attention_mask=encoded_inputs['attention_mask'],
# return_netsors='pt')
feature = model(**encoded_inputs,
return_netsors='pt')
print(feature[0])
print(type(feature[0]))
# feature = torch.as_tensor(feature)
# print(feature.shape)
print("***" * 48)
feature = classifier(["ATG", "ATGCATG", "ACTGGTCATGCAC"])
print(type(feature))
feature = torch.as_tensor(feature)
print(feature)
print(feature.shape)
print("***" * 48)
# Name Version Build Channel
_libgcc_mutex 0.1 main defaults
absl-py 0.12.0 pypi_0 pypi
astunparse 1.6.3 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
biopython 1.78 pypi_0 pypi
blas 1.0 mkl defaults
boto3 1.17.48 pypi_0 pypi
botocore 1.20.48 pypi_0 pypi
brotlipy 0.7.0 py36h27cfd23_1003 defaults
ca-certificates 2021.1.19 h06a4308_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cachetools 4.2.1 pypi_0 pypi
certifi 2020.12.5 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cffi 1.14.5 py36h261ae71_0 defaults
chardet 4.0.0 py36h06a4308_1003 defaults
click 7.1.2 pyhd3eb1b0_0 defaults
cryptography 3.4.7 py36hd23ed53_0 defaults
cudatoolkit 10.0.130 0 defaults
dataclasses 0.8 pyh4f3eec9_6 defaults
dill 0.3.3 pypi_0 pypi
filelock 3.0.12 pyhd3eb1b0_1 defaults
freetype 2.10.4 h5ab3b9f_0 defaults
future 0.18.2 pypi_0 pypi
google-auth 1.28.1 pypi_0 pypi
google-auth-oauthlib 0.4.4 pypi_0 pypi
grpcio 1.37.0 pypi_0 pypi
idna 2.10 pyhd3eb1b0_0 defaults
imageio 2.9.0 pypi_0 pypi
importlib-metadata 3.10.0 pypi_0 pypi
intel-openmp 2020.2 254 defaults
jmespath 0.10.0 pypi_0 pypi
joblib 1.0.1 pyhd3eb1b0_0 defaults
jpeg 9b h024ee3a_2 defaults
lcms2 2.12 h3be6417_0 defaults
ld_impl_linux-64 2.33.1 h53a641e_7 defaults
libffi 3.3 he6710b0_2 defaults
libgcc-ng 9.1.0 hdf63c60_0 defaults
libpng 1.6.37 hbc83047_0 defaults
libprotobuf 3.14.0 h8c45485_0 defaults
libstdcxx-ng 9.1.0 hdf63c60_0 defaults
libtiff 4.1.0 h2733197_1 defaults
longformer 0.1 pypi_0 pypi
lz4-c 1.9.3 h2531618_0 defaults
markdown 3.3.4 pypi_0 pypi
mkl 2020.2 256 defaults
mkl-service 2.3.0 py36he8ac12f_0 defaults
mkl_fft 1.3.0 py36h54f3939_0 defaults
mkl_random 1.1.1 py36h0573a6f_0 defaults
ncurses 6.2 he6710b0_1 defaults
ninja 1.10.2 py36hff7bd54_0 defaults
nlp 0.4.0 pypi_0 pypi
nltk 3.6.1 pypi_0 pypi
numpy 1.19.5 pypi_0 pypi
numpy-base 1.19.2 py36hfa32c7d_0 defaults
oauthlib 3.1.0 pypi_0 pypi
olefile 0.46 py36_0 defaults
openssl 1.1.1k h27cfd23_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
packaging 20.9 pyhd3eb1b0_0 defaults
pandas 1.1.5 pypi_0 pypi
patsy 0.5.1 pypi_0 pypi
pillow 8.2.0 py36he98fc37_0 defaults
pip 21.0.1 py36h06a4308_0 defaults
protobuf 3.15.8 pypi_0 pypi
pyahocorasick 1.4.2 pypi_0 pypi
pyarrow 3.0.0 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pybedtools 0.8.2 pypi_0 pypi
pycparser 2.20 py_2 defaults
pyopenssl 20.0.1 pyhd3eb1b0_1 defaults
pyparsing 2.4.7 pyhd3eb1b0_0 defaults
pysam 0.16.0.1 pypi_0 pypi
pysocks 1.7.1 py36h06a4308_0 defaults
python 3.6.13 hdb3f193_0 defaults
python-dateutil 2.8.1 pypi_0 pypi
python_abi 3.6 1_cp36m huggingface
pytorch-lightning 0.8.5 pypi_0 pypi
pytorch-transformers 1.2.0 pypi_0 pypi
pytz 2021.1 pypi_0 pypi
pyyaml 5.4.1 pypi_0 pypi
readline 8.1 h27cfd23_0 defaults
regex 2021.4.4 py36h27cfd23_0 defaults
requests 2.25.1 pyhd3eb1b0_0 defaults
requests-oauthlib 1.3.0 pypi_0 pypi
rouge-score 0.0.4 pypi_0 pypi
rsa 4.7.2 pypi_0 pypi
s3transfer 0.3.6 pypi_0 pypi
sacremoses 0.0.44 pypi_0 pypi
scikit-learn 0.24.1 pypi_0 pypi
scipy 1.5.4 pypi_0 pypi
sentencepiece 0.1.91 pypi_0 pypi
seqeval 1.2.2 pypi_0 pypi
setuptools 52.0.0 py36h06a4308_0 defaults
six 1.15.0 py36h06a4308_0 defaults
sqlite 3.35.4 hdfb4753_0 defaults
statsmodels 0.12.2 pypi_0 pypi
tensorboard 2.4.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.0 pypi_0 pypi
tensorboardx 2.2 pypi_0 pypi
test-tube 0.7.5 pypi_0 pypi
threadpoolctl 2.1.0 pypi_0 pypi
tk 8.6.10 hbc83047_0 defaults
tokenizers 0.5.0 pypi_0 pypi
torch 1.6.0 pypi_0 pypi
torchvision 0.5.0 py36_cu100 pytorch
tqdm 4.60.0 pypi_0 pypi
transformers 3.1.0 pypi_0 pypi
typing-extensions 3.7.4.3 pypi_0 pypi
urllib3 1.26.4 pyhd3eb1b0_0 defaults
werkzeug 1.0.1 pypi_0 pypi
wheel 0.36.2 pyhd3eb1b0_0 defaults
xxhash 2.0.2 pypi_0 pypi
xz 5.2.5 h7b6447c_0 defaults
zipp 3.4.1 pypi_0 pypi
zlib 1.2.11 h7b6447c_3 defaults
zstd 1.4.9 haebb681_0 defaults
# Name Version Build Channel
_libgcc_mutex 0.1 main https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
_tflow_select 2.1.0 gpu https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
absl-py 0.11.0 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
aiohttp 3.6.3 py36h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
apex 0.1 pypi_0 pypi
argon2-cffi 20.1.0 py36h7b6447c_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
astor 0.8.1 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
astunparse 1.6.3 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
async-timeout 3.0.1 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
async_generator 1.10 py36h28b3542_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
attrs 20.3.0 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
backcall 0.2.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
bert-serving-client 1.10.0 pypi_0 pypi
bert-serving-server 1.10.0 pypi_0 pypi
blas 1.0 mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
bleach 3.2.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
blinker 1.4 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
brotlipy 0.7.0 py36h27cfd23_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
c-ares 1.16.1 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ca-certificates 2021.4.13 h06a4308_1 defaults
cachetools 4.1.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
certifi 2020.12.5 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cffi 1.14.3 py36h261ae71_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
chardet 3.0.4 py36h06a4308_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
click 7.1.2 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cryptography 3.2.1 py36h3c74f83_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cudatoolkit 10.1.243 h6bb024c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cudnn 7.6.5 cuda10.1_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cupti 10.1.168 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cycler 0.10.0 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
dataclasses 0.7 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
dbus 1.13.18 hb2f20db_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
decorator 4.4.2 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
defusedxml 0.6.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
entrypoints 0.3 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
expat 2.2.10 he6710b0_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
filelock 3.0.12 pyhd3eb1b0_1 defaults
fontconfig 2.13.0 h9420a91_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
freetype 2.10.4 h5ab3b9f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gast 0.2.2 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
glib 2.66.1 h92f7085_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
google-auth 1.23.0 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
google-auth-oauthlib 0.4.2 pyhd3eb1b0_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
google-pasta 0.2.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gputil 1.4.0 pypi_0 pypi
grpcio 1.31.0 py36hf8bcb03_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gst-plugins-base 1.14.0 hbbd80ab_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gstreamer 1.14.0 hb31296c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
h5py 2.10.0 py36hd6299e0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
hdf5 1.10.6 hb1b8bf9_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
icu 58.2 he6710b0_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
idna 2.10 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
idna_ssl 1.1.0 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
importlib-metadata 2.0.0 py_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
importlib_metadata 2.0.0 1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
intel-openmp 2020.2 254 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ipykernel 5.3.4 py36h5ca1d4c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ipython 7.12.0 py36h5ca1d4c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ipython_genutils 0.2.0 pyhd3eb1b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ipywidgets 7.6.0 pyhd3eb1b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jedi 0.10.2 py36_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
jinja2 2.11.2 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
joblib 0.17.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jpeg 9b h024ee3a_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jsonschema 3.2.0 py_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jupyter 1.0.0 py36_7 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jupyter_client 6.1.7 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jupyter_console 6.2.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jupyter_core 4.7.0 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jupyterlab_pygments 0.1.2 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
keras 2.3.1 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
keras-applications 1.0.8 py_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
keras-base 2.3.1 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
keras-preprocessing 1.1.0 py_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
kiwisolver 1.3.0 py36h2531618_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
krb5 1.18.2 h173b8e3_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
lcms2 2.11 h396b838_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ld_impl_linux-64 2.33.1 h53a641e_7 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libcurl 7.71.1 h20c2e04_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libedit 3.1.20191231 h14c3975_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libffi 3.3 he6710b0_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgcc-ng 9.1.0 hdf63c60_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgfortran-ng 7.3.0 hdf63c60_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libpng 1.6.37 hbc83047_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libprotobuf 3.13.0.1 hd408876_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libsodium 1.0.18 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libssh2 1.9.0 h1ba5d50_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libstdcxx-ng 9.1.0 hdf63c60_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libtiff 4.1.0 h2733197_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libuuid 1.0.3 h1bed415_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libuv 1.40.0 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libxcb 1.14 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libxml2 2.9.10 hb55368b_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
lz4-c 1.9.2 heb0550a_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
markdown 3.3.3 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
markupsafe 1.1.1 py36h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
matplotlib 3.3.2 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
matplotlib-base 3.3.2 py36h817c723_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mistune 0.8.4 py36h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl 2020.2 256 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl-service 2.3.0 py36he904b0f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_fft 1.2.0 py36h23d657b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_random 1.1.1 py36h0573a6f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
multidict 4.7.6 py36h7b6447c_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
nbclient 0.5.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
nbconvert 6.0.7 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
nbformat 5.0.8 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ncurses 6.2 he6710b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
nest-asyncio 1.4.3 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ninja 1.10.1 py36hfd86e86_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
notebook 6.1.6 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy 1.19.2 py36h54aff64_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy-base 1.19.2 py36hfa32c7d_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
oauthlib 3.1.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
olefile 0.46 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
openssl 1.1.1k h27cfd23_0 defaults
opt_einsum 3.1.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
packaging 20.8 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pandas 1.1.3 py36he6710b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pandoc 2.11 hb0f4dca_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pandocfilters 1.4.3 py36h06a4308_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pcre 8.44 he6710b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pexpect 4.8.0 pyhd3eb1b0_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pickleshare 0.7.5 pyhd3eb1b0_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pillow 8.0.1 py36he98fc37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pip 20.2.4 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
prometheus_client 0.9.0 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
prompt-toolkit 3.0.8 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
prompt_toolkit 3.0.8 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
protobuf 3.13.0.1 py36he6710b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ptyprocess 0.6.0 pyhd3eb1b0_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyasn1 0.4.8 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyasn1-modules 0.2.8 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pycparser 2.20 py_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pygments 2.7.3 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyjwt 1.7.1 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyopenssl 19.1.0 pyhd3eb1b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyparsing 2.4.7 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyqt 5.9.2 py36h05f1152_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyrsistent 0.17.3 py36h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pysocks 1.7.1 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python 3.6.12 hcff3b4d_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python-dateutil 2.8.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python_abi 3.6 1_cp36m https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pytorch 1.7.0 py3.6_cuda10.1.243_cudnn7.6.3_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
pytz 2020.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyyaml 5.3.1 py36h7b6447c_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyzmq 20.0.0 py36h2531618_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
qt 5.9.7 h5867ecd_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
qtconsole 4.7.7 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
qtpy 1.9.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
readline 8.0 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
regex 2021.4.4 py36h27cfd23_0 defaults
requests 2.24.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
requests-oauthlib 1.3.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
rsa 4.6 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sacremoses 0.0.44 pypi_0 pypi
scikit-learn 0.23.2 py36h0573a6f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
scipy 1.5.2 py36h0b6359f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
seaborn 0.11.1 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
send2trash 1.5.0 pyhd3eb1b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
setuptools 50.3.1 py36h06a4308_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sip 4.19.8 py36hf484d3e_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
six 1.15.0 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sqlite 3.33.0 h62c20be_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorboard 2.3.0 pyh4dce500_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorboard-plugin-wit 1.6.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorflow 2.1.0 gpu_py36h2e5cdaa_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorflow-base 2.1.0 gpu_py36h6c5654b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorflow-estimator 2.1.0 pyhd54b08b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorflow-gpu 2.1.0 h0d30ee6_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
termcolor 1.1.0 py36_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
terminado 0.9.1 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
testpath 0.4.4 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
threadpoolctl 2.1.0 pyh5ca1d4c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tk 8.6.10 hbc83047_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tokenizers 0.10.2 pypi_0 pypi
torchaudio 0.7.0 py36 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
torchvision 0.1.8 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
tornado 6.0.4 py36h7b6447c_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tqdm 4.60.0 pypi_0 pypi
traitlets 4.3.3 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
transformers 4.4.2 py_0 huggingface
typing_extensions 3.7.4.3 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
urllib3 1.25.11 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
wcwidth 0.2.5 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
webencodings 0.5.1 py36_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
werkzeug 1.0.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
wheel 0.35.1 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
widgetsnbextension 3.5.1 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
wrapt 1.12.1 py36h7b6447c_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
xz 5.2.5 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
yaml 0.2.5 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
yarl 1.6.2 py36h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zeromq 4.3.3 he6710b0_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zipp 3.4.0 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zlib 1.2.11 h7b6447c_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zstd 1.4.5 h9ceee32_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
# Name Version Build Channel
_libgcc_mutex 0.1 main https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
absl-py 0.12.0 pypi_0 pypi
astunparse 1.6.3 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
blas 1.0 mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
boto3 1.17.53 pypi_0 pypi
botocore 1.20.53 pypi_0 pypi
brotlipy 0.7.0 py36h27cfd23_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ca-certificates 2021.4.13 h06a4308_1
cachetools 4.2.1 pypi_0 pypi
certifi 2020.12.5 py36h06a4308_0
cffi 1.14.5 py36h261ae71_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
chardet 4.0.0 py36h06a4308_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
click 7.1.2 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cryptography 3.4.7 py36hd23ed53_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cudatoolkit 10.0.130 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
dataclasses 0.8 pyh4f3eec9_6 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
dill 0.3.3 pypi_0 pypi
filelock 3.0.12 pyhd3eb1b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
freetype 2.10.4 h5ab3b9f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
future 0.18.2 pypi_0 pypi
google-auth 1.29.0 pypi_0 pypi
google-auth-oauthlib 0.4.4 pypi_0 pypi
grpcio 1.37.0 pypi_0 pypi
idna 2.10 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
imageio 2.9.0 pypi_0 pypi
importlib-metadata 2.0.0 py_1 anaconda
intel-openmp 2020.2 254 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jmespath 0.10.0 pypi_0 pypi
joblib 1.0.1 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jpeg 9b h024ee3a_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
lcms2 2.12 h3be6417_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ld_impl_linux-64 2.33.1 h53a641e_7 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libffi 3.3 he6710b0_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgcc-ng 9.1.0 hdf63c60_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libpng 1.6.37 hbc83047_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libprotobuf 3.14.0 h8c45485_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libstdcxx-ng 9.1.0 hdf63c60_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libtiff 4.1.0 h2733197_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
longformer 0.1 pypi_0 pypi
lz4-c 1.9.3 h2531618_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
markdown 3.3.4 pypi_0 pypi
mkl 2020.2 256 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl-service 2.3.0 py36he8ac12f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_fft 1.3.0 py36h54f3939_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_random 1.1.1 py36h0573a6f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ncurses 6.2 he6710b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ninja 1.10.2 py36hff7bd54_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
nlp 0.4.0 pypi_0 pypi
nltk 3.6.1 pypi_0 pypi
numpy 1.19.2 py36h54aff64_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy-base 1.19.2 py36hfa32c7d_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
oauthlib 3.1.0 pypi_0 pypi
olefile 0.46 py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
openssl 1.1.1k h27cfd23_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
packaging 20.9 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pandas 1.1.5 pypi_0 pypi
pillow 8.2.0 py36he98fc37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pip 21.0.1 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
protobuf 3.15.8 pypi_0 pypi
pyarrow 3.0.0 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycparser 2.20 py_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyopenssl 20.0.1 pyhd3eb1b0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyparsing 2.4.7 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pysocks 1.7.1 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python 3.6.13 hdb3f193_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python-dateutil 2.8.1 pypi_0 pypi
python_abi 3.6 1_cp36m huggingface
pytorch-lightning 0.8.5 pypi_0 pypi
pytorch-transformers 1.2.0 pypi_0 pypi
pytz 2021.1 pypi_0 pypi
pyyaml 5.4.1 pypi_0 pypi
readline 8.1 h27cfd23_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
regex 2021.4.4 py36h27cfd23_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
requests 2.25.1 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
requests-oauthlib 1.3.0 pypi_0 pypi
rouge-score 0.0.4 pypi_0 pypi
rsa 4.7.2 pypi_0 pypi
s3transfer 0.3.7 pypi_0 pypi
sacremoses 0.0.44 pypi_0 pypi
sentencepiece 0.1.95 pypi_0 pypi
setuptools 52.0.0 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
six 1.15.0 py36h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sqlite 3.35.4 hdfb4753_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorboard 2.4.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.0 pypi_0 pypi
tensorboardx 2.2 pypi_0 pypi
test-tube 0.7.5 pypi_0 pypi
tk 8.6.10 hbc83047_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tokenizers 0.8.1rc2 pypi_0 pypi
torch 1.6.0 pypi_0 pypi
torchvision 0.5.0 py36_cu100 pytorch
tqdm 4.60.0 pypi_0 pypi
transformers 3.1.0 pypi_0 pypi
urllib3 1.26.4 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
werkzeug 1.0.1 pypi_0 pypi
wheel 0.36.2 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
xxhash 2.0.2 pypi_0 pypi
xz 5.2.5 h7b6447c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zipp 3.4.1 pyhd3eb1b0_0
zlib 1.2.11 h7b6447c_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zstd 1.4.9 haebb681_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
next step i wan to use gene seqs to pretrain longformer, but i has been seemly dead in step 0 ....
Can you please post the output of:
type(model)
of your working environment?
In case it is showing something with ....BartModel
, can you please show us the definition of the class BertEncoder? You can locate it in the directory of:
import transformers
print(transformers.__file__)
Can you please post the output of:
type(model)
of your working environment? In case it is showing something with
....BartModel
, can you please show us the definition of the class BertEncoder? You can locate it in the directory of:import transformers print(transformers.__file__)
from transformers import AutoModel, AutoTokenizer # , pipeline
import transformers
print(transformers.__file__)
model_name = 'pre-model/' + 'longformer-encdec-base-16384'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
# classifier = pipeline('feature-extraction', model=model, tokenizer=tokenizer)
print(type(model))
/home/pbc/anaconda3/envs/dnabert/lib/python3.6/site-packages/transformers/__init__.py
Some weights of the model checkpoint at pre-model/longformer-encdec-base-16384 were not used when initializing BartModel: ['model.encoder.layers.0.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.0.self_attn.output.weight', 'model.encoder.layers.0.self_attn.output.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.1.self_attn.output.weight', 'model.encoder.layers.1.self_attn.output.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.2.self_attn.output.weight', 'model.encoder.layers.2.self_attn.output.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.3.self_attn.output.weight', 'model.encoder.layers.3.self_attn.output.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.4.self_attn.output.weight', 'model.encoder.layers.4.self_attn.output.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.5.self_attn.output.weight', 'model.encoder.layers.5.self_attn.output.bias']
- This IS expected if you are initializing BartModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BartModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BartModel were not initialized from the model checkpoint at pre-model/longformer-encdec-base-16384 and are newly initialized: ['model.encoder.layers.0.self_attn.k_proj.weight', 'model.encoder.layers.0.self_attn.k_proj.bias', 'model.encoder.layers.0.self_attn.v_proj.weight', 'model.encoder.layers.0.self_attn.v_proj.bias', 'model.encoder.layers.0.self_attn.q_proj.weight', 'model.encoder.layers.0.self_attn.q_proj.bias', 'model.encoder.layers.0.self_attn.out_proj.weight', 'model.encoder.layers.0.self_attn.out_proj.bias', 'model.encoder.layers.1.self_attn.k_proj.weight', 'model.encoder.layers.1.self_attn.k_proj.bias', 'model.encoder.layers.1.self_attn.v_proj.weight', 'model.encoder.layers.1.self_attn.v_proj.bias', 'model.encoder.layers.1.self_attn.q_proj.weight', 'model.encoder.layers.1.self_attn.q_proj.bias', 'model.encoder.layers.1.self_attn.out_proj.weight', 'model.encoder.layers.1.self_attn.out_proj.bias', 'model.encoder.layers.2.self_attn.k_proj.weight', 'model.encoder.layers.2.self_attn.k_proj.bias', 'model.encoder.layers.2.self_attn.v_proj.weight', 'model.encoder.layers.2.self_attn.v_proj.bias', 'model.encoder.layers.2.self_attn.q_proj.weight', 'model.encoder.layers.2.self_attn.q_proj.bias', 'model.encoder.layers.2.self_attn.out_proj.weight', 'model.encoder.layers.2.self_attn.out_proj.bias', 'model.encoder.layers.3.self_attn.k_proj.weight', 'model.encoder.layers.3.self_attn.k_proj.bias', 'model.encoder.layers.3.self_attn.v_proj.weight', 'model.encoder.layers.3.self_attn.v_proj.bias', 'model.encoder.layers.3.self_attn.q_proj.weight', 'model.encoder.layers.3.self_attn.q_proj.bias', 'model.encoder.layers.3.self_attn.out_proj.weight', 'model.encoder.layers.3.self_attn.out_proj.bias', 'model.encoder.layers.4.self_attn.k_proj.weight', 'model.encoder.layers.4.self_attn.k_proj.bias', 'model.encoder.layers.4.self_attn.v_proj.weight', 'model.encoder.layers.4.self_attn.v_proj.bias', 'model.encoder.layers.4.self_attn.q_proj.weight', 'model.encoder.layers.4.self_attn.q_proj.bias', 'model.encoder.layers.4.self_attn.out_proj.weight', 'model.encoder.layers.4.self_attn.out_proj.bias', 'model.encoder.layers.5.self_attn.k_proj.weight', 'model.encoder.layers.5.self_attn.k_proj.bias', 'model.encoder.layers.5.self_attn.v_proj.weight', 'model.encoder.layers.5.self_attn.v_proj.bias', 'model.encoder.layers.5.self_attn.q_proj.weight', 'model.encoder.layers.5.self_attn.q_proj.bias', 'model.encoder.layers.5.self_attn.out_proj.weight', 'model.encoder.layers.5.self_attn.out_proj.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
<class 'transformers.modeling_bart.BartModel'>
/home/pbc/anaconda3/envs/tf2_pt_kr2/lib/python3.6/site-packages/transformers-4.4.2-py3.8.egg/transformers/__init__.py
Some weights of the model checkpoint at pre-model/longformer-encdec-base-16384 were not used when initializing BartModel: ['model.encoder.layers.0.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.0.self_attn.output.weight', 'model.encoder.layers.0.self_attn.output.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.1.self_attn.output.weight', 'model.encoder.layers.1.self_attn.output.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.2.self_attn.output.weight', 'model.encoder.layers.2.self_attn.output.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.3.self_attn.output.weight', 'model.encoder.layers.3.self_attn.output.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.4.self_attn.output.weight', 'model.encoder.layers.4.self_attn.output.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.5.self_attn.output.weight', 'model.encoder.layers.5.self_attn.output.bias']
- This IS expected if you are initializing BartModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BartModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BartModel were not initialized from the model checkpoint at pre-model/longformer-encdec-base-16384 and are newly initialized: ['model.encoder.layers.0.self_attn.k_proj.weight', 'model.encoder.layers.0.self_attn.k_proj.bias', 'model.encoder.layers.0.self_attn.v_proj.weight', 'model.encoder.layers.0.self_attn.v_proj.bias', 'model.encoder.layers.0.self_attn.q_proj.weight', 'model.encoder.layers.0.self_attn.q_proj.bias', 'model.encoder.layers.0.self_attn.out_proj.weight', 'model.encoder.layers.0.self_attn.out_proj.bias', 'model.encoder.layers.1.self_attn.k_proj.weight', 'model.encoder.layers.1.self_attn.k_proj.bias', 'model.encoder.layers.1.self_attn.v_proj.weight', 'model.encoder.layers.1.self_attn.v_proj.bias', 'model.encoder.layers.1.self_attn.q_proj.weight', 'model.encoder.layers.1.self_attn.q_proj.bias', 'model.encoder.layers.1.self_attn.out_proj.weight', 'model.encoder.layers.1.self_attn.out_proj.bias', 'model.encoder.layers.2.self_attn.k_proj.weight', 'model.encoder.layers.2.self_attn.k_proj.bias', 'model.encoder.layers.2.self_attn.v_proj.weight', 'model.encoder.layers.2.self_attn.v_proj.bias', 'model.encoder.layers.2.self_attn.q_proj.weight', 'model.encoder.layers.2.self_attn.q_proj.bias', 'model.encoder.layers.2.self_attn.out_proj.weight', 'model.encoder.layers.2.self_attn.out_proj.bias', 'model.encoder.layers.3.self_attn.k_proj.weight', 'model.encoder.layers.3.self_attn.k_proj.bias', 'model.encoder.layers.3.self_attn.v_proj.weight', 'model.encoder.layers.3.self_attn.v_proj.bias', 'model.encoder.layers.3.self_attn.q_proj.weight', 'model.encoder.layers.3.self_attn.q_proj.bias', 'model.encoder.layers.3.self_attn.out_proj.weight', 'model.encoder.layers.3.self_attn.out_proj.bias', 'model.encoder.layers.4.self_attn.k_proj.weight', 'model.encoder.layers.4.self_attn.k_proj.bias', 'model.encoder.layers.4.self_attn.v_proj.weight', 'model.encoder.layers.4.self_attn.v_proj.bias', 'model.encoder.layers.4.self_attn.q_proj.weight', 'model.encoder.layers.4.self_attn.q_proj.bias', 'model.encoder.layers.4.self_attn.out_proj.weight', 'model.encoder.layers.4.self_attn.out_proj.bias', 'model.encoder.layers.5.self_attn.k_proj.weight', 'model.encoder.layers.5.self_attn.k_proj.bias', 'model.encoder.layers.5.self_attn.v_proj.weight', 'model.encoder.layers.5.self_attn.v_proj.bias', 'model.encoder.layers.5.self_attn.q_proj.weight', 'model.encoder.layers.5.self_attn.q_proj.bias', 'model.encoder.layers.5.self_attn.out_proj.weight', 'model.encoder.layers.5.self_attn.out_proj.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
File "/home/pbc/anaconda3/envs/tf2_pt_kr2/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-b1f8935f1cfa>", line 1, in <module>
runfile('/home/pbc/Documents/PycharmProjects/myEPI/src/github.py', wdir='/home/pbc/Documents/PycharmProjects/myEPI/src')
File "/home/pbc/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/202.7660.27/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/home/pbc/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/202.7660.27/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/pbc/Documents/PycharmProjects/myEPI/src/github.py", line 8, in <module>
model = AutoModel.from_pretrained(model_name)
File "/home/pbc/anaconda3/envs/tf2_pt_kr2/lib/python3.6/site-packages/transformers-4.4.2-py3.8.egg/transformers/models/auto/modeling_auto.py", line 815, in from_pretrained
pretrained_model_name_or_path, *model_args, config=config, **kwargs
File "/home/pbc/anaconda3/envs/tf2_pt_kr2/lib/python3.6/site-packages/transformers-4.4.2-py3.8.egg/transformers/modeling_utils.py", line 1183, in from_pretrained
model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for BartModel:
size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([16386, 768]) from checkpoint, the shape in current model is torch.Size([1026, 768]).
home/pbc/anaconda3/envs/dnabert/lib/python3.6/site-packages/transformers/__init__.py
Some weights of the model checkpoint at pre-model/longformer-encdec-base-16384 were not used when initializing BartModel: ['model.encoder.layers.0.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.0.self_attn.output.weight', 'model.encoder.layers.0.self_attn.output.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.1.self_attn.output.weight', 'model.encoder.layers.1.self_attn.output.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.2.self_attn.output.weight', 'model.encoder.layers.2.self_attn.output.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.3.self_attn.output.weight', 'model.encoder.layers.3.self_attn.output.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.4.self_attn.output.weight', 'model.encoder.layers.4.self_attn.output.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.5.self_attn.output.weight', 'model.encoder.layers.5.self_attn.output.bias']
- This IS expected if you are initializing BartModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BartModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BartModel were not initialized from the model checkpoint at pre-model/longformer-encdec-base-16384 and are newly initialized: ['model.encoder.layers.0.self_attn.k_proj.weight', 'model.encoder.layers.0.self_attn.k_proj.bias', 'model.encoder.layers.0.self_attn.v_proj.weight', 'model.encoder.layers.0.self_attn.v_proj.bias', 'model.encoder.layers.0.self_attn.q_proj.weight', 'model.encoder.layers.0.self_attn.q_proj.bias', 'model.encoder.layers.0.self_attn.out_proj.weight', 'model.encoder.layers.0.self_attn.out_proj.bias', 'model.encoder.layers.1.self_attn.k_proj.weight', 'model.encoder.layers.1.self_attn.k_proj.bias', 'model.encoder.layers.1.self_attn.v_proj.weight', 'model.encoder.layers.1.self_attn.v_proj.bias', 'model.encoder.layers.1.self_attn.q_proj.weight', 'model.encoder.layers.1.self_attn.q_proj.bias', 'model.encoder.layers.1.self_attn.out_proj.weight', 'model.encoder.layers.1.self_attn.out_proj.bias', 'model.encoder.layers.2.self_attn.k_proj.weight', 'model.encoder.layers.2.self_attn.k_proj.bias', 'model.encoder.layers.2.self_attn.v_proj.weight', 'model.encoder.layers.2.self_attn.v_proj.bias', 'model.encoder.layers.2.self_attn.q_proj.weight', 'model.encoder.layers.2.self_attn.q_proj.bias', 'model.encoder.layers.2.self_attn.out_proj.weight', 'model.encoder.layers.2.self_attn.out_proj.bias', 'model.encoder.layers.3.self_attn.k_proj.weight', 'model.encoder.layers.3.self_attn.k_proj.bias', 'model.encoder.layers.3.self_attn.v_proj.weight', 'model.encoder.layers.3.self_attn.v_proj.bias', 'model.encoder.layers.3.self_attn.q_proj.weight', 'model.encoder.layers.3.self_attn.q_proj.bias', 'model.encoder.layers.3.self_attn.out_proj.weight', 'model.encoder.layers.3.self_attn.out_proj.bias', 'model.encoder.layers.4.self_attn.k_proj.weight', 'model.encoder.layers.4.self_attn.k_proj.bias', 'model.encoder.layers.4.self_attn.v_proj.weight', 'model.encoder.layers.4.self_attn.v_proj.bias', 'model.encoder.layers.4.self_attn.q_proj.weight', 'model.encoder.layers.4.self_attn.q_proj.bias', 'model.encoder.layers.4.self_attn.out_proj.weight', 'model.encoder.layers.4.self_attn.out_proj.bias', 'model.encoder.layers.5.self_attn.k_proj.weight', 'model.encoder.layers.5.self_attn.k_proj.bias', 'model.encoder.layers.5.self_attn.v_proj.weight', 'model.encoder.layers.5.self_attn.v_proj.bias', 'model.encoder.layers.5.self_attn.q_proj.weight', 'model.encoder.layers.5.self_attn.q_proj.bias', 'model.encoder.layers.5.self_attn.out_proj.weight', 'model.encoder.layers.5.self_attn.out_proj.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/home/pbc/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/203.7148.72/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/home/pbc/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/203.7148.72/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/pbc/PycharmProjects/bert/github.py", line 7, in <module>
model = AutoModel.from_pretrained(model_name)
File "/home/pbc/anaconda3/envs/dnabert/lib/python3.6/site-packages/transformers/modeling_auto.py", line 523, in from_pretrained
return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
File "/home/pbc/anaconda3/envs/dnabert/lib/python3.6/site-packages/transformers/modeling_utils.py", line 972, in from_pretrained
model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for BartModel:
size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([16386, 768]) from checkpoint, the shape in current model is torch.Size([1026, 768]).
I found transformers.file all are different
Now please check this directory /home/pbc/anaconda3/envs/dnabert/lib/python3.6/site-packages/transformers/
and locate the file called modeling_bart.py
. Post the BertEncoder class definition here.
You should also pay attention to the weights that were not used from the pre-trained weights:
['model.encoder.layers.0.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.0.self_attn.output.weight', 'model.encoder.layers.0.self_attn.output.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.1.self_attn.output.weight', 'model.encoder.layers.1.self_attn.output.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.2.self_attn.output.weight', 'model.encoder.layers.2.self_attn.output.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.3.self_attn.output.weight', 'model.encoder.layers.3.self_attn.output.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.4.self_attn.output.weight', 'model.encoder.layers.4.self_attn.output.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.5.self_attn.output.weight', 'model.encoder.layers.5.self_attn.output.bias']
Are you sure that this model (including config and weights) can be used with the transformers AutoModel class? Currently, it looks to me that someone has built his own model with the transformers library (which is not supposed to work with the AutoClasses).
Now please check this directory
/home/pbc/anaconda3/envs/dnabert/lib/python3.6/site-packages/transformers/
and locate the file calledmodeling_bart.py
. Post the BertEncoder class definition here.You should also pay attention to the weights that were not used from the pre-trained weights:
['model.encoder.layers.0.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.0.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.0.self_attn.output.weight', 'model.encoder.layers.0.self_attn.output.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.1.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.1.self_attn.output.weight', 'model.encoder.layers.1.self_attn.output.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.2.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.2.self_attn.output.weight', 'model.encoder.layers.2.self_attn.output.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.3.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.3.self_attn.output.weight', 'model.encoder.layers.3.self_attn.output.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.4.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.4.self_attn.output.weight', 'model.encoder.layers.4.self_attn.output.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.query_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.key_global.bias', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.weight', 'model.encoder.layers.5.self_attn.longformer_self_attn.value_global.bias', 'model.encoder.layers.5.self_attn.output.weight', 'model.encoder.layers.5.self_attn.output.bias']
Are you sure that this model (including config and weights) can be used with the transformers AutoModel class? Currently, it looks to me that someone has built his own model with the transformers library (which is not supposed to work with the AutoClasses).
I am not sure, when I start to use bert model by the same way , it works, So I make it again in longformer, then many errors occured one by one
That is a different thing. What you have here longformer-encdec-base-16384
is something that is provided by someone that is not supposed to work with the provided AutoClasses by hugging face. Please check the code of this someone and see what this person did.
I think this is the repository you should check out: https://github.com/allenai/ms2
or maybe this code snippet: https://github.com/allenai/longformer/issues/154
That is a different thing. What you have here
longformer-encdec-base-16384
is something that is provided by someone that is not supposed to work with the provided AutoClasses by hugging face. Please check the code of this someone and see what this person did. I think this is the repository you should check out: https://github.com/allenai/ms2 or maybe this code snippet: allenai/longformer#154
yeah, you means that I should install env with allenai/longformer and not huggingface, at start I read allenai/longformer's readme, i just found that it may from huggingface and don't look for any things about how to use its longformer model by python code.
I have seen allenai/longformer#154, and I will try it through Imitating her code.
And another question is, if I want to use hugging face env to load model, that means I should download in https://huggingface.co/?
As for ms2, I will view it soon, Thanks!
Yes, the allenai/longformer
is the framework you should use for longformer-encdec-base-16384
.
And another question is, if I want to use hugging face env to load model, that means I should download in https://huggingface.co/?
Yes, you can check the pre-trained models here: https://huggingface.co/models
Okay, and I am curious that how do you find allenai/longformer#154
and https://github.com/allenai/ms2. If I have the skill, I can save myself quickly,haha.
Use a search engine of your choice and look for longformer-encdec-base-16384
;-)
longformer-encdec-base-16384
OK,thank you very much!
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
I use longformer model called longformer-encdec-base-16384 which is downloaded in https://github.com/allenai/longformer,and use huggingface to load the model,when transformers’ version is 3.1.0, the code can run, but when it is 4.4.2,the error happened.
MeanWhile,when I use the model to proposal pairs of sentences,I found it that the returned token_type_ids values are just zero without one. how ever,in the model's special_tokens_map.json, it has defined cls_token and sep_token.
Finally, I sincerely hope you would reply me soon. Thanks!