failed to load bart model: unexpected EOF

[ior308]

Dear Matteo,

i try to build abstract q&a but I got this error:

... ... {"level":"debug","time":"2023-06-27T19:58:00+02:00","message":"1.45 GiB of 1.51 GiB (95%) downloaded"} {"level":"debug","time":"2023-06-27T19:58:03+02:00","message":"1.46 GiB of 1.51 GiB (96%) downloaded"} {"level":"debug","time":"2023-06-27T19:58:06+02:00","message":"1.47 GiB of 1.51 GiB (97%) downloaded"} {"level":"debug","time":"2023-06-27T19:58:09+02:00","message":"1.48 GiB of 1.51 GiB (97%) downloaded"} {"level":"debug","time":"2023-06-27T19:58:12+02:00","message":"1.49 GiB of 1.51 GiB (98%) downloaded"} {"level":"debug","time":"2023-06-27T19:58:15+02:00","message":"1.50 GiB of 1.51 GiB (99%) downloaded"} {"level":"debug","time":"2023-06-27T19:58:18+02:00","message":"1.51 GiB of 1.51 GiB (99%) downloaded"} {"level":"debug","time":"2023-06-27T19:58:18+02:00","message":"1.51 GiB (100%) downloaded"} {"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/vocab.json","destination":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-27T19:58:18+02:00","message":"downloading"} {"level":"debug","time":"2023-06-27T19:58:19+02:00","message":"877.76 KiB (100%) downloaded"} {"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/merges.txt","destination":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-27T19:58:19+02:00","message":"downloading"} {"level":"debug","time":"2023-06-27T19:58:20+02:00","message":"445.62 KiB (100%) downloaded"} {"level":"trace","time":"2023-06-27T19:58:32+02:00","message":"Reporting possible conversion mapping anomalies"} {"level":"trace","parameter":"model.decoder.layer_norm.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"model.decoder.layer_norm.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.dense.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.out_proj.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.dense.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"model.encoder.layer_norm.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.out_proj.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"model.encoder.layer_norm.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"model.encoder.embed_positions.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"lm_head.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.encoder.embed_tokens.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.decoder.embed_tokens.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.decoder.embed_positions.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"} Serializing model to "models/vblagoje/bart_lfqa/spago_model.bin"... signal: killed

I tried again, and I got:

[root@minix8 cybertron@v0.1.2]# GOARCH=amd64 CYBERTRON_MODEL=Helsinki-NLP/opus-mt-en-it CYBERTRON_MODELS_DIR=models go run ./examples/abstractivequestionasnwering/main.go {"level":"debug","file":"models/vblagoje/bart_lfqa/config.json","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/vblagoje/bart_lfqa/pytorch_model.bin","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"} {"level":"info","model":"models/vblagoje/bart_lfqa/spago_model.bin","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping conversion"} {"level":"fatal","error":"failed to load bart model: unexpected EOF","time":"2023-06-27T19:59:17+02:00"} exit status 1

Any ideas ?

Thank you.

[mooijtech]

Please see the tags of a model on HuggingFace before using: This model supports textgeneration: CYBERTRON_MODELS_DIR=models/ CYBERTRON_MODEL=vblagoje/bart_lfqa go run examples/textgeneration/main.go

"signal: killed" indicates you have run out of memory while converting the model.

It seems the config.json of this model is missing max_length tokens, so it will not work.

[ior308]

Yes, I launch the binary and it starts downloading the model. The config.json is present in models/vblagoje/bart_lfqa/config.json -rw-r--r-- 1 root root 1321 Jun 28 10:15 models/vblagoje/bart_lfqa/config.json

and "max_length" is 142.

My VM has 8GB but maybe it needs more..

{"level":"debug","time":"2023-06-28T10:22:39+02:00","message":"1.51 GiB of 1.51 GiB (99%) downloaded"} {"level":"debug","time":"2023-06-28T10:22:41+02:00","message":"1.51 GiB (100%) downloaded"} {"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/vocab.json","destination":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-28T10:22:41+02:00","message":"downloading"} {"level":"debug","time":"2023-06-28T10:22:43+02:00","message":"877.76 KiB (100%) downloaded"} {"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/merges.txt","destination":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-28T10:22:43+02:00","message":"downloading"} {"level":"debug","time":"2023-06-28T10:22:44+02:00","message":"445.62 KiB (100%) downloaded"} {"level":"trace","time":"2023-06-28T10:23:21+02:00","message":"Reporting possible conversion mapping anomalies"} {"level":"trace","parameter":"model.encoder.layer_norm.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.dense.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.out_proj.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.dense.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"model.decoder.layer_norm.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"classification_head.out_proj.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"model.decoder.layer_norm.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"model.encoder.layer_norm.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"} {"level":"trace","parameter":"lm_head.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.decoder.embed_positions.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.decoder.embed_tokens.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.encoder.embed_positions.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.encoder.embed_tokens.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"} Serializing model to "models/vblagoje/bart_lfqa/spago_model.bin"... Killed [root@minix8 cybertron@v0.1.2]# GOARCH=amd64 CYBERTRON_MODELS_DIR=models go build examples/abstractivequestionasnwering/main.go

[ior308]

I increased RAM to 16GB and the spago serializing succeed, but then I got this error message:

{"level":"trace","parameter":"model.decoder.embed_positions.weight","time":"2023-06-28T19:11:39+02:00","message":"parameter not mapped"} {"level":"trace","parameter":"model.decoder.embed_tokens.weight","time":"2023-06-28T19:11:39+02:00","message":"parameter not mapped"} Serializing model to "models/vblagoje/bart_lfqa/spago_model.bin"... Done.

panic: input sequence too long: 145 > 0

goroutine 1 [running]: main.main() /root/go/pkg/mod/github.com/nlpodyssey/cybertron@v0.1.2/examples/abstractivequestionasnwering/main.go:52 +0x50e

If I launch again, the same error message:

[root@minix8 cybertron@v0.1.2]# ./main {"level":"debug","file":"models/vblagoje/bart_lfqa/config.json","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/vblagoje/bart_lfqa/pytorch_model.bin","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"} {"level":"info","model":"models/vblagoje/bart_lfqa/spago_model.bin","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping conversion"} panic: input sequence too long: 145 > 0

goroutine 1 [running]: main.main() /root/go/pkg/mod/github.com/nlpodyssey/cybertron@v0.1.2/examples/abstractivequestionasnwering/main.go:52 +0x50e

[mooijtech]

Yes that's a bug due to config.json not specifying max_length in the root. I set those values manually but then I get an error about slice length.

[ior308]

I'm using this model:

./models/vblagoje/bart_lfqa/config.json

and config.json contains max_length=142

[mooijtech]

I know, but the code doesn't look at task_specific_params.summarization.max_length.

[ior308]

Ok, so what do I need to do to make this example work ?

Do you know which model to use for the Q&A example ?

[ior308]

I downloaded models/deepset/bert-base-cased-squad2/ and now Q&A example works, BUT I don't understand the output. I supposed it should be able to answer to my question based on paragraph content..

./main {"level":"debug","file":"models/deepset/bert-base-cased-squad2/config.json","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/deepset/bert-base-cased-squad2/pytorch_model.bin","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/deepset/bert-base-cased-squad2/vocab.txt","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/deepset/bert-base-cased-squad2/tokenizer_config.json","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"} {"level":"info","model":"models/deepset/bert-base-cased-squad2/spago_model.bin","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping conversion"} paragraph.txt

what is artificial intelligence ? 0.249498236 { "Answers": [ { "Text": "txt", "Start": 10, "End": 13, "Score": 0.4198876741953599 }, { "Text": "paragraph.txt", "Start": 0, "End": 13, "Score": 0.2049869292790566 }, { "Text": ".txt", "Start": 9, "End": 13, "Score": 0.15559880539241716 } ] }

[mooijtech]

The t5 models may be more suited but are currently unsupported.

[ior308]

Ok, so I don't understand if this project is still maintained. What I should do to test Abstract Q&A examples over a text ??? :)

[mooijtech]

Try: https://github.com/nlpodyssey/verbaflow But I don't think you have enough memory.

[ior308]

Enough memory for SPAGO serialization ? Is it possible to do a buffered serialization instead of loading all into memory ?

Il giorno ven 30 giu 2023 alle ore 16:09 Marten Mooij < @.***> ha scritto:

Try: https://github.com/nlpodyssey/verbaflow But I don't think you have enough memory.

— Reply to this email directly, view it on GitHub https://github.com/nlpodyssey/cybertron/issues/24#issuecomment-1614709673, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL7ZDKHNQUEOCUO4O336SFLXN3M3LANCNFSM6AAAAAAZWAFBMM . You are receiving this because you authored the thread.Message ID: @.***>

--

Quando cambi il modo di guardare le cose, le cose che guardi cambieranno !! (Max Planck)

[mooijtech]

Sure, if you have time you can contribute the changes yourself, it's open source after all. Work is also being done on a new serialization format: https://github.com/nlpodyssey/safetensors

[mooijtech]

BART currently has no Question Answering support

[ior308]

It would be very interesting to work on these projects, but first I have to understand how they work and at the moment I still haven't figured out how I can use the Abstractive Q&A example... :(

I remember some years ago I was able to get the example working from Matteo's repository, but today the model is no longer available and I was told to go to https://github.com/nlpodyssey/cybertron. Unfortunately, the very example I was interested in investigating doesn't seem to work yet and I don't understand why. Is it a cybertron bug?

Il giorno sab 1 lug 2023 alle ore 12:42 Marten Mooij < @.***> ha scritto:

Sure, if you have time you can contribute the changes yourself, it's open source after all. Work is also being done on a new serialization format: https://github.com/nlpodyssey/safetensors

— Reply to this email directly, view it on GitHub https://github.com/nlpodyssey/cybertron/issues/24#issuecomment-1615842559, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL7ZDKGAQSAUPGEJSLC555DXN75KPANCNFSM6AAAAAAZWAFBMM . You are receiving this because you authored the thread.Message ID: @.***>

--

Quando cambi il modo di guardare le cose, le cose che guardi cambieranno !! (Max Planck)

[mooijtech]

Only BERT supports Q&A so you will have to find (or train?) a model based on BERT.

[ior308]

Some time ago, I did a test with code at https://github.com/matteo-grella/gophercon-eu-2021/blob/main/examples/question_answering/main.go

If you open it you can find " github.com/nlpodyssey/spago/pkg/nlp/transformers/bert".

At that time it worked. I Used a pdf manual text as input and I did some questions about the content.

Now this example does not work because Matteo created a new project. So BERT should be exist into your repository..

Il giorno mar 11 lug 2023 alle ore 14:29 Marten Mooij < @.***> ha scritto:

Only BERT supports Q&A so you will have to find (or train?) a model based on BERT.

— Reply to this email directly, view it on GitHub https://github.com/nlpodyssey/cybertron/issues/24#issuecomment-1630736367, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL7ZDKFFRFRG5YAEIH2LPGDXPVBKHANCNFSM6AAAAAAZWAFBMM . You are receiving this because you authored the thread.Message ID: @.***>

--

Quando cambi il modo di guardare le cose, le cose che guardi cambieranno !! (Max Planck)

[matteo-grella]

Sorry guys I've been a bit far from the project due to other compelling priorities. I'll be back asap and provide you support.

[ior308]

Many thank Matteo. I'd like to use the features made available by the project, downloading some templates from https://huggingface.co/openai-gpt, but right now I'm having some difficulties getting the examples to work.

Regards

Il giorno mer 2 ago 2023 alle ore 00:38 Matteo Grella < @.***> ha scritto:

Sorry guys I've been a bit far from the project due to other compelling priorities. I'll be back asap and provide you support.

— Reply to this email directly, view it on GitHub https://github.com/nlpodyssey/cybertron/issues/24#issuecomment-1661192776, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL7ZDKEU6P3BKDAO5HB7FTDXTGAORANCNFSM6AAAAAAZWAFBMM . You are receiving this because you authored the thread.Message ID: @.***>

--

Quando cambi il modo di guardare le cose, le cose che guardi cambieranno !! (Max Planck)

[ior308]

Dear, now Q&A works for me. Instead using an external paragraph file as in gopher examples, I put the content into a variable:

[root@minix8 cybertron@v0.1.2]# sh exa1.sh {"level":"debug","file":"models/deepset/bert-base-cased-squad2/config.json","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/deepset/bert-base-cased-squad2/pytorch_model.bin","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/deepset/bert-base-cased-squad2/vocab.txt","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"} {"level":"debug","file":"models/deepset/bert-base-cased-squad2/tokenizer_config.json","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"} {"level":"info","model":"models/deepset/bert-base-cased-squad2/spago_model.bin","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping conversion"} Go is a programming language designed at Google in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson to address criticism of other languages.

when go has been designed ? 0.93027006 { "Answers": [ { "Text": "2007", "Start": 51, "End": 55, "Score": 0.9280015216017522 } ] }