search

skeskinen / bert.cpp

ggml implementation of BERT
MIT License
463 stars 58 forks source link

Segfault on large inputs? #37

Open corani opened 1 year ago

corani commented 1 year ago

When I run the the build/bin/main example with a larger input I get a segfault:

ggml_new_tensor_impl: not enough space in the context's memory pool (needed 271388624, available 260703040)
Segmentation fault

I can work around this by doing an N *= 2; near the bottom of bert_load_from_file, but obviously that isn't the right solution. It seems somewhere a calculation is off (probably with mem_per_token).

ztkg257 commented 7 months ago

@corani is this issue solved by N *= 2, can you please let me know at what file and line need to change this. Thank you in advance.

corani commented 7 months ago

I believe here: https://github.com/skeskinen/bert.cpp/blob/master/bert.cpp#L687