std::bad_alloc

[ice-melt]

• env:ubuntu18.04,python3.6,
• describe:when i run the command as follow,there is a error (std::bad_alloc ),how can i fix it?

ice-melt@DELL:~$ bert-serving-start -model_dir /home/ice-melt/disk/DATASET/BERT/chinese_L-12_H-768_A-12/ -num_worker=1 
/usr/lib/python3/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.24.3) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
usage: /home/ice-melt/.local/bin/bert-serving-start -model_dir /home/ice-melt/disk/DATASET/BERT/chinese_L-12_H-768_A-12/ -num_worker=1
                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = False
          device_map = []
       do_lower_case = True
  fixed_embed_length = False
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = None
        mask_cls_sep = False
      max_batch_size = 256
         max_seq_len = 25
           model_dir = /home/ice-melt/disk/DATASET/BERT/chinese_L-12_H-768_A-12/
          num_worker = 1
       pooling_layer = [-2]
    pooling_strategy = REDUCE_MEAN
                port = 5555
            port_out = 5556
       prefetch_size = 10
 priority_batch_size = 16
show_tokens_to_client = False
     tuned_model_dir = None
             verbose = False
                 xla = False

I:VENTILATOR:[__i:__i: 66]:freeze, optimize and export graph, could take a while...
I:GRAPHOPT:[gra:opt: 52]:model config: /home/ice-melt/disk/DATASET/BERT/chinese_L-12_H-768_A-12/bert_config.json
I:GRAPHOPT:[gra:opt: 55]:checkpoint: /home/ice-melt/disk/DATASET/BERT/chinese_L-12_H-768_A-12/bert_model.ckpt
I:GRAPHOPT:[gra:opt: 59]:build graph...

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

I:GRAPHOPT:[gra:opt:128]:load parameters from checkpoint...
I:GRAPHOPT:[gra:opt:132]:optimize...
I:GRAPHOPT:[gra:opt:140]:freeze...
I:GRAPHOPT:[gra:opt:145]:write graph to a tmp file: /tmp/tmp1_02bsus
terminate called after throwing an instance of 'std::bad_alloc'

[kevin-rowney]

I'm getting it as well. The threads appear to report that they are "ready and listening!" but the client hangs when I try to connect. Wondering if it's because I don't have enough RAM on this instance. Am all the way up to a 13GB RAM instance, so maybe I should try for 16GB....or larger?

@ice-melt: How much RAM was on your system when you ran this?

[mn3mos]

Same issue here with 6GB of RAM

[yunoJ]

Same issue

[bimhud]

I had the same issue, with 8GB machine, after upgraded to 32GB and I'm able to run with 2 workers.

However, when running more than 2 workers on 8 Cores 32GB machine, the issue appears again.

There are heaps of memory available

See the log below

(bert-as-service) sh-4.2$ bert-serving-start -model_dir ./multi_cased_L-12_H-768_A-12 -num_worker=3 -cpu ............. OMP: Info #250: KMP_AFFINITY: pid 26690 tid 26690 thread 0 bound to OS proc set 0 OMP: Info #250: KMP_AFFINITY: pid 26687 tid 26753 thread 1 bound to OS proc set 1 OMP: Info #250: KMP_AFFINITY: pid 26690 tid 26776 thread 1 bound to OS proc set 1 OMP: Info #250: KMP_AFFINITY: pid 26684 tid 26751 thread 1 bound to OS proc set 1 terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc I:WORKER-2:[i:gen:558]:ready and listening! I0913 06:26:32.240278 140128425056000 init.py:558] ready and listening! I:WORKER-0:[i:gen:558]:ready and listening! I0913 06:26:32.241424 140128156620544 init.py:558] ready and listening!

[gitabtion]

i had the same issue, with 8Cores 16GB machine

[gitabtion]

I solved this porblem with seting overcommit_memory to 0 on my linux machine，you could do this when you have enough memory.