Unsuccessful TensorSliceReader Constructor with pretrained BERT

[JulianGerhard21]

Prerequisites

• [ x] Are you running the latest bert-as-service?
• [x ] Did you follow the installation and the usage instructions in README.md?
• [x ] Did you check the FAQ list in README.md?
• [x ] Did you perform a cursory search on existing issues?

System information

• OS Platform: Windows 10
• TensorFlow installed from (source or binary): binary
• TensorFlow version: 1.14.0
• Python version: 3.6.8
• bert-as-service version: newest
• GPU model and memory: GTX 1080 Ti

• CPU model and memory: Intel(R) Core(TM) i7-7700 CPU @ 3.60GH

Description

I'm using this command to start the server:

bert-serving-start -model_dir bert-cased-german/

Then this issue shows up:

                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = False
          device_map = []
       do_lower_case = True
  fixed_embed_length = False
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = None
        mask_cls_sep = False
      max_batch_size = 256
         max_seq_len = 25
           model_dir = bert-cased-german/
          num_worker = 1
       pooling_layer = [-2]
    pooling_strategy = REDUCE_MEAN
                port = 5555
            port_out = 5556
       prefetch_size = 10
 priority_batch_size = 16
show_tokens_to_client = False
     tuned_model_dir = None
             verbose = False
                 xla = False

I:VENTILATOR:[__i:__i: 67]:freeze, optimize and export graph, could take a while...
WARNING: Logging before flag parsing goes to stderr.
I0731 08:09:07.672543 140386794571584 __init__.py:67] freeze, optimize and export graph, could take a while...
W0731 08:09:07.679953 140386794571584 deprecation_wrapper.py:119] From /root/bert_venv/lib/python3.6/site-packages/bert_serving/server/helper.py:180: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

W0731 08:09:07.680354 140386794571584 deprecation_wrapper.py:119] From /root/bert_venv/lib/python3.6/site-packages/bert_serving/server/helper.py:180: The name tf.logging.ERROR is deprecated. Please use tf.compat.v1.logging.ERROR instead.

I:GRAPHOPT:[gra:opt: 52]:model config: bert-cased-german/bert_config.json
I0731 08:09:07.682091 140386794571584 graph.py:52] model config: bert-cased-german/bert_config.json
I:GRAPHOPT:[gra:opt: 55]:checkpoint: bert-cased-german/bert_model.ckpt
I0731 08:09:07.682227 140386794571584 graph.py:55] checkpoint: bert-cased-german/bert_model.ckpt
I:GRAPHOPT:[gra:opt: 59]:build graph...
I0731 08:09:07.683006 140386794571584 graph.py:59] build graph...
E:GRAPHOPT:[gra:opt:150]:fail to optimize the graph!
Traceback (most recent call last):
  File "/root/bert_venv/lib/python3.6/site-packages/bert_serving/server/graph.py", line 81, in optimize_graph
    ) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
  File "/root/bert_venv/lib/python3.6/site-packages/bert_serving/server/bert/modeling.py", line 331, in get_assignment_map_from_checkpoint
    init_vars = tf.train.list_variables(init_checkpoint)
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/training/checkpoint_utils.py", line 97, in list_variables
    reader = load_checkpoint(ckpt_dir_or_file)
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/training/checkpoint_utils.py", line 66, in load_checkpoint
    return pywrap_tensorflow.NewCheckpointReader(filename)
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 636, in NewCheckpointReader
    return CheckpointReader(compat.as_bytes(filepattern))
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 648, in __init__
    this = _pywrap_tensorflow_internal.new_CheckpointReader(filename)
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for bert-cased-german/bert_model.ckpt
E0731 08:09:11.066254 140386794571584 graph.py:150] fail to optimize the graph!
Traceback (most recent call last):
  File "/root/bert_venv/lib/python3.6/site-packages/bert_serving/server/graph.py", line 81, in optimize_graph
    ) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
  File "/root/bert_venv/lib/python3.6/site-packages/bert_serving/server/bert/modeling.py", line 331, in get_assignment_map_from_checkpoint
    init_vars = tf.train.list_variables(init_checkpoint)
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/training/checkpoint_utils.py", line 97, in list_variables
    reader = load_checkpoint(ckpt_dir_or_file)
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/training/checkpoint_utils.py", line 66, in load_checkpoint
    return pywrap_tensorflow.NewCheckpointReader(filename)
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 636, in NewCheckpointReader
    return CheckpointReader(compat.as_bytes(filepattern))
  File "/root/bert_venv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 648, in __init__
    this = _pywrap_tensorflow_internal.new_CheckpointReader(filename)
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for bert-cased-german/bert_model.ckpt
Traceback (most recent call last):
  File "/root/bert_venv/bin/bert-serving-start", line 10, in <module>
    sys.exit(main())
  File "/root/bert_venv/lib/python3.6/site-packages/bert_serving/server/cli/__init__.py", line 4, in main
    with BertServer(get_run_args()) as server:
  File "/root/bert_venv/lib/python3.6/site-packages/bert_serving/server/__init__.py", line 71, in __init__
    self.graph_path, self.bert_config = pool.apply(optimize_graph, (self.args,))
TypeError: 'NoneType' object is not iterable

The content of the model dir:

bert-base-german-cased.index bert-base-german-cased.meta bert-base-german.data-00000-of-00001 bert_config.json vocab.txt

I followed the Chinese Law Tutorial on the same machine and it works perfectly. Since I want to evaluate document classification especially for german, I want to serve the features for this pretrained BERT - tensorflow-version.

...

[qiu-nian]

@JulianGerhard21 hello! have you fixed this bug? I have the same proflem

[harper25]

@qiunian711

First, rename: bert-base-german.data-00000-of-00001 to bert-base-german-cased.data-00000-of-00001 so that you have index, meta and data files with the same name. The content of the folder should look like this:

bert-base-german-cased.index
bert-base-german-cased.meta
bert-base-german-cased.data-00000-of-00001
bert_config.json
vocab.txt

Now, you need to pass an additional parameter to bert-serving-start:

bert-serving-start -model_dir <folder-name> -ckpt_name bert-base-german-cased

This is because Bert expects ckpt_name = bert_model.ckpt by default, as shown in the log under ARG VALUE table.

Another option would be to rename index, meta and data files to bert_model.ckpt.index, etc.

[binhna]

@qiunian711

First, rename: bert-base-german.data-00000-of-00001 to bert-base-german-cased.data-00000-of-00001 so that you have index, meta and data files with the same name. The content of the folder should look like this:

bert-base-german-cased.index
bert-base-german-cased.meta
bert-base-german-cased.data-00000-of-00001
bert_config.json
vocab.txt

Now, you need to pass an additional parameter to bert-serving-start:

bert-serving-start -model_dir <folder-name> -ckpt_name bert-base-german-cased

This is because Bert expects ckpt_name = bert_model.ckpt by default, as shown in the log under ARG VALUE table.

Another option would be to rename index, meta and data files to bert_model.ckpt.index, etc.

It worked, but in my case, you have to rename 3 files index, meta and data to bert_model. If you add an additional param like -ckpt_name bert-base-german-cased, it will assume that you have a subfolder inside the pretrained model named bert-base-german-cased.

Thanks anyway

[meilanglang]

hello! have you fixed this bug? I have the same proflem

[anastasia-zhukova]

@meilanglang I made it work for me: 1) tensorflow==1.15.0 (tensorflow-gpu==1.15.0) Make sure to have CUDA = 10.0 installed 2) bert-serving-server==1.9.9 3) rename checkpoint files to:

• bert_model.ckpt.data-00000-of-00001
• bert_model.ckpt.index
• bert_model.ckpt.meta 4) set an absolute path to German BERT 5) among the parameters, set the following: max_seq_len=512 (or 128) and max_batch_size=1 6) go to this file of bert-serving-server, in my case it was in the virtual env: \venv\Lib\site-packages\bert_serving\server\graph.py. In the lines 61-63, replace the code with

  input_ids = tf.placeholder(tf.int32, (args.max_batch_size, args.max_seq_len), 'input_ids')
  input_mask = tf.placeholder(tf.int32, (args.max_batch_size, args.max_seq_len), 'input_mask')
  input_type_ids = tf.placeholder(tf.int32, (args.max_batch_size, args.max_seq_len), 'input_type_ids')

For debugging purpose, I used the following script from https://www.programmersought.com/article/20774372335/:

Server:

from bert_serving.server import BertServer
from bert_serving.server.helper import get_args_parser
def main():
    args = get_args_parser().parse_args(['-model_dir', r'C:\Users\annaz\PycharmProjects\bert-server\bert-base-german-cased',
                                         '-port', '86500',
                                         '-port_out', '86501',
                                         '-max_seq_len', '512',
                                        '-num_worker', '1',
                                         '-mask_cls_sep',
                                         # '-cpu',
                                         '-max_batch_size', '1'
                                         ])
    bs = BertServer(args)
    print("bert server start....")
    bs.start()

if __name__ == "__main__":
    main()

Client:

from bert_serving.client import BertClient
bc = BertClient(port=86500, port_out=86501, show_server_config=True, timeout=100000)
vec = bc.encode(['test', 'test 2'])

Running the command from the terminal still doesn't work for me, but this client-server code in .py files does it job.