search

google-research / bert

TensorFlow code and pre-trained models for BERT
https://arxiv.org/abs/1810.04805
Apache License 2.0
37.8k stars 9.56k forks source link

run run_classifier on colab with TPU got "^C" after the first checkpoint. #900

Open mzhl1111 opened 4 years ago

mzhl1111 commented 4 years ago 
WARNING:tensorflow:From /content/558-project/optimization.py:87: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING:tensorflow:From 558-project/run_classifier.py:1037: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

WARNING:tensorflow:From 558-project/run_classifier.py:838: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

W1104 22:22:27.343141 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:838: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From 558-project/run_classifier.py:838: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

W1104 22:22:27.343362 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:838: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /content/558-project/modeling.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W1104 22:22:27.343837 140008086570880 module_wrapper.py:139] From /content/558-project/modeling.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From 558-project/run_classifier.py:864: The name tf.gfile.MakeDirs is deprecated. Please use tf.io.gfile.makedirs instead.

W1104 22:22:27.345377 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:864: The name tf.gfile.MakeDirs is deprecated. Please use tf.io.gfile.makedirs instead.

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W1104 22:22:27.429106 140008086570880 lazy_loader.py:50] 
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

I1104 22:22:27.965064 140008086570880 utils.py:141] NumExpr defaulting to 2 threads.
WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f55fbe07048>) includes params argument, but params are not passed to Estimator.
W1104 22:22:29.017828 140008086570880 estimator.py:1994] Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f55fbe07048>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:Using config: {'_model_dir': './bert_output/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f55e910acf8>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}
I1104 22:22:29.019255 140008086570880 estimator.py:212] Using config: {'_model_dir': './bert_output/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f55e910acf8>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}
INFO:tensorflow:_TPUContext: eval_on_tpu True
I1104 22:22:29.019627 140008086570880 tpu_context.py:220] _TPUContext: eval_on_tpu True
WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.
W1104 22:22:29.020117 140008086570880 tpu_context.py:222] eval_on_tpu ignored because use_tpu is False.
WARNING:tensorflow:From 558-project/run_classifier.py:537: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

W1104 22:22:29.020364 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:537: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

WARNING:tensorflow:From 558-project/run_classifier.py:541: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

W1104 22:22:29.020726 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:541: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

INFO:tensorflow:Writing example 0 of 127864
I1104 22:22:29.020849 140008086570880 run_classifier.py:541] Writing example 0 of 127864
INFO:tensorflow:*** Example ***
I1104 22:22:29.022133 140008086570880 run_classifier.py:515] *** Example ***
INFO:tensorflow:guid: train-0
I1104 22:22:29.022325 140008086570880 run_classifier.py:516] guid: train-0
INFO:tensorflow:tokens: [CLS] j ##une – m ##oc ##te ##zu ##ma ii , a ##z ##tec ruler of ten ##och ##ti ##tl ##an , is declared deposed due to his captivity by con ##quist ##ador her ##nan co ##rte ##s . [SEP] the s ##wed ##ish regent s ##ten s ##ture the younger is mortal ##ly wounded in the battle . [SEP]
I1104 22:22:29.022446 140008086570880 run_classifier.py:518] tokens: [CLS] j ##une – m ##oc ##te ##zu ##ma ii , a ##z ##tec ruler of ten ##och ##ti ##tl ##an , is declared deposed due to his captivity by con ##quist ##ador her ##nan co ##rte ##s . [SEP] the s ##wed ##ish regent s ##ten s ##ture the younger is mortal ##ly wounded in the battle . [SEP]
INFO:tensorflow:input_ids: 101 179 10038 782 182 13335 1566 10337 1918 25550 117 170 1584 19000 7778 1104 1995 9962 3121 26414 1389 117 1110 3332 19866 1496 1106 1117 17737 1118 14255 19854 21784 1123 6509 1884 22460 1116 119 102 1103 188 11547 2944 19611 188 5208 188 5332 1103 3247 1110 11647 1193 4670 1107 1103 2321 119 102 0 0 0 0
I1104 22:22:29.022584 140008086570880 run_classifier.py:519] input_ids: 101 179 10038 782 182 13335 1566 10337 1918 25550 117 170 1584 19000 7778 1104 1995 9962 3121 26414 1389 117 1110 3332 19866 1496 1106 1117 17737 1118 14255 19854 21784 1123 6509 1884 22460 1116 119 102 1103 188 11547 2944 19611 188 5208 188 5332 1103 3247 1110 11647 1193 4670 1107 1103 2321 119 102 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
I1104 22:22:29.022703 140008086570880 run_classifier.py:520] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
I1104 22:22:29.022846 140008086570880 run_classifier.py:521] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
INFO:tensorflow:label: 1 (id = 1)
I1104 22:22:29.022929 140008086570880 run_classifier.py:522] label: 1 (id = 1)
INFO:tensorflow:*** Example ***
I1104 22:22:29.024294 140008086570880 run_classifier.py:515] *** Example ***
INFO:tensorflow:guid: train-1
I1104 22:22:29.024427 140008086570880 run_classifier.py:516] guid: train-1
INFO:tensorflow:tokens: [CLS] the population was 1 , 09 ##7 at the 2010 census . [SEP] like other la ##tino neighborhoods in new yo ##rk city , bush ##wick has an established pu ##ert ##o r ##ica ##n presence , along with an influx of many do ##mini ##cans , south am ##eric ##ans , central am ##eric ##ans , me ##xi ##cans , as [SEP]
I1104 22:22:29.024554 140008086570880 run_classifier.py:518] tokens: [CLS] the population was 1 , 09 ##7 at the 2010 census . [SEP] like other la ##tino neighborhoods in new yo ##rk city , bush ##wick has an established pu ##ert ##o r ##ica ##n presence , along with an influx of many do ##mini ##cans , south am ##eric ##ans , central am ##eric ##ans , me ##xi ##cans , as [SEP]
INFO:tensorflow:input_ids: 101 1103 1416 1108 122 117 4925 1559 1120 1103 1333 2314 119 102 1176 1168 2495 20064 12073 1107 1207 26063 4661 1331 117 13771 6196 1144 1126 1628 23609 7340 1186 187 4578 1179 2915 117 1373 1114 1126 19540 1104 1242 1202 25685 21242 117 1588 1821 26237 5443 117 2129 1821 26237 5443 117 1143 8745 21242 117 1112 102
I1104 22:22:29.024679 140008086570880 run_classifier.py:519] input_ids: 101 1103 1416 1108 122 117 4925 1559 1120 1103 1333 2314 119 102 1176 1168 2495 20064 12073 1107 1207 26063 4661 1331 117 13771 6196 1144 1126 1628 23609 7340 1186 187 4578 1179 2915 117 1373 1114 1126 19540 1104 1242 1202 25685 21242 117 1588 1821 26237 5443 117 2129 1821 26237 5443 117 1143 8745 21242 117 1112 102
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
I1104 22:22:29.024808 140008086570880 run_classifier.py:520] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
I1104 22:22:29.024909 140008086570880 run_classifier.py:521] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
INFO:tensorflow:label: 0 (id = 0)
I1104 22:22:29.024986 140008086570880 run_classifier.py:522] label: 0 (id = 0)
INFO:tensorflow:*** Example ***
I1104 22:22:29.025592 140008086570880 run_classifier.py:515] *** Example ***
INFO:tensorflow:guid: train-2
I1104 22:22:29.025707 140008086570880 run_classifier.py:516] guid: train-2
INFO:tensorflow:tokens: [CLS] euro ##pe and the is ##lam ##ic world : a history . [SEP] there are no plans to re ##su ##rre ##ct it . [SEP]
I1104 22:22:29.025813 140008086570880 run_classifier.py:518] tokens: [CLS] euro ##pe and the is ##lam ##ic world : a history . [SEP] there are no plans to re ##su ##rre ##ct it . [SEP]
INFO:tensorflow:input_ids: 101 27772 3186 1105 1103 1110 7609 1596 1362 131 170 1607 119 102 1175 1132 1185 2714 1106 1231 6385 11604 5822 1122 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1104 22:22:29.025919 140008086570880 run_classifier.py:519] input_ids: 101 27772 3186 1105 1103 1110 7609 1596 1362 131 170 1607 119 102 1175 1132 1185 2714 1106 1231 6385 11604 5822 1122 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1104 22:22:29.026021 140008086570880 run_classifier.py:520] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1104 22:22:29.026119 140008086570880 run_classifier.py:521] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:label: 0 (id = 0)
I1104 22:22:29.026195 140008086570880 run_classifier.py:522] label: 0 (id = 0)
INFO:tensorflow:*** Example ***
I1104 22:22:29.027196 140008086570880 run_classifier.py:515] *** Example ***
INFO:tensorflow:guid: train-3
I1104 22:22:29.027315 140008086570880 run_classifier.py:516] guid: train-3
INFO:tensorflow:tokens: [CLS] even where only a small charge is produced , it can result in dust particles being attracted to the rubbed surface . [SEP] the c ##lar ##ion - limestone area school district provides kindergarten through 12th grade public education for the community . [SEP]
I1104 22:22:29.027419 140008086570880 run_classifier.py:518] tokens: [CLS] even where only a small charge is produced , it can result in dust particles being attracted to the rubbed surface . [SEP] the c ##lar ##ion - limestone area school district provides kindergarten through 12th grade public education for the community . [SEP]
INFO:tensorflow:input_ids: 101 1256 1187 1178 170 1353 2965 1110 1666 117 1122 1169 1871 1107 6693 9150 1217 5666 1106 1103 6987 2473 119 102 1103 172 5815 1988 118 10718 1298 1278 1629 2790 15001 1194 5247 3654 1470 1972 1111 1103 1661 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1104 22:22:29.027522 140008086570880 run_classifier.py:519] input_ids: 101 1256 1187 1178 170 1353 2965 1110 1666 117 1122 1169 1871 1107 6693 9150 1217 5666 1106 1103 6987 2473 119 102 1103 172 5815 1988 118 10718 1298 1278 1629 2790 15001 1194 5247 3654 1470 1972 1111 1103 1661 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1104 22:22:29.027642 140008086570880 run_classifier.py:520] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1104 22:22:29.027749 140008086570880 run_classifier.py:521] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:label: 0 (id = 0)
I1104 22:22:29.027827 140008086570880 run_classifier.py:522] label: 0 (id = 0)
INFO:tensorflow:*** Example ***
I1104 22:22:29.029625 140008086570880 run_classifier.py:515] *** Example ***
INFO:tensorflow:guid: train-4
I1104 22:22:29.029757 140008086570880 run_classifier.py:516] guid: train-4
INFO:tensorflow:tokens: [CLS] the highlight of c ##roa ##tia ##s recent infrastructure developments is its rapidly developed motorway network , largely built in the late 1990s and especially in the 2000s . [SEP] the closest analogy with the modern web browser would be to create a list of book ##marks to articles relevant to a topic , and then to have some mechanism for automatically [SEP]
I1104 22:22:29.030055 140008086570880 run_classifier.py:518] tokens: [CLS] the highlight of c ##roa ##tia ##s recent infrastructure developments is its rapidly developed motorway network , largely built in the late 1990s and especially in the 2000s . [SEP] the closest analogy with the modern web browser would be to create a list of book ##marks to articles relevant to a topic , and then to have some mechanism for automatically [SEP]
INFO:tensorflow:input_ids: 101 1103 13426 1104 172 27380 10691 1116 2793 6557 9093 1110 1157 5223 1872 15316 2443 117 3494 1434 1107 1103 1523 3281 1105 2108 1107 1103 8509 119 102 1103 7064 26799 1114 1103 2030 5127 19089 1156 1129 1106 2561 170 2190 1104 1520 22328 1106 4237 7503 1106 170 8366 117 1105 1173 1106 1138 1199 6978 1111 7743 102
I1104 22:22:29.030241 140008086570880 run_classifier.py:519] input_ids: 101 1103 13426 1104 172 27380 10691 1116 2793 6557 9093 1110 1157 5223 1872 15316 2443 117 3494 1434 1107 1103 1523 3281 1105 2108 1107 1103 8509 119 102 1103 7064 26799 1114 1103 2030 5127 19089 1156 1129 1106 2561 170 2190 1104 1520 22328 1106 4237 7503 1106 170 8366 117 1105 1173 1106 1138 1199 6978 1111 7743 102
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
I1104 22:22:29.030360 140008086570880 run_classifier.py:520] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
I1104 22:22:29.030467 140008086570880 run_classifier.py:521] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
INFO:tensorflow:label: 0 (id = 0)
I1104 22:22:29.030546 140008086570880 run_classifier.py:522] label: 0 (id = 0)
INFO:tensorflow:Writing example 10000 of 127864
I1104 22:22:37.320161 140008086570880 run_classifier.py:541] Writing example 10000 of 127864
INFO:tensorflow:Writing example 20000 of 127864
I1104 22:22:45.621928 140008086570880 run_classifier.py:541] Writing example 20000 of 127864
INFO:tensorflow:Writing example 30000 of 127864
I1104 22:22:54.140741 140008086570880 run_classifier.py:541] Writing example 30000 of 127864
INFO:tensorflow:Writing example 40000 of 127864
I1104 22:23:02.613651 140008086570880 run_classifier.py:541] Writing example 40000 of 127864
INFO:tensorflow:Writing example 50000 of 127864
I1104 22:23:11.028967 140008086570880 run_classifier.py:541] Writing example 50000 of 127864
INFO:tensorflow:Writing example 60000 of 127864
I1104 22:23:19.384223 140008086570880 run_classifier.py:541] Writing example 60000 of 127864
INFO:tensorflow:Writing example 70000 of 127864
I1104 22:23:27.774798 140008086570880 run_classifier.py:541] Writing example 70000 of 127864
INFO:tensorflow:Writing example 80000 of 127864
I1104 22:23:36.249463 140008086570880 run_classifier.py:541] Writing example 80000 of 127864
INFO:tensorflow:Writing example 90000 of 127864
I1104 22:23:44.874969 140008086570880 run_classifier.py:541] Writing example 90000 of 127864
INFO:tensorflow:Writing example 100000 of 127864
I1104 22:23:53.373614 140008086570880 run_classifier.py:541] Writing example 100000 of 127864
INFO:tensorflow:Writing example 110000 of 127864
I1104 22:24:01.984724 140008086570880 run_classifier.py:541] Writing example 110000 of 127864
INFO:tensorflow:Writing example 120000 of 127864
I1104 22:24:10.430072 140008086570880 run_classifier.py:541] Writing example 120000 of 127864
INFO:tensorflow:***** Running training *****
I1104 22:24:16.899918 140008086570880 run_classifier.py:927] ***** Running training *****
INFO:tensorflow:  Num examples = 127864
I1104 22:24:16.900168 140008086570880 run_classifier.py:928]   Num examples = 127864
INFO:tensorflow:  Batch size = 4
I1104 22:24:16.900302 140008086570880 run_classifier.py:929]   Batch size = 4
INFO:tensorflow:  Num steps = 63932
I1104 22:24:16.900385 140008086570880 run_classifier.py:930]   Num steps = 63932
WARNING:tensorflow:From 558-project/run_classifier.py:568: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

W1104 22:24:16.900620 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:568: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W1104 22:24:16.907325 140008086570880 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
W1104 22:24:16.907893 140008086570880 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From 558-project/run_classifier.py:604: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
W1104 22:24:16.947152 140008086570880 deprecation.py:323] From 558-project/run_classifier.py:604: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/data/python/ops/batching.py:276: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
W1104 22:24:16.947457 140008086570880 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/data/python/ops/batching.py:276: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

W1104 22:24:17.011754 140008086570880 module_wrapper.py:139] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

WARNING:tensorflow:From 558-project/run_classifier.py:584: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1104 22:24:17.122890 140008086570880 deprecation.py:323] From 558-project/run_classifier.py:584: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:tensorflow:Calling model_fn.
I1104 22:24:17.140167 140008086570880 estimator.py:1148] Calling model_fn.
INFO:tensorflow:Running train on CPU
I1104 22:24:17.140485 140008086570880 tpu_estimator.py:3124] Running train on CPU
INFO:tensorflow:*** Features ***
I1104 22:24:17.140878 140008086570880 run_classifier.py:681] *** Features ***
INFO:tensorflow:  name = input_ids, shape = (4, 64)
I1104 22:24:17.141041 140008086570880 run_classifier.py:683]   name = input_ids, shape = (4, 64)
INFO:tensorflow:  name = input_mask, shape = (4, 64)
I1104 22:24:17.141160 140008086570880 run_classifier.py:683]   name = input_mask, shape = (4, 64)
INFO:tensorflow:  name = is_real_example, shape = (4,)
I1104 22:24:17.141267 140008086570880 run_classifier.py:683]   name = is_real_example, shape = (4,)
INFO:tensorflow:  name = label_ids, shape = (4,)
I1104 22:24:17.141367 140008086570880 run_classifier.py:683]   name = label_ids, shape = (4,)
INFO:tensorflow:  name = segment_ids, shape = (4, 64)
I1104 22:24:17.141475 140008086570880 run_classifier.py:683]   name = segment_ids, shape = (4, 64)
WARNING:tensorflow:From /content/558-project/modeling.py:171: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1104 22:24:17.142382 140008086570880 module_wrapper.py:139] From /content/558-project/modeling.py:171: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /content/558-project/modeling.py:409: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

W1104 22:24:17.143902 140008086570880 module_wrapper.py:139] From /content/558-project/modeling.py:409: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:From /content/558-project/modeling.py:490: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.

W1104 22:24:17.164891 140008086570880 module_wrapper.py:139] From /content/558-project/modeling.py:490: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.

WARNING:tensorflow:From /content/558-project/modeling.py:358: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
W1104 22:24:17.207432 140008086570880 deprecation.py:506] From /content/558-project/modeling.py:358: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /content/558-project/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
W1104 22:24:17.220921 140008086570880 deprecation.py:323] From /content/558-project/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W1104 22:24:17.222519 140008086570880 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From 558-project/run_classifier.py:701: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

W1104 22:24:21.277334 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:701: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From 558-project/run_classifier.py:715: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

W1104 22:24:21.281372 140008086570880 module_wrapper.py:139] From 558-project/run_classifier.py:715: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

INFO:tensorflow:**** Trainable Variables ****
I1104 22:24:22.363112 140008086570880 run_classifier.py:717] **** Trainable Variables ****
INFO:tensorflow:  name = bert/embeddings/word_embeddings:0, shape = (28996, 1024), *INIT_FROM_CKPT*
I1104 22:24:22.363372 140008086570880 run_classifier.py:723]   name = bert/embeddings/word_embeddings:0, shape = (28996, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/token_type_embeddings:0, shape = (2, 1024), *INIT_FROM_CKPT*
I1104 22:24:22.363523 140008086570880 run_classifier.py:723]   name = bert/embeddings/token_type_embeddings:0, shape = (2, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/position_embeddings:0, shape = (512, 1024), 
I1104 22:24:22.437641 140008086570880 run_classifier.py:723]   name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
*INIT_FROM_CKPT*
I1104 22:24:22.473621 140008086570880 run_classifier.py:723]   name = bert/encoder/layer_23/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/pooler/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1104 22:24:22.473752 140008086570880 run_classifier.py:723]   name = bert/pooler/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/pooler/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1104 22:24:22.473891 140008086570880 run_classifier.py:723]   name = bert/pooler/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = output_weights:0, shape = (2, 1024)
I1104 22:24:22.474067 140008086570880 run_classifier.py:723]   name = output_weights:0, shape = (2, 1024)
INFO:tensorflow:  name = output_bias:0, shape = (2,)
I1104 22:24:22.474245 140008086570880 run_classifier.py:723]   name = output_bias:0, shape = (2,)
WARNING:tensorflow:From /content/558-project/optimization.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.            

                             " I deleted most of the Init_from_ckpt information here"

W1104 22:24:22.474492 140008086570880 module_wrapper.py:139] From /content/558-project/optimization.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.

WARNING:tensorflow:From /content/558-project/optimization.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W1104 22:24:22.475817 140008086570880 module_wrapper.py:139] From /content/558-project/optimization.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1104 22:24:22.769862 140008086570880 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
INFO:tensorflow:Done calling model_fn.
I1104 22:24:34.384642 140008086570880 estimator.py:1150] Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
I1104 22:24:34.386167 140008086570880 basic_session_run_hooks.py:541] Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
I1104 22:24:39.428250 140008086570880 monitored_session.py:240] Graph was finalized.
2019-11-04 22:24:39.448672: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz
2019-11-04 22:24:39.448930: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2245480 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2019-11-04 22:24:39.448966: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
INFO:tensorflow:Running local_init_op.
I1104 22:25:00.012823 140008086570880 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1104 22:25:00.500351 140008086570880 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into ./bert_output/model.ckpt.
I1104 22:25:11.934243 140008086570880 basic_session_run_hooks.py:606] Saving checkpoints for 0 into ./bert_output/model.ckpt.
^C

The .sh I use to run the code is:

!python run_classifier.py \
--task_name=sim \
--do_train=true \
--do_eval=true \
--do_predict=true \
--data_dir=./data/ \
--vocab_file=./wwm_cased_L-24_H-1024_A-16/vocab.txt \
--bert_config_file=./wwm_cased_L-24_H-1024_A-16/bert_config.json \
--init_checkpoint=./wwm_cased_L-24_H-1024_A-16/bert_model.ckpt \
--max_seq_length=64 \
--train_batch_size=4 \
--learning_rate=5e-5 \
--num_train_epochs=2.0 \
--output_dir=./bert_output/
Garimagupta85 commented 3 years ago

Hey,

I got the same output. Did you get the solution ?