Closed kent252 closed 1 year ago
I ran following evaluate command to evaluate the public B/32 checkpoint on LVIS but it just killed. Even when I try on Colab pro or PC. Any help is appreciated.
I1215 11:27:14.304406 139930883561344 pipeline.py:185] Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. W1215 11:27:33.767053 139930883561344 pipeline_options.py:351] Discarding unparseable args: ['--alsologtostderr=true', '--platform=gpu', '--config=scenic/projects/owl_vit/configs/clip_b32.py', '--checkpoint_path=gs://scenic-bucket/owl_vit/checkpoints/clip_vit_b32_b0203fc', '--annotations_path=/content/scenic/lvis_v1_val.json', '--tfds_data_dir=/content/scenic/val2017', '--output_dir=/content/scenic/evaluator'] I1215 11:27:33.767558 139930883561344 environments.py:376] Default Python SDK image for environment is apache/beam_python3.8_sdk:2.43.0 tcmalloc: large alloc 1434370048 bytes == 0x7f3d47a56000 @ 0x7f4432586615 0x5d6f4c 0x51edd1 0x51ef5b 0x5aac95 0x4990ca 0x55cd91 0x5d8941 0x4997a2 0x4fd8b5 0x4990ca 0x5d8868 0x4997a2 0x5d8868 0x4990ca 0x4fd8b5 0x4990ca 0x5d8868 0x4997a2 0x5d8868 0x4990ca 0x4fd8b5 0x4990ca 0x5d8868 0x4997a2 0x5d8868 0x4997c7 0x5d8868 0x4990ca 0x5d8868 0x4990ca I1215 11:33:47.870071 139930883561344 translations.py:714] ==================== <function annotate_downstream_side_inputs at 0x7f3df3f1b700> ==================== I1215 11:33:48.162448 139930883561344 translations.py:714] ==================== <function fix_side_input_pcoll_coders at 0x7f3df3f1b820> ==================== I1215 11:33:48.453671 139930883561344 translations.py:714] ==================== <function pack_combiners at 0x7f3df3f1bd30> ==================== I1215 11:33:48.457019 139930883561344 translations.py:714] ==================== <function lift_combiners at 0x7f3df3f1bdc0> ==================== I1215 11:33:48.460954 139930883561344 translations.py:714] ==================== <function expand_sdf at 0x7f3df3f1bf70> ==================== I1215 11:33:48.752287 139930883561344 translations.py:714] ==================== <function expand_gbk at 0x7f3df3f1d040> ==================== I1215 11:33:48.755760 139930883561344 translations.py:714] ==================== <function sink_flattens at 0x7f3df3f1d160> ==================== I1215 11:33:48.757105 139930883561344 translations.py:714] ==================== <function greedily_fuse at 0x7f3df3f1d1f0> ==================== I1215 11:33:49.058572 139930883561344 translations.py:714] ==================== <function read_to_impulse at 0x7f3df3f1d280> ==================== I1215 11:33:49.059184 139930883561344 translations.py:714] ==================== <function impulse_to_input at 0x7f3df3f1d310> ==================== I1215 11:33:49.059814 139930883561344 translations.py:714] ==================== <function sort_stages at 0x7f3df3f1d550> ==================== I1215 11:33:49.061884 139930883561344 translations.py:714] ==================== <function add_impulse_to_dangling_transforms at 0x7f3df3f1d670> ==================== I1215 11:33:49.062419 139930883561344 translations.py:714] ==================== <function setup_timer_mapping at 0x7f3df3f1d4c0> ==================== I1215 11:33:49.352953 139930883561344 translations.py:714] ==================== <function populate_data_channel_coders at 0x7f3df3f1d5e0> ==================== I1215 11:33:54.947741 139930883561344 statecache.py:234] Creating state cache with size 104857600 I1215 11:33:54.948321 139930883561344 worker_handlers.py:903] Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f3dc4807a60> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'') Killed
@kent252 I am also facing the same issue. Did you manage to solve this?
I ran following evaluate command to evaluate the public B/32 checkpoint on LVIS but it just killed. Even when I try on Colab pro or PC. Any help is appreciated.
I1215 11:27:14.304406 139930883561344 pipeline.py:185] Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. W1215 11:27:33.767053 139930883561344 pipeline_options.py:351] Discarding unparseable args: ['--alsologtostderr=true', '--platform=gpu', '--config=scenic/projects/owl_vit/configs/clip_b32.py', '--checkpoint_path=gs://scenic-bucket/owl_vit/checkpoints/clip_vit_b32_b0203fc', '--annotations_path=/content/scenic/lvis_v1_val.json', '--tfds_data_dir=/content/scenic/val2017', '--output_dir=/content/scenic/evaluator'] I1215 11:27:33.767558 139930883561344 environments.py:376] Default Python SDK image for environment is apache/beam_python3.8_sdk:2.43.0 tcmalloc: large alloc 1434370048 bytes == 0x7f3d47a56000 @ 0x7f4432586615 0x5d6f4c 0x51edd1 0x51ef5b 0x5aac95 0x4990ca 0x55cd91 0x5d8941 0x4997a2 0x4fd8b5 0x4990ca 0x5d8868 0x4997a2 0x5d8868 0x4990ca 0x4fd8b5 0x4990ca 0x5d8868 0x4997a2 0x5d8868 0x4990ca 0x4fd8b5 0x4990ca 0x5d8868 0x4997a2 0x5d8868 0x4997c7 0x5d8868 0x4990ca 0x5d8868 0x4990ca I1215 11:33:47.870071 139930883561344 translations.py:714] ==================== <function annotate_downstream_side_inputs at 0x7f3df3f1b700> ==================== I1215 11:33:48.162448 139930883561344 translations.py:714] ==================== <function fix_side_input_pcoll_coders at 0x7f3df3f1b820> ==================== I1215 11:33:48.453671 139930883561344 translations.py:714] ==================== <function pack_combiners at 0x7f3df3f1bd30> ==================== I1215 11:33:48.457019 139930883561344 translations.py:714] ==================== <function lift_combiners at 0x7f3df3f1bdc0> ==================== I1215 11:33:48.460954 139930883561344 translations.py:714] ==================== <function expand_sdf at 0x7f3df3f1bf70> ==================== I1215 11:33:48.752287 139930883561344 translations.py:714] ==================== <function expand_gbk at 0x7f3df3f1d040> ==================== I1215 11:33:48.755760 139930883561344 translations.py:714] ==================== <function sink_flattens at 0x7f3df3f1d160> ==================== I1215 11:33:48.757105 139930883561344 translations.py:714] ==================== <function greedily_fuse at 0x7f3df3f1d1f0> ==================== I1215 11:33:49.058572 139930883561344 translations.py:714] ==================== <function read_to_impulse at 0x7f3df3f1d280> ==================== I1215 11:33:49.059184 139930883561344 translations.py:714] ==================== <function impulse_to_input at 0x7f3df3f1d310> ==================== I1215 11:33:49.059814 139930883561344 translations.py:714] ==================== <function sort_stages at 0x7f3df3f1d550> ==================== I1215 11:33:49.061884 139930883561344 translations.py:714] ==================== <function add_impulse_to_dangling_transforms at 0x7f3df3f1d670> ==================== I1215 11:33:49.062419 139930883561344 translations.py:714] ==================== <function setup_timer_mapping at 0x7f3df3f1d4c0> ==================== I1215 11:33:49.352953 139930883561344 translations.py:714] ==================== <function populate_data_channel_coders at 0x7f3df3f1d5e0> ==================== I1215 11:33:54.947741 139930883561344 statecache.py:234] Creating state cache with size 104857600 I1215 11:33:54.948321 139930883561344 worker_handlers.py:903] Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f3dc4807a60> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'') Killed