Tensorflow Version Conflict

google-research / tensor2robot

Distributed machine learning infrastructure for large-scale robotics research

Apache License 2.0

537 stars 116 forks source link

absl-py==2.0.0 astor==0.8.1 cloudpickle==1.1.1 decorator==5.1.1 gast==0.2.2 gin-config==0.5.0 google-pasta==0.2.0 grpcio==1.59.2 gym==0.12.5 h5py==2.10.0 importlib-metadata==6.7.0 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.2 Markdown==3.4.4 MarkupSafe==2.1.3 numpy==1.18.5 opt-einsum==3.3.0 Pillow==5.3.0 protobuf==3.20.3 pybullet==2.5.0 pyglet==2.0.9 scipy==1.7.3 six==1.16.0 tensorboard==1.15.0 tensorflow==1.15.5 tensorflow-estimator==1.15.1 tensorflow-probability==0.8.0 tensorflow-serving-api==1.15.0 termcolor==2.3.0 tf-slim==1.1.0 typing_extensions==4.7.1 Werkzeug==2.2.3 wrapt==1.15.0 zipp==3.15.0

robotics@e03335ff5f81:~/app$ python -m tensor2robot.research.pose_env.pose_env_models_test pybullet build time: Nov 6 2023 17:50:41 Running tests under Python 3.7.17: /usr/local/bin/python [ RUN ] PoseEnvModelsTest.test_mc I1219 22:53:21.963839 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode train I1219 22:53:21.964352 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.964405 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.964439 140259491276608 train_eval.py:83] Preprocessor in label specification. I1219 22:53:21.964628 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.964658 140259491276608 train_eval.py:86] Preprocessor out feature specification. I1219 22:53:21.964856 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.964883 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.964909 140259491276608 train_eval.py:88] Preprocessor out label specification. I1219 22:53:21.965075 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.965101 140259491276608 train_eval.py:91] Model feature specification. I1219 22:53:21.965171 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.965197 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.965221 140259491276608 train_eval.py:93] Model label specification. I1219 22:53:21.965259 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.965282 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode infer I1219 22:53:21.965620 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.965649 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.965674 140259491276608 train_eval.py:83] Preprocessor in label specification. I1219 22:53:21.965846 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.965874 140259491276608 train_eval.py:86] Preprocessor out feature specification. I1219 22:53:21.966076 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.966102 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.966126 140259491276608 train_eval.py:88] Preprocessor out label specification. I1219 22:53:21.966290 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.966316 140259491276608 train_eval.py:91] Model feature specification. I1219 22:53:21.966381 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.966405 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.966430 140259491276608 train_eval.py:93] Model label specification. I1219 22:53:21.966466 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I1219 22:53:21.966503 140259491276608 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [ FAILED ] PoseEnvModelsTest.test_mc [ RUN ] PoseEnvModelsTest.test_regression /home/robotics/.local/lib/python3.7/site-packages/gin/config.py:1582: DeprecationWarning: RegressionModel is deprecated. Subclass AbstractT2RModel instead return fn(*new_args, **new_kwargs) I1219 22:53:21.967823 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode train I1219 22:53:21.968000 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.968040 140259491276608 train_eval.py:83] Preprocessor in label specification. I1219 22:53:21.968181 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.968210 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.968235 140259491276608 train_eval.py:86] Preprocessor out feature specification. I1219 22:53:21.968365 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.968395 140259491276608 train_eval.py:88] Preprocessor out label specification. I1219 22:53:21.968533 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.968561 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.968584 140259491276608 train_eval.py:91] Model feature specification. I1219 22:53:21.968624 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.968646 140259491276608 train_eval.py:93] Model label specification. I1219 22:53:21.968694 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.968718 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.968744 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode infer I1219 22:53:21.968879 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.968906 140259491276608 train_eval.py:83] Preprocessor in label specification. I1219 22:53:21.969034 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.969062 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.969088 140259491276608 train_eval.py:86] Preprocessor out feature specification. I1219 22:53:21.969214 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.969244 140259491276608 train_eval.py:88] Preprocessor out label specification. I1219 22:53:21.969391 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.969415 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.969438 140259491276608 train_eval.py:91] Model feature specification. I1219 22:53:21.969475 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.969500 140259491276608 train_eval.py:93] Model label specification. I1219 22:53:21.969545 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.969567 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I1219 22:53:21.969590 140259491276608 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [ FAILED ] PoseEnvModelsTest.test_regression [ RUN ] PoseEnvModelsTest.test_regression_maml I1219 22:53:21.969906 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode train I1219 22:53:21.970395 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.970435 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.970461 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.970485 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.970516 140259491276608 train_eval.py:83] Preprocessor in label specification. I1219 22:53:21.970721 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.970749 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.970772 140259491276608 train_eval.py:86] Preprocessor out feature specification. I1219 22:53:21.971091 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.971124 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971148 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971168 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.971189 140259491276608 train_eval.py:88] Preprocessor out label specification. I1219 22:53:21.971381 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971409 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971432 140259491276608 train_eval.py:91] Model feature specification. I1219 22:53:21.971624 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.971651 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971672 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971692 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.971713 140259491276608 train_eval.py:93] Model label specification. I1219 22:53:21.971803 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971828 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.971851 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode infer I1219 22:53:21.972169 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.972199 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.972221 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.972241 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.972261 140259491276608 train_eval.py:83] Preprocessor in label specification. I1219 22:53:21.972444 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.972471 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.972495 140259491276608 train_eval.py:86] Preprocessor out feature specification. I1219 22:53:21.972911 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.972942 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.972967 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.972989 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.973011 140259491276608 train_eval.py:88] Preprocessor out label specification. I1219 22:53:21.973208 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.973237 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.973261 140259491276608 train_eval.py:91] Model feature specification. I1219 22:53:21.973460 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.973491 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.973516 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.973538 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:53:21.973562 140259491276608 train_eval.py:93] Model label specification. I1219 22:53:21.973659 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:53:21.973688 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I1219 22:53:21.973716 140259491276608 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [ FAILED ] PoseEnvModelsTest.test_regression_maml [ RUN ] PoseEnvModelsTest.test_regression_maml_policy_interface WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.__call__` method instead. W1219 22:53:22.019067 140259491276608 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.__call__` method instead. /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/variable_scope.py:452: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working shape is not None and isinstance(shape, collections_lib.Sequence) and WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers. W1219 22:53:22.023470 140259491276608 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers. WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead W1219 22:53:22.093517 140259491276608 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where W1219 22:53:22.291660 140259491276608 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where 2023-12-19 22:53:22.698627: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2023-12-19 22:53:22.708522: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2995200000 Hz 2023-12-19 22:53:22.708725: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x562fd61f7050 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2023-12-19 22:53:22.708745: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version I1219 22:53:22.721928 140259491276608 checkpoint_predictor.py:131] Initializing model with random weights [ FAILED ] PoseEnvModelsTest.test_regression_maml_policy_interface [ RUN ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin') I1219 22:53:22.831758 140259491276608 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg_maml.gin [ FAILED ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin') [ RUN ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin') I1219 22:53:22.832791 140259491276608 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg.gin [ FAILED ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin') ====================================================================== ERROR: test_mc (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_mc ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 75, in test_mc create_exporters_fn=None) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper utils.augment_exception_message_and_reraise(e, err_str) File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise raise proxy.with_traceback(exception.__traceback__) from None File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper return fn(*new_args, **new_kwargs) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model mode=tf_estimator.ModeKeys.TRAIN, File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression)) NameError: name 'guzzler_use_compression' is not defined In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f9039627c20>) ====================================================================== ERROR: test_regression (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_regression ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 82, in test_regression create_exporters_fn=None) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper utils.augment_exception_message_and_reraise(e, err_str) File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise raise proxy.with_traceback(exception.__traceback__) from None File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper return fn(*new_args, **new_kwargs) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model mode=tf_estimator.ModeKeys.TRAIN, File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression)) NameError: name 'guzzler_use_compression' is not defined In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f9039627c20>) ====================================================================== ERROR: test_regression_maml (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_regression_maml ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 91, in test_regression_maml create_exporters_fn=None) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper utils.augment_exception_message_and_reraise(e, err_str) File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise raise proxy.with_traceback(exception.__traceback__) from None File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper return fn(*new_args, **new_kwargs) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model mode=tf_estimator.ModeKeys.TRAIN, File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression)) NameError: name 'guzzler_use_compression' is not defined In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f9039627c20>) ====================================================================== ERROR: test_regression_maml_policy_interface (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_regression_maml_policy_interface ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 118, in test_regression_maml_policy_interface self._test_policy_interface(policy, restore=False) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 103, in _test_policy_interface action = policy.SelectAction(obs, None, 0) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/meta_learning/meta_policies.py", line 122, in SelectAction action = self._predictor.predict(np_features)['inference_output'] File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/predictors/checkpoint_predictor.py", line 117, in predict tensorspec_utils.map_feed_dict(self._features, features)) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 956, in map_feed_dict maybe_ignore_batch(spec_numpy, ignore_batch)) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1206, in assert_required flat_expected_spec, flat_actual_spec, ignore_batch) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1165, in assert_equal flattened_actual_tensors_or_spec) File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in map_structure structure[0], [func(*x) for x in entries], File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in <listcomp> structure[0], [func(*x) for x in entries], File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1126, in assert_equal_spec_or_tensor expected_spec.dtype, actual_spec.dtype, expected_spec, actual_spec)) ValueError: TensorSpec.dtype <dtype: 'uint8'> does not match TensorSpec.dtype <dtype: 'int64'> in specs expected: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_ep0/state/image', is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None) actual: ExtendedTensorSpec(shape=(1, 64, 64, 3), dtype=tf.int64, name=None, is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None) ====================================================================== ERROR: test_train_eval_gin0 ('run_train_reg_maml.gin') (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin') test_train_eval_gin('run_train_reg_maml.gin') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test return test_method(self, *testcase_params) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin eval_steps=EVAL_STEPS) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin gin.parse_config_file(full_gin_path, print_includes_and_imports=True) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file if existence_check(config_file_with_prefix): File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists path = _parse_config_path(config_path) File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path path = os.path.join(os.path.dirname(file_sys_path), filename) File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType ====================================================================== ERROR: test_train_eval_gin1 ('run_train_reg.gin') (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin') test_train_eval_gin('run_train_reg.gin') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test return test_method(self, *testcase_params) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin eval_steps=EVAL_STEPS) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin gin.parse_config_file(full_gin_path, print_includes_and_imports=True) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file if existence_check(config_file_with_prefix): File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists path = _parse_config_path(config_path) File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path path = os.path.join(os.path.dirname(file_sys_path), filename) File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType ---------------------------------------------------------------------- Ran 6 tests in 0.869s FAILED (errors=6) robotics@e03335ff5f81:~/app$ robotics@e03335ff5f81:~/app$ python -m tensor2robot.research.pose_env.pose_env_models_test pybullet build time: Nov 6 2023 17:50:41 Running tests under Python 3.7.17: /usr/local/bin/python [ RUN ] PoseEnvModelsTest.test_mc I1219 22:54:13.255325 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode train I1219 22:54:13.255872 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.255935 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.255967 140043058079552 train_eval.py:83] Preprocessor in label specification. I1219 22:54:13.256154 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.256184 140043058079552 train_eval.py:86] Preprocessor out feature specification. I1219 22:54:13.256396 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.256423 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.256448 140043058079552 train_eval.py:88] Preprocessor out label specification. I1219 22:54:13.256624 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.256651 140043058079552 train_eval.py:91] Model feature specification. I1219 22:54:13.256721 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.256746 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.256771 140043058079552 train_eval.py:93] Model label specification. I1219 22:54:13.256809 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.256833 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode infer I1219 22:54:13.257177 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.257204 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.257230 140043058079552 train_eval.py:83] Preprocessor in label specification. I1219 22:54:13.257411 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.257437 140043058079552 train_eval.py:86] Preprocessor out feature specification. I1219 22:54:13.257647 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.257673 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.257698 140043058079552 train_eval.py:88] Preprocessor out label specification. I1219 22:54:13.257999 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.258025 140043058079552 train_eval.py:91] Model feature specification. I1219 22:54:13.258110 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.258135 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.258159 140043058079552 train_eval.py:93] Model label specification. I1219 22:54:13.258204 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I1219 22:54:13.258255 140043058079552 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [ FAILED ] PoseEnvModelsTest.test_mc [ RUN ] PoseEnvModelsTest.test_regression /home/robotics/.local/lib/python3.7/site-packages/gin/config.py:1582: DeprecationWarning: RegressionModel is deprecated. Subclass AbstractT2RModel instead return fn(*new_args, **new_kwargs) I1219 22:54:13.259181 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode train I1219 22:54:13.259412 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.259459 140043058079552 train_eval.py:83] Preprocessor in label specification. I1219 22:54:13.259619 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.259653 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.259681 140043058079552 train_eval.py:86] Preprocessor out feature specification. I1219 22:54:13.259829 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.259861 140043058079552 train_eval.py:88] Preprocessor out label specification. I1219 22:54:13.260016 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.260049 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.260080 140043058079552 train_eval.py:91] Model feature specification. I1219 22:54:13.260133 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.260167 140043058079552 train_eval.py:93] Model label specification. I1219 22:54:13.260229 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.260260 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.260290 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode infer I1219 22:54:13.260476 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.260512 140043058079552 train_eval.py:83] Preprocessor in label specification. I1219 22:54:13.260689 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.260721 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.260751 140043058079552 train_eval.py:86] Preprocessor out feature specification. I1219 22:54:13.260913 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.260948 140043058079552 train_eval.py:88] Preprocessor out label specification. I1219 22:54:13.261127 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.261158 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.261188 140043058079552 train_eval.py:91] Model feature specification. I1219 22:54:13.261240 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.261273 140043058079552 train_eval.py:93] Model label specification. I1219 22:54:13.261333 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.261364 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I1219 22:54:13.261396 140043058079552 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [ FAILED ] PoseEnvModelsTest.test_regression [ RUN ] PoseEnvModelsTest.test_regression_maml I1219 22:54:13.261789 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode train I1219 22:54:13.262426 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.262538 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.262617 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.262667 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.262716 140043058079552 train_eval.py:83] Preprocessor in label specification. I1219 22:54:13.263183 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.263238 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.263284 140043058079552 train_eval.py:86] Preprocessor out feature specification. I1219 22:54:13.263968 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.264023 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.264066 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.264106 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.264147 140043058079552 train_eval.py:88] Preprocessor out label specification. I1219 22:54:13.264546 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.264599 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.264643 140043058079552 train_eval.py:91] Model feature specification. I1219 22:54:13.265053 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.265121 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.265157 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.265185 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.265213 140043058079552 train_eval.py:93] Model label specification. I1219 22:54:13.265345 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.265381 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.265410 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode infer I1219 22:54:13.265819 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.265857 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.265944 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.265970 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.265997 140043058079552 train_eval.py:83] Preprocessor in label specification. I1219 22:54:13.266239 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.266274 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.266303 140043058079552 train_eval.py:86] Preprocessor out feature specification. I1219 22:54:13.266672 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.266707 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.266734 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.266758 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.266783 140043058079552 train_eval.py:88] Preprocessor out label specification. I1219 22:54:13.267011 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.267043 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.267071 140043058079552 train_eval.py:91] Model feature specification. I1219 22:54:13.267297 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.267330 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.267358 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.267383 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None) I1219 22:54:13.267408 140043058079552 train_eval.py:93] Model label specification. I1219 22:54:13.267512 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) I1219 22:54:13.267543 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None) INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I1219 22:54:13.267574 140043058079552 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [ FAILED ] PoseEnvModelsTest.test_regression_maml [ RUN ] PoseEnvModelsTest.test_regression_maml_policy_interface WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.__call__` method instead. W1219 22:54:13.314008 140043058079552 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.__call__` method instead. /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/variable_scope.py:452: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working shape is not None and isinstance(shape, collections_lib.Sequence) and WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers. W1219 22:54:13.317741 140043058079552 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers. WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead W1219 22:54:13.385801 140043058079552 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where W1219 22:54:13.570754 140043058079552 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where 2023-12-19 22:54:13.985586: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2023-12-19 22:54:13.987903: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2995200000 Hz 2023-12-19 22:54:13.988047: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ccf81c35b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2023-12-19 22:54:13.988061: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version I1219 22:54:14.002703 140043058079552 checkpoint_predictor.py:131] Initializing model with random weights [ FAILED ] PoseEnvModelsTest.test_regression_maml_policy_interface [ RUN ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin') I1219 22:54:14.098639 140043058079552 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg_maml.gin [ FAILED ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin') [ RUN ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin') I1219 22:54:14.099249 140043058079552 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg.gin [ FAILED ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin') ====================================================================== ERROR: test_mc (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_mc ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 75, in test_mc create_exporters_fn=None) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper utils.augment_exception_message_and_reraise(e, err_str) File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise raise proxy.with_traceback(exception.__traceback__) from None File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper return fn(*new_args, **new_kwargs) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model mode=tf_estimator.ModeKeys.TRAIN, File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression)) NameError: name 'guzzler_use_compression' is not defined In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f5dd4f1ec20>) ====================================================================== ERROR: test_regression (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_regression ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 82, in test_regression create_exporters_fn=None) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper utils.augment_exception_message_and_reraise(e, err_str) File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise raise proxy.with_traceback(exception.__traceback__) from None File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper return fn(*new_args, **new_kwargs) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model mode=tf_estimator.ModeKeys.TRAIN, File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression)) NameError: name 'guzzler_use_compression' is not defined In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f5dd4f1ec20>) ====================================================================== ERROR: test_regression_maml (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_regression_maml ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 91, in test_regression_maml create_exporters_fn=None) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper utils.augment_exception_message_and_reraise(e, err_str) File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise raise proxy.with_traceback(exception.__traceback__) from None File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper return fn(*new_args, **new_kwargs) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model mode=tf_estimator.ModeKeys.TRAIN, File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression)) NameError: name 'guzzler_use_compression' is not defined In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f5dd4f1ec20>) ====================================================================== ERROR: test_regression_maml_policy_interface (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_regression_maml_policy_interface ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 118, in test_regression_maml_policy_interface self._test_policy_interface(policy, restore=False) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 103, in _test_policy_interface action = policy.SelectAction(obs, None, 0) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/meta_learning/meta_policies.py", line 122, in SelectAction action = self._predictor.predict(np_features)['inference_output'] File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/predictors/checkpoint_predictor.py", line 117, in predict tensorspec_utils.map_feed_dict(self._features, features)) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 956, in map_feed_dict maybe_ignore_batch(spec_numpy, ignore_batch)) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1206, in assert_required flat_expected_spec, flat_actual_spec, ignore_batch) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1165, in assert_equal flattened_actual_tensors_or_spec) File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in map_structure structure[0], [func(*x) for x in entries], File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in <listcomp> structure[0], [func(*x) for x in entries], File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1126, in assert_equal_spec_or_tensor expected_spec.dtype, actual_spec.dtype, expected_spec, actual_spec)) ValueError: TensorSpec.dtype <dtype: 'uint8'> does not match TensorSpec.dtype <dtype: 'int64'> in specs expected: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_ep0/state/image', is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None) actual: ExtendedTensorSpec(shape=(1, 64, 64, 3), dtype=tf.int64, name=None, is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None) ====================================================================== ERROR: test_train_eval_gin0 ('run_train_reg_maml.gin') (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin') test_train_eval_gin('run_train_reg_maml.gin') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test return test_method(self, *testcase_params) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin eval_steps=EVAL_STEPS) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin gin.parse_config_file(full_gin_path, print_includes_and_imports=True) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file if existence_check(config_file_with_prefix): File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists path = _parse_config_path(config_path) File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path path = os.path.join(os.path.dirname(file_sys_path), filename) File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType ====================================================================== ERROR: test_train_eval_gin1 ('run_train_reg.gin') (__main__.PoseEnvModelsTest) PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin') test_train_eval_gin('run_train_reg.gin') ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test return test_method(self, *testcase_params) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin eval_steps=EVAL_STEPS) File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin gin.parse_config_file(full_gin_path, print_includes_and_imports=True) File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file if existence_check(config_file_with_prefix): File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists path = _parse_config_path(config_path) File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path path = os.path.join(os.path.dirname(file_sys_path), filename) File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType ---------------------------------------------------------------------- Ran 6 tests in 0.844s FAILED (errors=6)

Hey @colorcatliu, this is the full logs. I haven't had time to look at this

I made a repo here so that you can try to reproduce that yourself: https://github.com/ngochieu642-org/run-robotics The container image install tensor2robot and robotics_transformer. Once the container is running, you can exec into it. This is the command I used

robotics@e03335ff5f81:~/app$ python -m tensor2robot.research.pose_env.pose_env_models_test

pybullet build time: Nov  6 2023 17:50:41
Running tests under Python 3.7.17: /usr/local/bin/python
[ RUN      ] PoseEnvModelsTest.test_mc
I1219 22:53:21.963839 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode train
I1219 22:53:21.964352 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.964405 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.964439 140259491276608 train_eval.py:83] Preprocessor in label specification.
I1219 22:53:21.964628 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.964658 140259491276608 train_eval.py:86] Preprocessor out feature specification.
I1219 22:53:21.964856 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.964883 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.964909 140259491276608 train_eval.py:88] Preprocessor out label specification.
I1219 22:53:21.965075 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.965101 140259491276608 train_eval.py:91] Model feature specification.
I1219 22:53:21.965171 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.965197 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.965221 140259491276608 train_eval.py:93] Model label specification.
I1219 22:53:21.965259 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.965282 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode infer
I1219 22:53:21.965620 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.965649 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.965674 140259491276608 train_eval.py:83] Preprocessor in label specification.
I1219 22:53:21.965846 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.965874 140259491276608 train_eval.py:86] Preprocessor out feature specification.
I1219 22:53:21.966076 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.966102 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.966126 140259491276608 train_eval.py:88] Preprocessor out label specification.
I1219 22:53:21.966290 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.966316 140259491276608 train_eval.py:91] Model feature specification.
I1219 22:53:21.966381 140259491276608 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.966405 140259491276608 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.966430 140259491276608 train_eval.py:93] Model label specification.
I1219 22:53:21.966466 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I1219 22:53:21.966503 140259491276608 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[  FAILED  ] PoseEnvModelsTest.test_mc
[ RUN      ] PoseEnvModelsTest.test_regression
/home/robotics/.local/lib/python3.7/site-packages/gin/config.py:1582: DeprecationWarning: RegressionModel is deprecated. Subclass AbstractT2RModel instead
  return fn(*new_args, **new_kwargs)
I1219 22:53:21.967823 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode train
I1219 22:53:21.968000 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.968040 140259491276608 train_eval.py:83] Preprocessor in label specification.
I1219 22:53:21.968181 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.968210 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.968235 140259491276608 train_eval.py:86] Preprocessor out feature specification.
I1219 22:53:21.968365 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.968395 140259491276608 train_eval.py:88] Preprocessor out label specification.
I1219 22:53:21.968533 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.968561 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.968584 140259491276608 train_eval.py:91] Model feature specification.
I1219 22:53:21.968624 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.968646 140259491276608 train_eval.py:93] Model label specification.
I1219 22:53:21.968694 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.968718 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.968744 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode infer
I1219 22:53:21.968879 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.968906 140259491276608 train_eval.py:83] Preprocessor in label specification.
I1219 22:53:21.969034 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.969062 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.969088 140259491276608 train_eval.py:86] Preprocessor out feature specification.
I1219 22:53:21.969214 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.969244 140259491276608 train_eval.py:88] Preprocessor out label specification.
I1219 22:53:21.969391 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.969415 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.969438 140259491276608 train_eval.py:91] Model feature specification.
I1219 22:53:21.969475 140259491276608 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.969500 140259491276608 train_eval.py:93] Model label specification.
I1219 22:53:21.969545 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.969567 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I1219 22:53:21.969590 140259491276608 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[  FAILED  ] PoseEnvModelsTest.test_regression
[ RUN      ] PoseEnvModelsTest.test_regression_maml
I1219 22:53:21.969906 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode train
I1219 22:53:21.970395 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.970435 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.970461 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.970485 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.970516 140259491276608 train_eval.py:83] Preprocessor in label specification.
I1219 22:53:21.970721 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.970749 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.970772 140259491276608 train_eval.py:86] Preprocessor out feature specification.
I1219 22:53:21.971091 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.971124 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971148 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971168 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.971189 140259491276608 train_eval.py:88] Preprocessor out label specification.
I1219 22:53:21.971381 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971409 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971432 140259491276608 train_eval.py:91] Model feature specification.
I1219 22:53:21.971624 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.971651 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971672 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971692 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.971713 140259491276608 train_eval.py:93] Model label specification.
I1219 22:53:21.971803 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971828 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.971851 140259491276608 train_eval.py:81] Preprocessor in feature specification for mode infer
I1219 22:53:21.972169 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.972199 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.972221 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.972241 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.972261 140259491276608 train_eval.py:83] Preprocessor in label specification.
I1219 22:53:21.972444 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.972471 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.972495 140259491276608 train_eval.py:86] Preprocessor out feature specification.
I1219 22:53:21.972911 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.972942 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.972967 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.972989 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.973011 140259491276608 train_eval.py:88] Preprocessor out label specification.
I1219 22:53:21.973208 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.973237 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.973261 140259491276608 train_eval.py:91] Model feature specification.
I1219 22:53:21.973460 140259491276608 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.973491 140259491276608 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.973516 140259491276608 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.973538 140259491276608 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:53:21.973562 140259491276608 train_eval.py:93] Model label specification.
I1219 22:53:21.973659 140259491276608 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:53:21.973688 140259491276608 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I1219 22:53:21.973716 140259491276608 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[  FAILED  ] PoseEnvModelsTest.test_regression_maml
[ RUN      ] PoseEnvModelsTest.test_regression_maml_policy_interface
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W1219 22:53:22.019067 140259491276608 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/variable_scope.py:452: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  shape is not None and isinstance(shape, collections_lib.Sequence) and
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W1219 22:53:22.023470 140259491276608 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
W1219 22:53:22.093517 140259491276608 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1219 22:53:22.291660 140259491276608 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
2023-12-19 22:53:22.698627: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-12-19 22:53:22.708522: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2995200000 Hz
2023-12-19 22:53:22.708725: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x562fd61f7050 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2023-12-19 22:53:22.708745: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
I1219 22:53:22.721928 140259491276608 checkpoint_predictor.py:131] Initializing model with random weights
[  FAILED  ] PoseEnvModelsTest.test_regression_maml_policy_interface
[ RUN      ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin')
I1219 22:53:22.831758 140259491276608 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg_maml.gin
[  FAILED  ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin')
[ RUN      ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin')
I1219 22:53:22.832791 140259491276608 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg.gin
[  FAILED  ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin')
======================================================================
ERROR: test_mc (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_mc
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 75, in test_mc
    create_exporters_fn=None)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model
    mode=tf_estimator.ModeKeys.TRAIN,
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information
    tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression))
NameError: name 'guzzler_use_compression' is not defined
  In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f9039627c20>)

======================================================================
ERROR: test_regression (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_regression
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 82, in test_regression
    create_exporters_fn=None)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model
    mode=tf_estimator.ModeKeys.TRAIN,
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information
    tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression))
NameError: name 'guzzler_use_compression' is not defined
  In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f9039627c20>)

======================================================================
ERROR: test_regression_maml (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_regression_maml
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 91, in test_regression_maml
    create_exporters_fn=None)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model
    mode=tf_estimator.ModeKeys.TRAIN,
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information
    tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression))
NameError: name 'guzzler_use_compression' is not defined
  In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f9039627c20>)

======================================================================
ERROR: test_regression_maml_policy_interface (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_regression_maml_policy_interface
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 118, in test_regression_maml_policy_interface
    self._test_policy_interface(policy, restore=False)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 103, in _test_policy_interface
    action = policy.SelectAction(obs, None, 0)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/meta_learning/meta_policies.py", line 122, in SelectAction
    action = self._predictor.predict(np_features)['inference_output']
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/predictors/checkpoint_predictor.py", line 117, in predict
    tensorspec_utils.map_feed_dict(self._features, features))
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 956, in map_feed_dict
    maybe_ignore_batch(spec_numpy, ignore_batch))
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1206, in assert_required
    flat_expected_spec, flat_actual_spec, ignore_batch)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1165, in assert_equal
    flattened_actual_tensors_or_spec)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1126, in assert_equal_spec_or_tensor
    expected_spec.dtype, actual_spec.dtype, expected_spec, actual_spec))
ValueError: TensorSpec.dtype <dtype: 'uint8'> does not match TensorSpec.dtype <dtype: 'int64'> in specs
 expected: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_ep0/state/image', is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None)
 actual: ExtendedTensorSpec(shape=(1, 64, 64, 3), dtype=tf.int64, name=None, is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None)

======================================================================
ERROR: test_train_eval_gin0 ('run_train_reg_maml.gin') (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin')
test_train_eval_gin('run_train_reg_maml.gin')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test
    return test_method(self, *testcase_params)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin
    eval_steps=EVAL_STEPS)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin
    gin.parse_config_file(full_gin_path, print_includes_and_imports=True)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file
    if existence_check(config_file_with_prefix):
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists
    path = _parse_config_path(config_path)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path
    path = os.path.join(os.path.dirname(file_sys_path), filename)
  File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

======================================================================
ERROR: test_train_eval_gin1 ('run_train_reg.gin') (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin')
test_train_eval_gin('run_train_reg.gin')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test
    return test_method(self, *testcase_params)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin
    eval_steps=EVAL_STEPS)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin
    gin.parse_config_file(full_gin_path, print_includes_and_imports=True)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file
    if existence_check(config_file_with_prefix):
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists
    path = _parse_config_path(config_path)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path
    path = os.path.join(os.path.dirname(file_sys_path), filename)
  File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

----------------------------------------------------------------------
Ran 6 tests in 0.869s

FAILED (errors=6)
robotics@e03335ff5f81:~/app$ 
robotics@e03335ff5f81:~/app$ python -m tensor2robot.research.pose_env.pose_env_models_test
pybullet build time: Nov  6 2023 17:50:41
Running tests under Python 3.7.17: /usr/local/bin/python
[ RUN      ] PoseEnvModelsTest.test_mc
I1219 22:54:13.255325 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode train
I1219 22:54:13.255872 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.255935 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.255967 140043058079552 train_eval.py:83] Preprocessor in label specification.
I1219 22:54:13.256154 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.256184 140043058079552 train_eval.py:86] Preprocessor out feature specification.
I1219 22:54:13.256396 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.256423 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.256448 140043058079552 train_eval.py:88] Preprocessor out label specification.
I1219 22:54:13.256624 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.256651 140043058079552 train_eval.py:91] Model feature specification.
I1219 22:54:13.256721 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.256746 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.256771 140043058079552 train_eval.py:93] Model label specification.
I1219 22:54:13.256809 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.256833 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode infer
I1219 22:54:13.257177 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.257204 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.257230 140043058079552 train_eval.py:83] Preprocessor in label specification.
I1219 22:54:13.257411 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.257437 140043058079552 train_eval.py:86] Preprocessor out feature specification.
I1219 22:54:13.257647 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.257673 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.257698 140043058079552 train_eval.py:88] Preprocessor out label specification.
I1219 22:54:13.257999 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.258025 140043058079552 train_eval.py:91] Model feature specification.
I1219 22:54:13.258110 140043058079552 train_eval.py:70] action/pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.258135 140043058079552 train_eval.py:70] state/image: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.258159 140043058079552 train_eval.py:93] Model label specification.
I1219 22:54:13.258204 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I1219 22:54:13.258255 140043058079552 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[  FAILED  ] PoseEnvModelsTest.test_mc
[ RUN      ] PoseEnvModelsTest.test_regression
/home/robotics/.local/lib/python3.7/site-packages/gin/config.py:1582: DeprecationWarning: RegressionModel is deprecated. Subclass AbstractT2RModel instead
  return fn(*new_args, **new_kwargs)
I1219 22:54:13.259181 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode train
I1219 22:54:13.259412 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.259459 140043058079552 train_eval.py:83] Preprocessor in label specification.
I1219 22:54:13.259619 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.259653 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.259681 140043058079552 train_eval.py:86] Preprocessor out feature specification.
I1219 22:54:13.259829 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.259861 140043058079552 train_eval.py:88] Preprocessor out label specification.
I1219 22:54:13.260016 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.260049 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.260080 140043058079552 train_eval.py:91] Model feature specification.
I1219 22:54:13.260133 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.260167 140043058079552 train_eval.py:93] Model label specification.
I1219 22:54:13.260229 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.260260 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.260290 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode infer
I1219 22:54:13.260476 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.uint8, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.260512 140043058079552 train_eval.py:83] Preprocessor in label specification.
I1219 22:54:13.260689 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.260721 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.260751 140043058079552 train_eval.py:86] Preprocessor out feature specification.
I1219 22:54:13.260913 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.260948 140043058079552 train_eval.py:88] Preprocessor out label specification.
I1219 22:54:13.261127 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.261158 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.261188 140043058079552 train_eval.py:91] Model feature specification.
I1219 22:54:13.261240 140043058079552 train_eval.py:70] state: ExtendedTensorSpec(shape=(64, 64, 3), dtype=tf.float32, name='state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.261273 140043058079552 train_eval.py:93] Model label specification.
I1219 22:54:13.261333 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(1,), dtype=tf.float32, name='reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.261364 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(2,), dtype=tf.float32, name='target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I1219 22:54:13.261396 140043058079552 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[  FAILED  ] PoseEnvModelsTest.test_regression
[ RUN      ] PoseEnvModelsTest.test_regression_maml
I1219 22:54:13.261789 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode train
I1219 22:54:13.262426 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.262538 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.262617 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.262667 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.262716 140043058079552 train_eval.py:83] Preprocessor in label specification.
I1219 22:54:13.263183 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.263238 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.263284 140043058079552 train_eval.py:86] Preprocessor out feature specification.
I1219 22:54:13.263968 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.264023 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.264066 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.264106 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.264147 140043058079552 train_eval.py:88] Preprocessor out label specification.
I1219 22:54:13.264546 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.264599 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.264643 140043058079552 train_eval.py:91] Model feature specification.
I1219 22:54:13.265053 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.265121 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.265157 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.265185 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.265213 140043058079552 train_eval.py:93] Model label specification.
I1219 22:54:13.265345 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.265381 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.265410 140043058079552 train_eval.py:81] Preprocessor in feature specification for mode infer
I1219 22:54:13.265819 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.265857 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.265944 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.265970 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.265997 140043058079552 train_eval.py:83] Preprocessor in label specification.
I1219 22:54:13.266239 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.266274 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.266303 140043058079552 train_eval.py:86] Preprocessor out feature specification.
I1219 22:54:13.266672 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.266707 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.266734 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.266758 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.266783 140043058079552 train_eval.py:88] Preprocessor out label specification.
I1219 22:54:13.267011 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.267043 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.267071 140043058079552 train_eval.py:91] Model feature specification.
I1219 22:54:13.267297 140043058079552 train_eval.py:70] condition/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='condition_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.267330 140043058079552 train_eval.py:70] condition/labels/reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='condition_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.267358 140043058079552 train_eval.py:70] condition/labels/target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='condition_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.267383 140043058079552 train_eval.py:70] inference/features/state: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.float32, name='inference_features/state/image', is_optional=False, is_sequence=False, is_extracted=False, data_format='jpeg', dataset_key='', varlen_default_value=None)
I1219 22:54:13.267408 140043058079552 train_eval.py:93] Model label specification.
I1219 22:54:13.267512 140043058079552 train_eval.py:70] reward: ExtendedTensorSpec(shape=(?, 1), dtype=tf.float32, name='meta_labels/reward', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
I1219 22:54:13.267543 140043058079552 train_eval.py:70] target_pose: ExtendedTensorSpec(shape=(?, 2), dtype=tf.float32, name='meta_labels/target_pose', is_optional=False, is_sequence=False, is_extracted=False, data_format=None, dataset_key='', varlen_default_value=None)
INFO:tensorflow:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I1219 22:54:13.267574 140043058079552 train_eval.py:119] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[  FAILED  ] PoseEnvModelsTest.test_regression_maml
[ RUN      ] PoseEnvModelsTest.test_regression_maml_policy_interface
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W1219 22:54:13.314008 140043058079552 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/variable_scope.py:452: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  shape is not None and isinstance(shape, collections_lib.Sequence) and
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W1219 22:54:13.317741 140043058079552 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
W1219 22:54:13.385801 140043058079552 deprecation.py:506] From /home/robotics/.local/lib/python3.7/site-packages/tensor2robot/layers/spatial_softmax.py:80: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1219 22:54:13.570754 140043058079552 deprecation.py:323] From /home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
2023-12-19 22:54:13.985586: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-12-19 22:54:13.987903: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2995200000 Hz
2023-12-19 22:54:13.988047: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ccf81c35b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2023-12-19 22:54:13.988061: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
I1219 22:54:14.002703 140043058079552 checkpoint_predictor.py:131] Initializing model with random weights
[  FAILED  ] PoseEnvModelsTest.test_regression_maml_policy_interface
[ RUN      ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin')
I1219 22:54:14.098639 140043058079552 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg_maml.gin
[  FAILED  ] PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin')
[ RUN      ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin')
I1219 22:54:14.099249 140043058079552 resource_reader.py:50] system_path_file_exists:tensor2robot/research/pose_env/configs/run_train_reg.gin
[  FAILED  ] PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin')
======================================================================
ERROR: test_mc (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_mc
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 75, in test_mc
    create_exporters_fn=None)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model
    mode=tf_estimator.ModeKeys.TRAIN,
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information
    tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression))
NameError: name 'guzzler_use_compression' is not defined
  In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f5dd4f1ec20>)

======================================================================
ERROR: test_regression (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_regression
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 82, in test_regression
    create_exporters_fn=None)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model
    mode=tf_estimator.ModeKeys.TRAIN,
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information
    tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression))
NameError: name 'guzzler_use_compression' is not defined
  In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f5dd4f1ec20>)

======================================================================
ERROR: test_regression_maml (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_regression_maml
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 91, in test_regression_maml
    create_exporters_fn=None)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 492, in train_eval_model
    mode=tf_estimator.ModeKeys.TRAIN,
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval.py", line 120, in provide_input_generator_with_model_information
    tf.logging.info('guzzler_use_compression %s', str(guzzler_use_compression))
NameError: name 'guzzler_use_compression' is not defined
  In call to configurable 'train_eval_model' (<function train_eval_model at 0x7f5dd4f1ec20>)

======================================================================
ERROR: test_regression_maml_policy_interface (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_regression_maml_policy_interface
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 118, in test_regression_maml_policy_interface
    self._test_policy_interface(policy, restore=False)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 103, in _test_policy_interface
    action = policy.SelectAction(obs, None, 0)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/meta_learning/meta_policies.py", line 122, in SelectAction
    action = self._predictor.predict(np_features)['inference_output']
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/predictors/checkpoint_predictor.py", line 117, in predict
    tensorspec_utils.map_feed_dict(self._features, features))
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 956, in map_feed_dict
    maybe_ignore_batch(spec_numpy, ignore_batch))
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1206, in assert_required
    flat_expected_spec, flat_actual_spec, ignore_batch)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1165, in assert_equal
    flattened_actual_tensors_or_spec)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/home/robotics/.local/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/tensorspec_utils.py", line 1126, in assert_equal_spec_or_tensor
    expected_spec.dtype, actual_spec.dtype, expected_spec, actual_spec))
ValueError: TensorSpec.dtype <dtype: 'uint8'> does not match TensorSpec.dtype <dtype: 'int64'> in specs
 expected: ExtendedTensorSpec(shape=(?, 64, 64, 3), dtype=tf.uint8, name='condition_ep0/state/image', is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None)
 actual: ExtendedTensorSpec(shape=(1, 64, 64, 3), dtype=tf.int64, name=None, is_optional=False, is_sequence=False, is_extracted=True, data_format=None, dataset_key='', varlen_default_value=None)

======================================================================
ERROR: test_train_eval_gin0 ('run_train_reg_maml.gin') (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_train_eval_gin0 ('run_train_reg_maml.gin')
test_train_eval_gin('run_train_reg_maml.gin')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test
    return test_method(self, *testcase_params)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin
    eval_steps=EVAL_STEPS)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin
    gin.parse_config_file(full_gin_path, print_includes_and_imports=True)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file
    if existence_check(config_file_with_prefix):
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists
    path = _parse_config_path(config_path)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path
    path = os.path.join(os.path.dirname(file_sys_path), filename)
  File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

======================================================================
ERROR: test_train_eval_gin1 ('run_train_reg.gin') (__main__.PoseEnvModelsTest)
PoseEnvModelsTest.test_train_eval_gin1 ('run_train_reg.gin')
test_train_eval_gin('run_train_reg.gin')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robotics/.local/lib/python3.7/site-packages/absl/testing/parameterized.py", line 321, in bound_param_test
    return test_method(self, *testcase_params)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/research/pose_env/pose_env_models_test.py", line 133, in test_train_eval_gin
    eval_steps=EVAL_STEPS)
  File "/home/robotics/.local/lib/python3.7/site-packages/tensor2robot/utils/train_eval_test_utils.py", line 122, in test_train_eval_gin
    gin.parse_config_file(full_gin_path, print_includes_and_imports=True)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/config.py", line 2448, in parse_config_file
    if existence_check(config_file_with_prefix):
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 52, in system_path_file_exists
    path = _parse_config_path(config_path)
  File "/home/robotics/.local/lib/python3.7/site-packages/gin/resource_reader.py", line 89, in _parse_config_path
    path = os.path.join(os.path.dirname(file_sys_path), filename)
  File "/usr/local/lib/python3.7/posixpath.py", line 156, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

----------------------------------------------------------------------
Ran 6 tests in 0.844s

FAILED (errors=6)

@ngochieu642 I haven't met the error you faced, maybe I had when debug with the other dependency packages. The follows is what I built in my docker image:

Package Version

absl-py 0.15.0 astor 0.8.1 astunparse 1.6.3 cachetools 5.3.2 certifi 2023.11.17 charset-normalizer 3.3.2 cloudpickle 2.2.1 decorator 5.1.1 dm-sonnet 2.0.0 dm-tree 0.1.8 flatbuffers 1.12 gast 0.2.2 gin-config 0.5.0 google-auth 2.23.4 google-auth-oauthlib 0.4.6 google-pasta 0.2.0 grpcio 1.32.0 gym 0.12.5 gym-notices 0.0.8 h5py 2.10.0 idna 3.6 importlib-metadata 6.7.0 Keras 2.1.6 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.2 Markdown 3.4.4 MarkupSafe 2.1.3 numpy 1.19.5 oauthlib 3.2.2 opt-einsum 3.3.0 Pillow 5.3.0 pip 23.0.1 protobuf 3.20.3 pyasn1 0.5.1 pyasn1-modules 0.3.0 pybullet 2.5.0 pygame 2.1.3 pyglet 2.0.9 PyYAML 6.0.1 requests 2.31.0 requests-oauthlib 1.3.1 rsa 4.9 scipy 1.7.3 setuptools 47.1.0 six 1.15.0 tabulate 0.9.0 tensorboard 1.15.0 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorflow 1.15.0 tensorflow-estimator 1.15.1 tensorflow-gpu 1.15.0 tensorflow-graphics 2020.5.20 tensorflow-probability 0.7.0 tensorflow-serving-api 1.15.0 termcolor 1.1.0 tf-agents 0.3.0 tf-slim 1.1.0 tfp-nightly 0.20.0.dev20230516 typing-extensions 3.7.4.3 urllib3 2.0.7 Werkzeug 2.2.3 wheel 0.42.0 wrapt 1.12.1 zipp 3.15.0

After you install the package, you need to comment the TPU wraper support, and keep mind of the old tf import format:"import tensorflow.compat.v1 as tf"

google-research / tensor2robot

Tensorflow Version Conflict #20