isl-org / Open3D-ML

An extension of Open3D to address 3D Machine Learning tasks
Other
1.88k stars 321 forks source link

OpenVino is not working with TensorFlow backend #603

Open maffan-96 opened 1 year ago

maffan-96 commented 1 year ago

Checklist

Describe the issue

I have been trying to run RANDLANET based on the S3DIS dataset using OpenVino backend with TensorFlow. I am facing this issue.

Steps to reproduce the bug

from datetime import datetime
import pyrealsense2 as rs
import numpy as np
from open3d.geometry import PointCloud, Image
import open3d as o3d 
from open3d.camera import PinholeCameraIntrinsic
from open3d.utility import Vector3dVector
from open3d.visualization import Visualizer
import os
import open3d.ml as _ml3d
import open3d.ml.tf as ml3d
import glob

# Load an ML configuration file
cfg_file = "./randlanet_s3dis.yml"
cfg = _ml3d.utils.Config.load_from_file(cfg_file)

# Load the RandLANet model
model = ml3d.models.RandLANet(**cfg.model)
model = ml3d.models.OpenVINOModel(model)
# Add path to the SemanticKitti dataset and your own custom dataset
cfg.dataset['dataset_path'] = './logs/'

# Load the datasets
dataset = ml3d.datasets.S3DIS(cfg.dataset.pop('dataset_path', None), **cfg.dataset)
dataset.get_label_to_names

# Create the ML pipeline
pipeline_ml = ml3d.pipelines.SemanticSegmentation(model, device="gpu", **cfg.pipeline)

# Download the weights.
ckpt_folder = "./logs/tf/randlanet_s3dis_202201071330utc/"
os.makedirs(ckpt_folder, exist_ok=True)
ckpt_path = ckpt_folder + "ckpt-1"
#Please download the tf randlanet from the weblink given in the readme section
if not os.path.exists(ckpt_path):
    #cmd = "wget {} -O {}".format(randlanet_url, ckpt_path)
    #os.system(cmd)
    raise Exception("**********path didn't have the checkpoint")

# Load the parameters of the model.
pipeline_ml.load_ckpt(ckpt_path=ckpt_path)

Error message

File "/home/maffan/open3d_/open3d_experiments/open3d_randlanet_s3ds_tf_openvino.py", line 108, in pipeline_ml.load_ckpt(ckpt_path=ckpt_path) File "/home/maffan/miniconda3/envs/tf/lib/python3.9/site-packages/open3d/_ml3d/tf/pipelines/semantic_segmentation.py", line 509, in load_ckpt self.ckpt = tf.train.Checkpoint(step=tf.Variable(1), net=self.model) File "/home/maffan/miniconda3/envs/tf/lib/python3.9/site-packages/tensorflow/python/training/tracking/util.py", line 2017, in init _assert_trackable(converted_v, k) File "/home/maffan/miniconda3/envs/tf/lib/python3.9/site-packages/tensorflow/python/training/tracking/util.py", line 1463, in _assert_trackable raise ValueError( ValueError: Checkpoint was expecting net to be a trackable object (an object derived from Trackable), got <open3d._ml3d.tf.models.openvino_model.OpenVINOModel object at 0x7ff14840baf0>. If you believe this object should be trackable (i.e. it is part of the TensorFlow Python API and manages state), please open an issue.

Expected behavior

Ideally as per the the Open3D documentation, the OpnVino wrapper for TensorFlow should work without any error.

Open3D, Python and System information

- Operating system: Ubuntu 20.04
- Python version: 3.9.16
- Open3D version: 0.17.0
- OpenVino version: 2023.0.0
- TensorFlow version: Version: 2.8.2
- System type: x84 
- Is this a remote workstation?: no
- How did you install Open3D?: pip
- Compiler version (if built from source): gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

Additional information

No response