convert yolact to ONNX

[sdimantsd]

Hello again, I'm try to convert yolact to ONNX with the following code:

weights_path = '/home/ws/DL/yolact/weights/yolact_im700_54_800000.pth'

import torch
import torch.onnx
import yolact
import torchvision

model = yolact.Yolact()

# state_dict = torch.load(weights_path)
# model.load_state_dict(state_dict)

model.load_weights(weights_path)

dummy_input = torch.randn(1, 3, 640, 480)

torch.onnx.export(model, dummy_input, "onnx_model_name.onnx")

error msg:

/home/ws/DL/yolact/yolact.py:256: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  for j, i in product(range(conv_h), range(conv_w)):
/home/ws/DL/yolact/yolact.py:279: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  self.priors = torch.Tensor(prior_data).view(-1, 4)
/home/ws/DL/yolact/yolact.py:279: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  self.priors = torch.Tensor(prior_data).view(-1, 4)
/home/ws/DL/yolact/layers/functions/detection.py:74: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  for batch_idx in range(batch_size):
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-a796dc0eef97> in <module>
     13 dummy_input = torch.randn(1, 3, 700, 700)
     14 
---> 15 torch.onnx.export(model, dummy_input, "onnx_model_name.onnx")

~/.local/lib/python3.6/site-packages/torch/onnx/__init__.py in export(*args, **kwargs)
     23 def export(*args, **kwargs):
     24     from torch.onnx import utils
---> 25     return utils.export(*args, **kwargs)
     26 
     27 

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in export(model, args, f, export_params, verbose, training, input_names, output_names, aten, export_raw_ir, operator_export_type, opset_version, _retain_param_name, do_constant_folding, strip_doc_string)
    129             operator_export_type=operator_export_type, opset_version=opset_version,
    130             _retain_param_name=_retain_param_name, do_constant_folding=do_constant_folding,
--> 131             strip_doc_string=strip_doc_string)
    132 
    133 

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in _export(model, args, f, export_params, verbose, training, input_names, output_names, operator_export_type, export_type, example_outputs, propagate, opset_version, _retain_param_name, do_constant_folding, strip_doc_string)
    361                                                         output_names, operator_export_type,
    362                                                         example_outputs, propagate,
--> 363                                                         _retain_param_name, do_constant_folding)
    364 
    365         # TODO: Don't allocate a in-memory string for the protobuf

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in _model_to_graph(model, args, verbose, training, input_names, output_names, operator_export_type, example_outputs, propagate, _retain_param_name, do_constant_folding, _disable_torch_constant_prop)
    264             model.graph, tuple(args), example_outputs, False, propagate)
    265     else:
--> 266         graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
    267         state_dict = _unique_state_dict(model)
    268         params = list(state_dict.values())

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in _trace_and_get_graph_from_model(model, args, training)
    223     # training mode was.)
    224     with set_training(model, training):
--> 225         trace, torch_out = torch.jit.get_trace_graph(model, args, _force_outplace=True)
    226 
    227     if orig_state_dict_keys != _unique_state_dict(model).keys():

~/.local/lib/python3.6/site-packages/torch/jit/__init__.py in get_trace_graph(f, args, kwargs, _force_outplace, return_inputs)
    229     if not isinstance(args, tuple):
    230         args = (args,)
--> 231     return LegacyTracedModule(f, _force_outplace, return_inputs)(*args, **kwargs)
    232 
    233 

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/.local/lib/python3.6/site-packages/torch/jit/__init__.py in forward(self, *args)
    292         try:
    293             trace_inputs = _unflatten(all_trace_inputs[:len(in_vars)], in_desc)
--> 294             out = self.inner(*trace_inputs)
    295             out_vars, _ = _flatten(out)
    296             torch._C._tracer_exit(tuple(out_vars))

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    489             hook(self, input)
    490         if torch._C._get_tracing_state():
--> 491             result = self._slow_forward(*input, **kwargs)
    492         else:
    493             result = self.forward(*input, **kwargs)

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
    479         tracing_state._traced_module_stack.append(self)
    480         try:
--> 481             result = self.forward(*input, **kwargs)
    482         finally:
    483             tracing_state.pop_scope()

~/DL/yolact/yolact.py in forward(self, x)
    615                 pred_outs['conf'] = F.softmax(pred_outs['conf'], -1)
    616 
--> 617             return self.detect(pred_outs)
    618 
    619 

~/DL/yolact/layers/functions/detection.py in __call__(self, predictions)
     73 
     74             for batch_idx in range(batch_size):
---> 75                 decoded_boxes = decode(loc_data[batch_idx], prior_data)
     76                 result = self.detect(batch_idx, conf_preds, decoded_boxes, mask_data, inst_data)
     77 

RuntimeError: isTensor() ASSERT FAILED at /pytorch/aten/src/ATen/core/ivalue.h:209, please report a bug to PyTorch. (toTensor at /pytorch/aten/src/ATen/core/ivalue.h:209)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7f721e0ac441 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7f721e0abd7a in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #2: <unknown function> + 0x979ad2 (0x7f721d130ad2 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #3: torch::jit::tracer::getNestedValueTrace(c10::IValue const&) + 0x41 (0x7f721d3939a1 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #4: <unknown function> + 0xa7651b (0x7f721d22d51b in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #5: <unknown function> + 0xa766db (0x7f721d22d6db in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #6: <unknown function> + 0x457942 (0x7f725d6d2942 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #7: <unknown function> + 0x130cfc (0x7f725d3abcfc in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #8: _PyCFunction_FastCallDict + 0x35c (0x56204c in /usr/bin/python3)
frame #9: /usr/bin/python3() [0x5a1501]
frame #10: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #11: /usr/bin/python3() [0x5136c6]
frame #12: _PyObject_FastCallKeywords + 0x19c (0x57ec0c in /usr/bin/python3)
frame #13: /usr/bin/python3() [0x4f88ba]
frame #14: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #15: _PyFunction_FastCallDict + 0xf5 (0x4f4065 in /usr/bin/python3)
frame #16: /usr/bin/python3() [0x5a1481]
frame #17: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #18: /usr/bin/python3() [0x513601]
frame #19: _PyObject_FastCallKeywords + 0x19c (0x57ec0c in /usr/bin/python3)
frame #20: /usr/bin/python3() [0x4f88ba]
frame #21: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #22: /usr/bin/python3() [0x4f6128]
frame #23: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #24: /usr/bin/python3() [0x5a1481]
frame #25: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #26: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #27: /usr/bin/python3() [0x4f6128]
frame #28: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #29: /usr/bin/python3() [0x5a1481]
frame #30: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #31: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #32: /usr/bin/python3() [0x4f6128]
frame #33: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #34: /usr/bin/python3() [0x5a1481]
frame #35: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #36: /usr/bin/python3() [0x513601]
frame #37: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #38: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #39: /usr/bin/python3() [0x4f6128]
frame #40: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #41: /usr/bin/python3() [0x5a1481]
frame #42: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #43: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #44: /usr/bin/python3() [0x4f6128]
frame #45: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #46: /usr/bin/python3() [0x5a1481]
frame #47: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #48: /usr/bin/python3() [0x513601]
frame #49: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #50: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #51: /usr/bin/python3() [0x4f6128]
frame #52: /usr/bin/python3() [0x4f7d60]
frame #53: /usr/bin/python3() [0x4f876d]
frame #54: _PyEval_EvalFrameDefault + 0x1260 (0x4fa6c0 in /usr/bin/python3)
frame #55: /usr/bin/python3() [0x4f7a28]
frame #56: /usr/bin/python3() [0x4f876d]
frame #57: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #58: /usr/bin/python3() [0x4f6128]
frame #59: /usr/bin/python3() [0x4f7d60]
frame #60: /usr/bin/python3() [0x4f876d]
frame #61: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #62: /usr/bin/python3() [0x4f6128]
frame #63: /usr/bin/python3() [0x4f7d60]

[dbolya]

See #59. You'll have to put some elbow grease in if you want to get YOLACT traceable (i.e., exportable to ONNX) since I use a lot of pythonic code. I hear @Wilber529 was able to do it following these steps: https://github.com/dbolya/yolact/issues/59#issuecomment-501609792. You have to rewrite how I pass around variables (dictionaries are not supported I think) and you'll have to rewrite anything after Yolact's forward function (starting with self.detect) in your target language because I wrote it in a super pythonic way to make the model faster.

[sdimantsd]

Hi @dbolya thanks for you'r answer! I'm not sure I understood you, Can you please expand?

[dbolya]

Yolact does not support conversion to ONNX, which is why you get an error. You'd need to change a lot of things to get conversion to ONNX to work, as outlined by @Wilber529 in https://github.com/dbolya/yolact/issues/59#issuecomment-501609792. I'm not making these changes to the main branch because they'd make the Python version run slower and make it harder to develop.

[sdimantsd]

thx

[Ma-Dan]

I have converted yolact to onnx without Detect part, and also modified some upsampling code. https://github.com/Ma-Dan/yolact/tree/onnx Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods. I also converted onnx model to CoreML model, 4 custom layers need to be implemented to make it work.

[abhigoku10]

@Ma-Dan thanks for sharing the reference code ,i shalll look into this process and get back to if i have queries

[aweissen1]

@Ma-Dan thank you very much for sharing your work. I am wondering, what needs to be implemented to execute the onnx model again. What does this mean? "Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods." Thanks for your help!

[ABlueLight]

@Ma-Dan Thank you for your code! I convert the model to onnx ,but the results is different from pytorch outpus,such loc , mask and proto, but conf is same! Do you see the problem?

[aweissen1]

@abhigoku10 actually I just used the onnx branch from Ma-Dan to create an onnx file. Do you get an error while converting?

[abhigoku10]

@aweissen1 i was facing some package issues i shall look into to more in depth and solve it , where there any difference i the output generated

[ABlueLight]

@Ma-Dan Hi, i convert to onnx ssuccessfully，but i found results is not corrent . can you share the version of the pytorch and onnxruntime are you using? Thx

[sicarioakki]

@Ma-Dan Can you give more information about the package dependencies for your Yolact-ONNX implementation? And also, have compared the results of Yolact and that of your Yolact-ONNX implementations? If so, please give us insight on it.

[abhigoku10]

@ABlueLight and @aweissen1 should us the base code given by @Ma-Dan and train the model , or just load the trained model with this code what is the command to be used . Please share the process Can i run it on gpu how much fps r u getting

[ABlueLight]

i convert to onnx successfully and results is correct, today. Thx @Ma-Dan @sicarioakki @abhigoku10 My package dependencies include pytorch1.0.0 torchvision0.2.1 onnx-tf1.3.0 onnxruntime0.4.0 onnx1.5.0 tensorflow-gpu1.14.0. Just use @Ma-Dan code is ok , i don't modify the codes,just replaced my trained model. Mayby the package version is a Important factors.

[abhigoku10]

@ABlueLight after conversion to onnx which platform are you going to deploy it . and did u convert to tensorflow based model

[ABlueLight]

@abhigoku10 TensorFlow and it can run correctly

[Ma-Dan]

Sorry for the delayed reply, I just fixed code on my repo to use correct onnx output. https://github.com/Ma-Dan/yolact/commit/a0648974369762445bc2095c2318f3f5c7fb7297 The previous version move prior constant output to a separate file to make CoreML file correct, and I forgot to fix onnx output index. Sorry again! And also notice that to make conversion to onnx correct, I hard coded sizes here. https://github.com/Ma-Dan/yolact/blob/onnx/yolact.py#L344 So this code could not work correctly on yolact_im700_54_800000.pth weight, you need to fix the size here.

[Ma-Dan]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1

Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

[abhigoku10]

@Ma-Dan thanks for the response, i have few queries

1. the onnx model which you obtained can be used with C++
2. can i convert that model to other framework like tf or caffe
3. In the command shared u have mentioned as --cuda=False , does it mean that it can run only on CPU and not on GPU , i wanted to run it on GPU

[aweissen1]

@Ma-Dan Thank you! Great job.

[ridasalam]

@ABlueLight how did you import it to Tensorflow?

[sicarioakki]

file_name= yolact_base_0_4000.onnx params= ['yolact', 'base', '0', '4000'] model_name= yolact_base epoch= 0 iteration= 4000 Config not specified. Parsed yolact_base_config from the file name.

Loading model...Traceback (most recent call last): File "onnxeval.py", line 1035, in net.load_weights(args.trained_model) File "/home/aeye/yolact-onnx/yolact_onnx_1/yolact.py", line 469, in load_weights state_dict = torch.load(path, map_location='cpu') File "/home/aeye/yolact-onnx/Yolact_ONNX/lib/python3.6/site-packages/torch/serialization.py", line 368, in load return _load(f, map_location, pickle_module) File "/home/aeye/yolact-onnx/Yolact_ONNX/lib/python3.6/site-packages/torch/serialization.py", line 532, in _load magic_number = pickle_module.load(f) _pickle.UnpicklingError: invalid load key, '\x08'.

I was able to covert the model to .onnx format. But while inferencing, i am facing the above issue.

[ABlueLight]

@ABlueLight how did you import it to Tensorflow? https://github.com/onnx/onnx-tensorflow

[abhigoku10]

@aweissen1 @ABlueLight hi guys , i am facing the same issue as above in my inference after conversion

file_name= yolact_base_0_4000.onnx params= ['yolact', 'base', '0', '4000'] model_name= yolact_base epoch= 0 iteration= 4000 Config not specified. Parsed yolact_base_config from the file name.

Loading model...Traceback (most recent call last): File "onnxeval.py", line 1035, in net.load_weights(args.trained_model) File "/home/aeye/yolact-onnx/yolact_onnx_1/yolact.py", line 469, in load_weights state_dict = torch.load(path, map_location='cpu') File "/home/aeye/yolact-onnx/Yolact_ONNX/lib/python3.6/site-packages/torch/serialization.py", line 368, in load return _load(f, map_location, pickle_module) File "/home/aeye/yolact-onnx/Yolact_ONNX/lib/python3.6/site-packages/torch/serialization.py", line 532, in _load magic_number = pickle_module.load(f) _pickle.UnpicklingError: invalid load key, '\x08'.

Any suggestions

[sicarioakki]

@Ma-Dan @aweissen1 @ABlueLight How are guys able to load the ONNX model using torch.load() function? Only onnx.load() can be used right?

[ridasalam]

@ABlueLight, do you have a huge difference in speed of inference?

I used @Ma-Dan 's helpful work to generate yolact.onnx and I load it through onnx load and onnx_tf.backend import prepare. All other post processing is still torch based. It takes 2 mins per image inference (compared to a couple of seconds in Pytorch)

Also, were you able to convert it to pure Tensorflow? (use Tensorflow pb file instead of onnx)

[ABlueLight]

@ridasalam I convert it to pure tensorflow and it const about 400~500ms on i5 cpu。 On GPU，pytorch and tensorflow cost time are almost equal.

[ABlueLight]

@sicarioakki ONNX model should be loaded by onnx.load(),i think..

[sdimantsd]

@ridasalam I convert it to pure tensorflow and it const about 400~500ms on i5 cpu。 On GPU，pytorch and tensorflow cost time are almost equal.

Can you share the project of tensorflow?

[JINGTING92]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1

Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

Hi, thanks for your codes, but I did not find the codes to generate onnx in eval.py, could anybody share the link?

[NinjaPerson24119]

Can someone provide their method for converting YOLACT to ONNX? I'm not getting sufficient speed on a Jetson Xavier, and would like to try C++ solution using TensorRT.

[GracefulTabby]

Can someone provide their method for converting YOLACT to ONNX? I'm not getting sufficient speed on a Jetson Xavier, and would like to try C++ solution using TensorRT.

How fast was it running when running with Xavier? Sorry for the issue

[NinjaPerson24119]

~3Hz with fp16 using darknet backend at 550x550

On Mon, Nov 25, 2019, 6:41 PM akaneko1019, notifications@github.com wrote:

Can someone provide their method for converting YOLACT to ONNX? I'm not getting sufficient speed on a Jetson Xavier, and would like to try C++ solution using TensorRT.

How fast was it running when running with Xavier? Sorry for the issue

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dbolya/yolact/issues/74?email_source=notifications&email_token=AHXXQOUBFADHCZRNXIAA3L3QVR5FHA5CNFSM4H2YZ7SKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFEM7HI#issuecomment-558419869, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHXXQOULURAICZFC7YPT22LQVR5FHANCNFSM4H2YZ7SA .

[HoangTienDuc]

@JING switch to branch onnx. =)

[saisubramani]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1

Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned. the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU0 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU1 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU2 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU3 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) Traceback (most recent call last): File "train.py", line 382, in train() File "train.py", line 143, in train yolact_net = Yolact() File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init self.backbone = construct_backbone(cfg.backbone) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone backbone = cfg.type(*cfg.args) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init False, _pair(0), groups, bias, padding_mode) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init self.reset_parameters() File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters init.kaiminguniform(self.weight, a=math.sqrt(5)) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiminguniform return tensor.uniform_(-bound, bound) RuntimeError: CUDA error: no kernel image is available for execution on the device

[Ma-Dan]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1 Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned. the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU0 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU1 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU2 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU3 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) Traceback (most recent call last): File "train.py", line 382, in train() File "train.py", line 143, in train yolact_net = Yolact() File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init self.backbone = construct_backbone(cfg.backbone) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone backbone = cfg.type(*cfg.args) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init False, _pair(0), groups, bias, padding_mode) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init self.reset_parameters() File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters init.kaiminguniform(self.weight, a=math.sqrt(5)) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiminguniform return tensor.uniform_(-bound, bound) RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

[saisubramani]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1 Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned. the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU0 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU1 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU2 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU3 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) Traceback (most recent call last): File "train.py", line 382, in train() File "train.py", line 143, in train yolact_net = Yolact() File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init self.backbone = construct_backbone(cfg.backbone) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone backbone = cfg.type(*cfg.args) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init False, _pair(0), groups, bias, padding_mode) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init self.reset_parameters() File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters init.kaiminguniform(self.weight, a=math.sqrt(5)) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiminguniform return tensor.uniform_(-bound, bound) RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

thanks for the reply @Ma-Dan, I understand that , we are going to use the trained model (yolact_base_63_8000.pth) in your script. Can i know which script is used for converting the (yolact_base_63_8000.pth)------> (yolact_base_63_800.onnx) file.

[saisubramani]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1 Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned. the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU0 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU1 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU2 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU3 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) Traceback (most recent call last): File "train.py", line 382, in train() File "train.py", line 143, in train yolact_net = Yolact() File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init self.backbone = construct_backbone(cfg.backbone) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone backbone = cfg.type(*cfg.args) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init False, _pair(0), groups, bias, padding_mode) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init self.reset_parameters() File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters init.kaiminguniform(self.weight, a=math.sqrt(5)) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiminguniform return tensor.uniform_(-bound, bound) RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

Can you Please tell the steps for converting the yolact.pth model to .onnx model, and mention which script should be used for converting,so that it can be helpful to me. My idea is to convert the model into tensorRT, so i am trying to convert the [yolact to onnx to tensorRT].

[saisubramani]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1 Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned. the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU0 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU1 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU2 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) /home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning: Found GPU3 GRID K520 which is of cuda capability 3.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1])) Traceback (most recent call last): File "train.py", line 382, inhow can train() File "train.py", line 143, in train yolact_net = Yolact() File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init self.backbone = construct_backbone(cfg.backbone) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone backbone = cfg.type(*cfg.args) File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init False, _pair(0), groups, bias, padding_mode) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init self.reset_parameters() File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters init.kaiminguniform(self.weight, a=math.sqrt(5)) File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiminguniform return tensor.uniform_(-bound, bound) RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

hi i had founded that you are using eval.py script for converting the yolact model to onnx model i having a doubt pred_outs = net(batch) This give a list which having an size of 1, how you are using the index in , preds = detect({'loc': pred_outs[0], 'conf': pred_outs[1], 'mask':pred_outs[2], 'priors': pred_outs[3], 'proto': pred_outs[4]})

it showing

IndexError: list index out of range

so what i did is , i just added few lines pred_outs = dict(pred_outs[0]) pred_outs=pred_outs['detection']

now its in dictionary format by using the key value i can take the values of detection, but when i cross checked the detection which is in (dictionary format) it having a key values of

('mask','class','score','proto','net')

what value can i assign for the

pred_out[0],pred_outs[1],pred_outs[2], pred_outs[3],pred_outs[4]

In my understanding 'conf' ':mean score,'mask':mask,'proto': means proto what about 'loc' and 'priors'

preds = detect({'loc': pred_outs[0], 'conf': pred_outs[1], 'mask':pred_outs[2], 'priors': pred_outs[3], 'proto': pred_outs[4]})

i tried like this preds = detect({'loc': pred_outs['box'], 'conf': pred_outs['score'], 'mask':pred_outs['mask'], 'priors': pred_outs['class'], 'proto': pred_outs['proto']}) It showing Error: TypeError: __call__() missing 1 required positional argument: 'net' can you help me to sort this issue ? if i am wrong please tell me.@Ma-Dan

[AlexanderSlav]

Hi,@Ma-Dan I'm trying to convert yolact model to TensorRT and facing number of issues.

Here is my working environment :

• pytorch == 1.4.0

• TensorRT == 7.0.0(official docker release 20.01)

There are links to yolact model in onnx format opset version == 9,

The Error is : UNSUPPORTED_NODE: Assertion failed: scales_input.is_weights()

opset version == 11 The Error is : INVALID_GRAPH: Assertion failed: ctx->tensors().count(inputName)

Thank you in advance for your help

[dzyjjpy]

I have converted yolact to onnx without Detect part, and also modified some upsampling code. https://github.com/Ma-Dan/yolact/tree/onnx Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods. @Ma-Dan I also converted onnx model to CoreML model, 4 custom layers need to be implemented to make it work.

Thanks for your gread job. I follow your code to convert onnx success, but convert onnx to coreml, it shows errors about upsample layer(you mentioned you modify some upsampling code, could you pls share the modification part? you men the function: def _convert_upsample(builder, node, graph, err): in /home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators.py" ): 175/308: Converting Node Type Upsample 176/308: Converting Node Type Conv 177/308: Converting Node Type Add 178/308: Converting Node Type Upsample Traceback (most recent call last): File "/home/jiapy/workspace/segmentation/yolact-coreml/onnx_to_coreml.py", line 15, in minimum_ios_deployment_target='12' # TypeError: 'set' object is not callable File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/converter.py", line 629, in convert _convert_node(builder, node, graph, err) File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators.py", line 2017, in _convert_node return converter_fn(builder, node, graph, err) File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators.py", line 1654, in _convert_upsample input_shape = graph.shape_dict[node.inputs[0]] KeyError: '533'

Process finished with exit code 1

[bbico]

I have converted yolact to onnx without Detect part, and also modified some upsampling code. https://github.com/Ma-Dan/yolact/tree/onnx Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods. I also converted onnx model to CoreML model, 4 custom layers need to be implemented to make it work.

Hi. I'm pretty newbie of ML. I followed Ma-Dan's work for a while, and finally I got yolact.onnx model. But my ultimate goal is to run Unity program with yolact. And now, I got another error dealing with onnx model.

Unexpected error while evaluating model output 783. System.ArgumentException: Cannot reshape array of size 62208 into shape with multiple of 1024 elements at Barracuda.TensorExtensions.Reshape

I used opset=9, input=(1,550,550,3), model=resnet50-54 Someone posted simillar issues about importing onnx, but I couldn't find the exact solution. So, if anyone had same issues or solved it, please give me a tip.

[biyuehuang]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1

Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

Hi @Ma-Dan , Do you know how to convert yolact_plus_base_54_800000.pth to ONNX. I run $python eval.py --trained_model=weights/yolact_plus_base_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg, got Eorror: Multiple GPUs detected! Turning off JIT. Config not specified. Parsed yolact_plus_base_config from the file name.

Traceback (most recent call last): File "eval.py", line 980, in set_cfg(args.config) File "/home/username/Document/yolact/yolact/data/config.py", line 676, in set_cfg cfg.replace(eval(config_name)) File "", line 1, in NameError: name 'yolact_plus_base_config' is not defined

[amitkumar-delhivery]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1

Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

Thank you so much @Ma-Dan , have you tried converting it to tflite?

[amitkumar-delhivery]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1 Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

Hi @Ma-Dan , Do you know how to convert yolact_plus_base_54_800000.pth to ONNX. I run $python eval.py --trained_model=weights/yolact_plus_base_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg, got Eorror: Multiple GPUs detected! Turning off JIT. Config not specified. Parsed yolact_plus_base_config from the file name.

Traceback (most recent call last): File "eval.py", line 980, in set_cfg(args.config) File "/home/username/Document/yolact/yolact/data/config.py", line 676, in set_cfg cfg.replace(eval(config_name)) File "", line 1, in NameError: name 'yolact_plus_base_config' is not defined

give config file --config=custom_config as argument when executing eval.py as you might have provided while running the train.py!

[amitkumar-delhivery]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1 Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

Thank you so much @Ma-Dan , have you tried converting it to tflite?

Done :)

[bmabir17]

The environment I used: onnx 1.4.1 onnxruntime 0.4.0 torch 1.0.1 torchvision 0.2.1 Run python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to generate onnx file. And run python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg to evaluate with onnx.

Thank you so much @Ma-Dan , have you tried converting it to tflite?

Done :)

@amitkumar-delhivery were you able to convert it into tflite? if so did you used it on mobile devices(Android) ?

[Chase2816]

@ridasalam 我将其转换为纯tensorflow，在i5 cpu上约400〜500ms。 在GPU上，pytorch和tensorflow的花费时间几乎相等。 yolact转为纯tensorflow得项目能共享吗？

[carlsummer]

RuntimeError: Only tuples, lists and Variables supported as JIT inputs/outputs. Dictionaries and strings are also accepted but their usage is not recommended. But got unsupported type Yolact

[h-aboutalebi]

@ABlueLight you said you were successful in converting the yolact to onnx and deploy it on TensorFlow. I was wondering if you could share your code? I am still trying to figure out how to convert Yolact to ONNX and then deploy it on TensorFlow. Thanks!