Question about TMO and TMO++

[JawadTawhidi]

Hi, sorry I got a bit confused while using your updated repo. Before there was only code for TMO and there was no any option for choosing encoder or output selection, so it means when we were downloading the code we had TMO's code.

But now we have many options， that is why I have few questions:

1. If I download the code directly and train it without making any changes in the options, is it TMO++ using mitb1 encoder? I mean with the following default settings:

   set device

   torch.cuda.set_device(0)

   define model

   ver = 'mitb1' aos = True model = TMO(ver, aos).eval()

   training stage

   if options.train: model = torch.nn.DataParallel(model) train_duts_davis(model, ver)

2. In the above default settings if I only replace the 'mitb1' with 'rn101' and assume it is TMO++ with Resnet 101 encoder, am I right?

   set device

   torch.cuda.set_device(0)

   define model

   ver = 'rn101' aos = True model = TMO(ver, aos).eval()

   training stage

   if options.train: model = torch.nn.DataParallel(model) train_duts_davis(model, ver)

3. In this case what if I want to use TMO? if I only set the aos = False, the code will be TMO's code? For example like this:

TMO with mitb1 encoder:

# set device
torch.cuda.set_device(1)

# define model
ver = 'mitb1'
aos = False
model = TMO(ver, aos).eval()

# training stage
if options.train:
   # model = torch.nn.DataParallel(model)
    train_duts_davis(model, ver)

TMO with Resnet 101 encoder:

# set device
torch.cuda.set_device(1)

# define model
ver = 'rn101'
aos = False
model = TMO(ver, aos).eval()

# training stage
if options.train:
   # model = torch.nn.DataParallel(model)
    train_duts_davis(model, ver)

[suhwan-cho]

Hi, yes they are all correct. You can choose TMO or TMO++ by controlling "aos" option.

[JawadTawhidi]

Hi, Thank you so much for your response. However, while I want to train TMO or TMO++ using mitb1, it gives me this error:

python run.py --train Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connection.py", line 200, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection raise err File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request raise new_e File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request self._validate_conn(conn) File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn conn.connect() File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connection.py", line 604, in connect self.sock = sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connection.py", line 209, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f9fec1c6950>, 'Connection to huggingface.co timed out. (connect timeout=10)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /nvidia/mit-b1/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f9fec1c6950>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download metadata = get_hf_file_metadata( ^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(args, kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper response = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 408, in _request_wrapper response = get_session().request(method=method, url=url, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 67, in send return super().send(request, args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/requests/adapters.py", line 507, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /nvidia/mit-b1/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f9fec1c6950>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 1e590f4a-050f-4a01-91ff-461562bb92f6)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/transformers/utils/hub.py", line 417, in cached_file resolved_file = hf_hub_download( ^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1406, in hf_hub_download raise LocalEntryNotFoundError( huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/users/JawadT/TMO++(RN-101)/TMO/run.py", line 145, in model = TMO(ver, aos).eval() ^^^^^^^^^^^^^ File "/home/users/JawadT/TMO++(RN-101)/TMO/tmo.py", line 171, in init self.vos = VOS(ver) ^^^^^^^^ File "/home/users/JawadT/TMO++(RN-101)/TMO/tmo.py", line 162, in init self.app_encoder = Encoder(ver) ^^^^^^^^^^^^ File "/home/users/JawadT/TMO++(RN-101)/TMO/tmo.py", line 70, in init self.backbone = SegformerModel.from_pretrained('nvidia/mit-b1') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/transformers/modeling_utils.py", line 2305, in from_pretrained config, model_kwargs = cls.config_class.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/transformers/configuration_utils.py", line 547, in from_pretrained config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/transformers/configuration_utils.py", line 574, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/transformers/configuration_utils.py", line 629, in _get_config_dict resolved_config_file = cached_file( ^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/First/lib/python3.11/site-packages/transformers/utils/hub.py", line 452, in cached_file raise EnvironmentError( OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like nvidia/mit-b1 is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

[JawadTawhidi]

• It is giving me this error while I have installed all the required packages, I even installed the transformers :

[JawadTawhidi]

And Please respond this question as well, I really need it.

[suhwan-cho]

Hi, I believe the error occurred temporarily due to issues with the huggingface server. Please try again as I have not encountered such errors.