Questions during Run Gradio demo for amodal completion and segmentation

[dahaigui]

When I run the command python app.py, it gives me the error

Loading model from ./ckpt/epoch=000005.ckpt
Global Step: 3200
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.54 M params.
Keeping EMAs of 688.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py:101: UserWarning: Failed to download model from HuggingFaceCompatibleDownloader downloader. Trying to download from HuggingFaceCompatibleDownloader downloader.
  warnings.warn(
/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py:107: UserWarning: Failed to download model from HuggingFaceCompatibleDownloader downloader. No fallback downloader available.
  warnings.warn(
Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f18e4694070>: Failed to establish a new connection: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Carve/tracer_b7/resolve/d8a8fd9e7b3fa0d2f1506fe7242966b34381e9c5/tracer_b7.pth (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f18e4694070>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 170, in download_model_base
    r = requests.get(hugging_face_url, stream=True)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 725, in send
    history = [resp for resp in gen]
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 725, in <listcomp>
    history = [resp for resp in gen]
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 266, in resolve_redirects
    resp = self.send(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Carve/tracer_b7/resolve/d8a8fd9e7b3fa0d2f1506fe7242966b34381e9c5/tracer_b7.pth (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f18e4694070>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 98, in download_model
    return self.download_model_base(file_name)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 190, in download_model_base
    raise ConnectionError(
ConnectionError: Exception caught when downloading model! Model name: tracer_b7.pth. Exception: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Carve/tracer_b7/resolve/d8a8fd9e7b3fa0d2f1506fe7242966b34381e9c5/tracer_b7.pth (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f18e4694070>: Failed to establish a new connection: [Errno 101] Network is unreachable')).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f18f0108c40>: Failed to establish a new connection: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Carve/tracer_b7/resolve/d8a8fd9e7b3fa0d2f1506fe7242966b34381e9c5/tracer_b7.pth (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f18f0108c40>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 170, in download_model_base
    r = requests.get(hugging_face_url, stream=True)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Carve/tracer_b7/resolve/d8a8fd9e7b3fa0d2f1506fe7242966b34381e9c5/tracer_b7.pth (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f18f0108c40>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chenhx/software/pix2gestalt/pix2gestalt/app.py", line 307, in <module>
    fire.Fire(run_demo)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/chenhx/software/pix2gestalt/pix2gestalt/app.py", line 208, in run_demo
    interface = create_carvekit_interface()
  File "/home/chenhx/software/pix2gestalt/pix2gestalt/ldm/util.py", line 48, in create_carvekit_interface
    interface = HiInterface(object_type="object",
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/api/high.py", line 56, in __init__
    self.u2net = TracerUniversalB7(
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/ml/wrap/tracer_b7.py", line 50, in __init__
    model_path = tracer_b7_pretrained()
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/ml/files/models_loc.py", line 53, in tracer_b7_pretrained
    return downloader("tracer_b7.pth")
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 118, in __call__
    return self.download_model(file_name)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 105, in download_model
    return self.fallback_downloader.download_model(file_name)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 111, in download_model
    raise e
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 98, in download_model
    return self.download_model_base(file_name)
  File "/home/chenhx/miniconda3/envs/pix2gestalt/lib/python3.9/site-packages/carvekit/utils/download_models.py", line 190, in download_model_base
    raise ConnectionError(
ConnectionError: Exception caught when downloading model! Model name: tracer_b7.pth. Exception: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Carve/tracer_b7/resolve/d8a8fd9e7b3fa0d2f1506fe7242966b34381e9c5/tracer_b7.pth (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f18f0108c40>: Failed to establish a new connection: [Errno 101] Network is unreachable')).

[RunpuWei]

It seems to be a problem with your network, you can try using a proxy, or you can try downloading the weight locally, specify the local path and then running app.py.

[dahaigui]

Where should I put it in my local path?

[dahaigui]

I found the path I needed, $HOME/.cache/carvekit/checkpoints

[egeozguroglu]

Hi @dahaigui, is this resolved?

[dahaigui]

Hi @dahaigui, is this resolved?

Yes, this problem is solved.

[dahaigui]

Another question arise here

[dahaigui]

Is it possible to train with our own dataset?

[egeozguroglu]

As the "Could not create share link" message indicates, the Gradio share link error is due to your network connection. You can always use the local URL too.

Yes, to train on your own domain specific dataset you can simply replicate the format of our dataset, change the config path here, then execute the provided training command. You'd need occlusion images (256x256x3), corresponding whole (amodal) images (256x256x3), and binary visible (modal) masks (256x256x1).

The loader available here should further clarify the expected format. That said, instead of training from the Stable Diffusion weights (--finetune_from ckpt/sd-image-conditioned-v2.ckpt), fine-tuning from our weights (--finetune_from ckpt/epoch=000005.ckpt) can better preserve the generalization. This would highly depend on the # of samples in your domain specific dataset.

[dahaigui]

My local URL is not accessible, I've added port number 7860.