Layout-Parser / layout-parser

A Unified Toolkit for Deep Learning Based Document Image Analysis
https://layout-parser.github.io/
Apache License 2.0
4.88k stars 467 forks source link

lp.Detectron2LayoutModel often hangs on download step (config/model) #43

Open Danferno opened 3 years ago

Danferno commented 3 years ago

Describe the bug When downloading the config.yaml or model_final.pth of a new (to user) model, the program often hangs.

To Reproduce Steps to reproduce the behavior:

  1. What command or script did you run?
    model_PLN_a = lp.Detectron2LayoutModel(
            config_path ='lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config', # In model catalog
            label_map   ={0: "Text", 1: "Title", 2: "List", 3:"Table", 4:"Figure"}, # In model`label_map`
            extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.8] # Optional
        )

Environment

  1. Please describe your Platform [Windows/MacOS/Linux]: Windows 10, WSL2
  2. Please show the Layout Parser version: 0.2.0

Error traceback When I manually end the script, this is the traceback

>>> model_PLN_b = lp.Detectron2LayoutModel(
...             config_path ='lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config', # In model catalog
...             label_map   ={0: "Text", 1: "Title", 2: "List", 3:"Table", 4:"Figure"}, # In model`label_map`
...             extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.4] # Optional
...         )
config.yaml?dl=1: 8.19kB [00:01, 5.99kB/s]
model_final.pth?dl=1: 0.00B [04:57, ?B/s]^C
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/bin/venv-wsl/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 124, in __init__
    self._create_model()
  File "/usr/bin/venv-wsl/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 149, in _create_model
    self.model = self._engine.DefaultPredictor(self.cfg)
  File "/usr/bin/venv-wsl/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 222, in __init__
    checkpointer.load(cfg.MODEL.WEIGHTS)
  File "/usr/bin/venv-wsl/lib/python3.8/site-packages/fvcore/common/checkpoint.py", line 140, in load
    path = self.path_manager.get_local_path(path)
  File "/usr/bin/venv-wsl/lib/python3.8/site-packages/iopath/common/file_io.py", line 1107, in get_local_path
    return handler._get_local_path(
  File "/usr/bin/venv-wsl/lib/python3.8/site-packages/iopath/common/file_io.py", line 766, in _get_local_path
    cached = download(path, dirname, filename=filename)
  File "/usr/bin/venv-wsl/lib/python3.8/site-packages/iopath/common/download.py", line 58, in download
    tmp, _ = request.urlretrieve(url, filename=tmp, reporthook=hook(t))
  File "/usr/lib/python3.8/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1360, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1317, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1230, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1276, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1225, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1004, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 944, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 1392, in connect
    super().connect()
  File "/usr/lib/python3.8/http/client.py", line 915, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)

** Extra comments Cancelling the script and re-running it eventually leads to a successful download.

I assume the issue is in how WSL2 handles network connections. Would it be possible to include a timeout and/or retry element to the download instruction? The first would at least indicate to the user that there is a network issue, the latter might fix the issue.

lolipopshock commented 3 years ago

Thanks for reporting the issue - downloading failure is always annoying! And your suggestions is great - let me try to figure out a solution for fixing this issue.