a bug when download a folder contains folder

chengyu0910 commented 1 year ago

Provide environment information

ubantu==20.04 python==3.8 gdown==4.5.4

What OS are you using?

ubantu

Describe the Bug

This is the command and errors:

gdown --folder https://drive.google.com/drive/folders/1Zk6TaSJPbpnqbz1w4kfhkKFCEzQbjfp_ -c --proxy http://127.0.0.1:10809
Retrieving folder list
Processing file 1DFMUkL-dc-j8-fibcJIq-46Xoq_bFoO9 AFLink_epoch20.pth
Processing file 1MduGC_JFXf3yGBlwwYiRSw1CZv9kNEqh MOT17_ECC_test.json
Processing file 1-6qmXRZ9GD1RUaMZ-y_f55M4_9jJ_1Sn MOT17_ECC_train.json
Processing file 1I1Sk6a1i377UhXykqN9jZmbvJLc6izbz MOT17_ECC_val.json
Retrieving folder 16oC71eiWhDe5wJaYOyWkgE3h-zq2WGgi MOT17_test_YOLOX+BoT
Processing file 1vuBB7L9o8EJGXAIjrGhcSg1e80k69j49 MOT17-01-FRCNN.npy
Processing file 1DSPjcaL3rzTTlCoQ5z1dBvJcoFy4JRRM MOT17-03-FRCNN.npy
Processing file 1jPk5drcSqZbUsQVWoH7pbK9sAJaesRoL MOT17-06-FRCNN.npy
Processing file 1XbeCeEcKHDLmAGpiItWg9UKYKzjAvenw MOT17-07-FRCNN.npy
Processing file 1RhfsnX9AX49r8MHO0MTvxxUAIVISzjhU MOT17-08-FRCNN.npy
Processing file 1-Hneg3NT5SfcrVY2jstRtigXTE3qAxqH MOT17-12-FRCNN.npy
Processing file 1B_RY_Ey3soFw6wNn95gJipR7YwtE11Ak MOT17-14-FRCNN.npy
Retrieving folder 1nWFbqcehQeT-v0DcTTtSSRR9iSf6fvxo MOT17_test_YOLOX+simpleCNN
Processing file 1j2ik0A3Si72-3riXcM1dkovW1BMPOFvX MOT17-01-FRCNN.npy
Processing file 1s9lI6XU3GCARH8q4FsgB7hbjfy6uwari MOT17-03-FRCNN.npy
Processing file 181Er5fl-nKLqv6pQs3qVnLDyBoEs_NrW MOT17-06-FRCNN.npy
Processing file 1H0EeDPag3fG1PIjH8Rw0sjLw7BNdjXGu MOT17-07-FRCNN.npy
Processing file 14G0RP8GWhgmtpLykQ7oZNbZSK8Mv9P23 MOT17-08-FRCNN.npy
Processing file 1Ot1OnmDMnqk-pMyJ5zgD1oOgizSwv9d7 MOT17-12-FRCNN.npy
Processing file 1cpo_m11ZJEAkErDgvKB3jpf37DHIxRxG MOT17-14-FRCNN.npy
Retrieving folder 1hBnln_EMwOqUWK6CwGp5P_s9vFm1MdQw MOT17_trainval_GT_for_AFLink
Retrieving folder 1mI169LUO5EQ537Xhw4uql833NRVqJnmN MOT17-02-FRCNN
Processing file 1g0H_PKQmjkP0QwOIMuVk6McQy0ks3LQz gt_train_half.txt
Processing file 1bMTXtX-kUmLwILqksoDwydUoMqCGuj-W gt_val_half.txt
Retrieving folder 1bYIdPLml0fCha0WIvL373Yi_ZV6-hRMo MOT17-04-FRCNN
Processing file 1iDUFzFAp4QnE7UA-lQWOuqimzkmhd2hF gt_train_half.txt
Processing file 1Nt6E7SWSp1kUnnALbot10bzbi7HIui8D gt_val_half.txt
Retrieving folder 1gqEfoffOeE0FDMHBzaZhNc18hJCH5y2a MOT17-05-FRCNN
Processing file 1BvGs10wuH-4OovHPUlgJD8_RMuGu8ghp gt_train_half.txt
Processing file 1tTusJEjNGiVCHtcnOgT4k31aTmFYhAVL gt_val_half.txt
Retrieving folder 1XCiMT0xQIAuo9Gt281EbngJCe4gifaqw MOT17-09-FRCNN
Processing file 1MfworAXa-1UB6Yb74KFj1SWB7VYylb7v gt_train_half.txt
Processing file 1uuSN-V7JShFO-qt4ISj-56VN5ZFmAM_W gt_val_half.txt
Retrieving folder 1qDIuLH3qrHILmdASTpajIgAlM5w1HsSE MOT17-10-FRCNN
Processing file 1O37uV0l1NTAVHDqt_ALtZvJ72A1U3C4H gt_train_half.txt
Processing file 1FsYXD5sHNwb8J2BlIoUwK6zY0rCJks79 gt_val_half.txt
Retrieving folder 1YC4S1HYlpZZuIziG_YC_w7xMC1yTQNsw MOT17-11-FRCNN
Processing file 1TbCKygns05YmM5fCPKPQwooMt9hA9Rsm gt_train_half.txt
Processing file 16PSb3X_Fokgv4LpzhB1siberCAuki8cB gt_val_half.txt
Retrieving folder 1vHfl0Yim6YQ5QuL3X_Mz4MVYrzrpo_Rd MOT17-13-FRCNN
Processing file 1N-G_yDC6b_V85OJ7NdXNNu8d_GGvYhg3 gt_train_half.txt
Processing file 1z18lnQVKNKcO23FHFViZuA7Tv6EOOdE3 gt_val_half.txt
Retrieving folder 1wdG-vJOMGynf5QjqEa1jNZNYtz0PQqPu MOT17_val_GT_for_TrackEval
Processing file 1-QNiquYRmrFpfAX1aSlOvqwdvE5JdTge MOT17-02-FRCNN.txt
Processing file 1FN5vBapOa2QNxlRfS15QuIAz5D6iZ_VZ MOT17-04-FRCNN.txt
Processing file 1thvC1PFJe8FCDwcxQjjraylvH7Gf0Le5 MOT17-05-FRCNN.txt
Processing file 1XNP1aFIDdNWZBx4EG7ScZPnsWI18Zpq7 MOT17-09-FRCNN.txt
Processing file 13N4t_Wha3NXhDiqAQ6iCpNsTF8qTP3IW MOT17-10-FRCNN.txt
Processing file 1uDyHXcEHR8bcmgCQHfQprIA9N_ASjAcM MOT17-11-FRCNN.txt
Processing file 1cZaPxzA3cM3a8tMLvkYgNa0kbCCQcukW MOT17-13-FRCNN.txt
Retrieving folder 1zzzUROXYXt8NjxO1WUcwSzqD-nn7rPNr MOT17_val_YOLOX+BoT
Processing file 1dQuT24ysQRwm6JqI_jOwh0Gyg0LrNnyt MOT17-02-FRCNN.npy
Processing file 1TlIAt2-QCAv3zI3zzwgA0COnF-p5yfFV MOT17-04-FRCNN.npy
Processing file 1uBBu3rzd8ZbKklWA--MFzr8_6HAHkQgu MOT17-05-FRCNN.npy
Processing file 1nuopeCT6UqAKA1UgHknTuBmCWTXltBIO MOT17-09-FRCNN.npy
Processing file 1uAxtHFNoLKLKLuK6KUCbTkRf_o39Zt7k MOT17-10-FRCNN.npy
Processing file 1q5c75avA174L59S9fHilrwVR0lHdeWWT MOT17-11-FRCNN.npy
Processing file 14h8qGbHQNCiY-SXQgPzDRJjFtuwciH5z MOT17-13-FRCNN.npy
Retrieving folder 18qjvGOJK37W5PqfvLB0PcqOum6J-PlEy MOT17_val_YOLOX+simpleCNN
Processing file 1X1FjGHdFL_wERgFfgBrT2rwMdGUTb45g MOT17-02-FRCNN.npy
Processing file 1V4_51DVs53WAbRMC_4-GHFE7SXXAMqTC MOT17-04-FRCNN.npy
Processing file 1HBFAoyWbhcz9CBz_slmGAltV-CHpPQon MOT17-05-FRCNN.npy
Processing file 1Q-U4Cx4ISF4qrP7_Atfo7lSMjfb6fqhI MOT17-09-FRCNN.npy
Processing file 185cjW-T69KTLQZze-_vGjYjZzCAUnJec MOT17-10-FRCNN.npy
Processing file 1FredouZzYGbqnIh8piNa90avQJITA0rl MOT17-11-FRCNN.npy
Processing file 1Mm8pBvMsTf79OysaHaPB4b64P-97BAhZ MOT17-13-FRCNN.npy
Processing file 1qkbWgOQD4QLrsB5EW8GTQMZKYInxhCYZ MOT20_ECC_test.json
Processing file 1algMx4Ioc85PQ3L1ZCmU9z-vvH_UvE76 MOT20_ECC_train.json
Retrieving folder 1YEFR7Q8RcPsY8h0k6KPoSX-lkQSXVg6v MOT20_test_YOLOX+BoT
Processing file 1E1GVaH3qEYNdI76zvWh4-XzheRUJgI-M MOT20-04.npy
Processing file 1d4kc9qx83KQqzuoO33pMzeUV4IuP9Ppd MOT20-06.npy
Processing file 1AyBt7p9M0najd8jaI2ALMlianM1IMbbK MOT20-07.npy
Processing file 1Y8OhIimh8ZpbvOMuvrFT6k7SqHFQMwoD MOT20-08.npy
Retrieving folder 18tS-IMonPtoeg1dRslL_gpWvCcg6yEzQ MOT20_test_YOLOX+simpleCNN
Processing file 1Tw3cBu3y-kIqYdvgVqJRQ4cxZQbSW6Sx MOT20-04.npy
Processing file 1S4WY5t0JbzjBP71sqr6tBzQ7Ee1wQ3gP MOT20-06.npy
Processing file 13UUL4xWucP2zuHkKAJzQX3g8y1w5GfZz MOT20-07.npy
Processing file 1IciJyf4DIolHvvPAmBfVQXoU_UnU1xEP MOT20-08.npy
Retrieving folder list completed
Building directory structure
Building directory structure completed
Using proxy: http://127.0.0.1:10809
Downloading...
From: https://drive.google.com/uc?id=1DFMUkL-dc-j8-fibcJIq-46Xoq_bFoO9
To: /home/chengyu/StrongSORT/AFLink_epoch20.pth
100%|█████████████████████████████████████████████████████████████████████████████████████| 4.35M/4.35M [00:03<00:00, 1.39MB/s]
Using proxy: http://127.0.0.1:10809
Downloading...
From: https://drive.google.com/uc?id=1MduGC_JFXf3yGBlwwYiRSw1CZv9kNEqh
To: /home/chengyu/StrongSORT/MOT17_ECC_test.json
100%|███████████████████████████████████████████████████████████████████████████████████████| 878k/878k [00:00<00:00, 1.24MB/s]
Using proxy: http://127.0.0.1:10809
Downloading...
From: https://drive.google.com/uc?id=1-6qmXRZ9GD1RUaMZ-y_f55M4_9jJ_1Sn
To: /home/chengyu/StrongSORT/MOT17_ECC_train.json
100%|███████████████████████████████████████████████████████████████████████████████████████| 797k/797k [00:00<00:00, 1.14MB/s]
Using proxy: http://127.0.0.1:10809
Downloading...
From: https://drive.google.com/uc?id=1I1Sk6a1i377UhXykqN9jZmbvJLc6izbz
To: /home/chengyu/StrongSORT/MOT17_ECC_val.json
100%|███████████████████████████████████████████████████████████████████████████████████████| 398k/398k [00:00<00:00, 1.03MB/s]
Using proxy: http://127.0.0.1:10809
Traceback (most recent call last):
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 700, in urlopen
    self._prepare_proxy(conn)
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 996, in _prepare_proxy
    conn.connect()
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/urllib3/connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/chengyu/miniconda3/lib/python3.9/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/home/chengyu/miniconda3/lib/python3.9/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/home/chengyu/miniconda3/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1123)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/home/chengyu/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='doc-00-as-docs.googleusercontent.com', port=443): Max retries exceeded with url: /docs/securesc/3hlpt6hsed8d1a1k2n8qrarebbm35kl5/e4d81gjhpn88o342ortdseugd8qb6vrk/1669820850000/11236608090961097174/17047942503052043527Z/1vuBB7L9o8EJGXAIjrGhcSg1e80k69j49?uuid=224dfc29-ec91-4c10-b4c6-a05ff8d2014a (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1123)')))

Description: If a folder contains folders, SSL error occurs when the folder in folder(MOT17_test_YOLOX+BoT) start downloading.

Expected Behavior

No response

To Reproduce

use the command: gdown --folder https://drive.google.com/drive/folders/1Zk6TaSJPbpnqbz1w4kfhkKFCEzQbjfp_ -c --proxy http://127.0.0.1:10809

I use the proxy because I'm in China. You can test without proxy.

wkentaro commented 1 year ago

Hmm, it is stochastic?

doveppp commented 1 year ago

i have the same question. the process stuck at "Retrieving folder list"

doveppp commented 1 year ago

i have the same question. the process stuck at "Retrieving folder list"

It looks like proxy not used when "Retrieving folder list"

wkentaro commented 1 year ago

@doveppp thanks for the report. Can you try with the latest version?

pip install gdown==4.6.4 --no-cache-dir

doveppp commented 1 year ago

@doveppp thanks for the report. Can you try with the latest version?
pip install gdown==4.6.4 --no-cache-dir

It Work, thanks

wkentaro / gdown