WinError 10048 appearing on failed downloads, possible socket issue?

dartigen commented 6 years ago

[X] I think my problem is NOT with youtube-dl
[X] I've verified and i assure that I'm running youtube-dl-gui 0.4
[X] I assure that i am using the latest version of youtube-dl
[X] Searched bugtracker
[X] I've read the FAQs file

What is the purpose of your issue?

[X] Bug report

Full log output from one instance of this (it took a while for it to happen):

[01/08/18 05:37:47] [debug] System config: []
[01/08/18 05:37:47] [debug] User config: []
[01/08/18 05:37:47] [debug] Custom config: []
[01/08/18 05:37:47] [debug] Command-line args: ['--newline', '--restrict-filenames', '-o', 'F:\\Videos\\%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s', '-f', '18', '-v', '--hls-prefer-native', '--add-metadata', '-v', '--playlist-items', '1,2,3,5,6-8,12-30,32,34-36,39,41,43-46,49,52-54,56-63,65-69,71,72,74,76-82', 'https://www.youtube.com/playlist?list=PL7pGJQV-jlzBE67ZzVOT-c6uONv2-eu-r']
[01/08/18 05:37:47] [debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[01/08/18 05:37:47] [debug] youtube-dl version 2018.01.07
[01/08/18 05:37:47] [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.15063
[01/08/18 05:37:47] [debug] exe versions: ffmpeg 3.3.2, ffprobe 3.3.2
[01/08/18 05:37:47] [debug] Proxy map: {}
[01/08/18 05:37:47] WARNING: [youtube] ZWqG7nsa7es: Skipping DASH manifest: ExtractorError("Failed to download MPD manifest: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted> (caused by URLError(OSError(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted', None, 10048, None),))",)
[01/08/18 05:37:47] WARNING: [youtube] bvlWden6I9I: Skipping DASH manifest: ExtractorError("Failed to download MPD manifest: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted> (caused by URLError(OSError(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted', None, 10048, None),))",)
[01/08/18 05:37:47] WARNING: [youtube] auW48MWxgfI: Skipping DASH manifest: ExtractorError("Failed to download MPD manifest: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted> (caused by URLError(OSError(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted', None, 10048, None),))",)
[01/08/18 05:37:47] WARNING: [youtube] ynT4FnrC0OE: Skipping DASH manifest: ExtractorError("Failed to download MPD manifest: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted> (caused by URLError(OSError(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted', None, 10048, None),))",)
[01/08/18 05:37:47] WARNING: [youtube] 3a7KKihZSso: Skipping DASH manifest: ExtractorError("Failed to download MPD manifest: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted> (caused by URLError(OSError(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted', None, 10048, None),))",)
[01/08/18 05:37:47] WARNING: [youtube] 3_FypBXVkHU: Skipping DASH manifest: ExtractorError("Failed to download MPD manifest: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted> (caused by URLError(OSError(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted', None, 10048, None),))",)
[01/08/18 05:37:47] WARNING: [youtube] DQMBdh7ThHk: Skipping DASH manifest: ExtractorError("Failed to download MPD manifest: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted> (caused by URLError(OSError(10048, 'Only one usage of each socket address (protocol/network address/port) is normally permitted', None, 10048, None),))",)
[01/08/18 05:37:47] ERROR: unable to download video data: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted>
[01/08/18 05:37:47] Traceback (most recent call last):
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\urllib\request.py", line 1183, in do_open
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\http\client.py", line 1137, in request
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\http\client.py", line 1182, in _send_request
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\http\client.py", line 1133, in endheaders
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\http\client.py", line 963, in _send_output
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\http\client.py", line 898, in send
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\http\client.py", line 1279, in connect
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\http\client.py", line 871, in connect
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\socket.py", line 516, in create_connection
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\socket.py", line 507, in create_connection
[01/08/18 05:37:47] OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
[01/08/18 05:37:47] 
[01/08/18 05:37:47] During handling of the above exception, another exception occurred:
[01/08/18 05:37:47] 
[01/08/18 05:37:47] Traceback (most recent call last):
[01/08/18 05:37:47]   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp4k56gdgt\build\youtube_dl\YoutubeDL.py", line 1895, in process_info
[01/08/18 05:37:47]   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp4k56gdgt\build\youtube_dl\YoutubeDL.py", line 1834, in dl
[01/08/18 05:37:47]   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp4k56gdgt\build\youtube_dl\downloader\common.py", line 361, in download
[01/08/18 05:37:47]   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp4k56gdgt\build\youtube_dl\downloader\http.py", line 286, in real_download
[01/08/18 05:37:47]   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp4k56gdgt\build\youtube_dl\downloader\http.py", line 74, in establish_connection
[01/08/18 05:37:47]   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp4k56gdgt\build\youtube_dl\YoutubeDL.py", line 2198, in urlopen
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\urllib\request.py", line 464, in open
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\urllib\request.py", line 482, in _open
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
[01/08/18 05:37:47]   File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp4k56gdgt\build\youtube_dl\utils.py", line 1089, in https_open
[01/08/18 05:37:47]   File "C:\Python\Python34\lib\urllib\request.py", line 1185, in do_open
[01/08/18 05:37:47] urllib.error.URLError: <urlopen error [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted>
[01/08/18 05:37:47]

What operating system do you use ? Windows 10

List of actions to perform to reproduce the problem:

Pick a video or playlist to download.
Try to download it.
Sometimes, this happens.

What is the expected behaviour ?

The video file should download in full, and once finished the Status should read 'Completed'.

What happens instead ?

The video file downloads, but that Status reads 'Warning'.

Or, the video file doesn't download at all (sometimes a .part file is found in the output directory), and the Status reads 'Error'. (Reattempting the download works, but the Status then reads 'Already Downloaded'.

I've only started to experience this recently (as of about November 2017), but I have experienced it on multiple devices (two PCs, as well as a laptop, all running Windows 10). One of the PCs was on a very, very good connection, and the laptop has moved from that network to less-good connections, which makes me think it's not network-related.

From some Googling, this seems to be an issue with Python multithreading and network connections (based on this thread and a few others: https://stackoverflow.com/questions/23596009/python-server-client-program-error-oserror-winerror-10048 ). I'm not sure if that would be part of youtube-dlg or youtube-dl though.

I'm not sure how to fix it, other than maybe installing Python independently of youtube-dlg, or reinstalling youtube-dlg. I couldn't find anything else about it in the issues tracker about that error message specifically.

MrS0m30n3 commented 6 years ago

That's a youtube-dl issue, but in some degree youtube-dl-gui is also responsible for the error due to the multithreading.

On the youtube-dl's side, in file YoutubeDL.py#L2198 they never close down the open socket hence leaking sockets, which in turn results to TCP/IP ports exhaustion.

Under normal circumstances this should not be a problem since Windows allocate ports 1025 -> 5000 for client usage (according to this), but for some reason on your system all ports are exhausted (too many videos downloading? or too many apps that use sockets?).

What you can do:

Open a new issue at the youtube-dl's repo (you can refer to this issue if you want).
If you have too many applications that use connections close some of them.
Increase the available client ports on Windows (i believe you can achieve this through the Windows registry).
Keep the threads number on the youtube-dl-gui's side low (see FAQs file "Is there an option to change the maximum parallel downloads?").

NOTE

If you have changed the workers_number in your youtube-dl-gui's settings file to some value like 100+ use a smaller value (default is 3).

dartigen commented 6 years ago

I'll refer it over to youtube-dl and see what they can do.

I'm not sure if there's a way to check how many or what applications are using connections, but it did happen more often when I had stuff like Google Drive Sync running alongside it, yeah. I did find it very strange though that it still sometimes happened when I only had one video downloading though - I guess that was the other applications getting in the way.

Thanks for clearing that up!

EDIT: I just checked settings.json, and workers_number is at the default 3 - so I've turned it down to 1 to see if that fixes it until I can figure out what's eating all the ports.

MrS0m30n3 commented 6 years ago

The easiest way to view open connections if you don't know how to use the command line is TCPView, else you can use netstat from the command line.

Also, if you open a new issue at the youtube-dl's repo please link to this issue.

MrS0m30n3 / youtube-dl-gui