josw123 / dart-fss

한국 금융감독원에서 운영하는 다트(Dart) 시스템 크롤링을 위한 라이브러리
https://github.com/josw123/dart-fss
MIT License
319 stars 111 forks source link

ConnectionError(err, request=request) #167

Closed 0x68756d8616e6f69664 closed 4 months ago

0x68756d8616e6f69664 commented 4 months ago

안녕하세요. 아래와 같은 실행메세지로 오류가 뜨는데요? 무선적으로 뜨는 것으로 보아 코드의 오류는 아닌 것 같은데 이번주말에 유달리 많이 떠서 이렇게 문의드립니다. (기존에는 다시 실행하면 원하는대로 동작했습니다.)

어떻게 해결하면 될까요? (기존처럼 반복실행하는 것이 유일한 해결책일까요?)

미리 감사드립니다. dart_fss v0.4.8 입니다.

Traceback (most recent call last): File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request httplib_response = conn.getresponse() File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\http\client.py", line 1348, in getresponse response.begin() File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\http\client.py", line 316, in begin version, status, reason = self._read_status() File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\http\client.py", line 285, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\requests\adapters.py", line 489, in send resp = conn.urlopen( File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen retries = retries.increment( File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\util\retry.py", line 550, in increment raise six.reraise(type(error), error, _stacktrace) File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\packages\six.py", line 769, in reraise raise value.with_traceback(tb) File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request httplib_response = conn.getresponse() File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\http\client.py", line 1348, in getresponse response.begin() File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\http\client.py", line 316, in begin version, status, reason = self._read_status() File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\http\client.py", line 285, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:/Users/실행폴더/실행파일.py", line 954, in samsung_num = dart.api.info.stock_totqy_sttus(samsung.corp_code, year_p, type_y, File "C:\Users\실행폴더\Python\Python38\site-packages\dart_fss\api\info\stock_totqy_sttus.py", line 32, in stock_totqy_sttus return api_request( File "C:\Users\실행폴더\Python\Python38\site-packages\dart_fss\api\helper.py", line 112, in api_request resp = request.get(url=url, payload=payload) File "C:\Users\실행폴더\Python\Python38\site-packages\dart_fss\utils\request.py", line 173, in get return self.request(url=url, method='GET', payload=payload, referer=referer, stream=stream, timeout=timeout) File "C:\Users\실행폴더\Python\Python38\site-packages\dart_fss\utils\request.py", line 143, in request resp = self.s.send(prepped, stream=stream, timeout=timeout) File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\requests\sessions.py", line 701, in send r = adapter.send(request, **kwargs) File "C:\ProgramData\Anaconda3\envs\실행폴더\lib\site-packages\requests\adapters.py", line 547, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

josw123 commented 4 months ago

안녕하세요.

DART 쪽에서 크롤링 방지를 위해서 서버단에서 연결을 끊는 것으로 생각됩니다. 다음과 같이 딜레이를 좀더 길게 주시거나

import dart_fss as dart

# DART_FSS Request Delay 1s로 변경
dart.utils.request.set_delay(1)

다음과 같이 Useragent를 일정주기로 변경해서 시도해보시는 것을 추천드립니다.

import dart_fss as dart

# User-Agent 강제 변경
dart.utils.request.update_user_agent(force=True)
0x68756d8616e6f69664 commented 4 months ago

알려주신 방법으로 한 결과 오류없이 진행이 되는 것으로 보아 말씀하신 이슈로 확인됩니다. 가이드라인 감사합니다 ^^!.