shinnytech / tqsdk-python

天勤量化开发包, 期货量化, 实时行情/历史数据/实盘交易
https://doc.shinnytech.com/tqsdk/latest
Apache License 2.0
3.63k stars 650 forks source link

期权链数据缺失 #482

Open SAMPLE-42 opened 5 months ago

SAMPLE-42 commented 5 months ago

DataDownloader下载器指定历史区间,选定’2015-01-01‘ --- 至今,下载所有合约时,下载结果中存在合约缺失(多数为郑商所数据和股指期权数据),请问是什么问题。总共缺失3000+,其中存在一些较近的时间的期权也缺失。使用DataDownloader下载器单独下载缺失合约数据也是空的,例如CZCE.SM403C5300

SAMPLE-42 commented 5 months ago
def fetch_data(self, contract_symbols, start_date, end_date, duration, update_date=False):
    """
        功能:获取指定合约,指定时间区间的历史数据
        Args:
            contract_symbols: 合约代码, list
            start_date: 下载的起始时间
            end_date: 下载的结束时间
            duration: k线级别
            update_date: 数据更新, bool
        return: 历史行情数据, dataframe
    """

    data_dir_path = os.path.join(self.root_dir, 'data', f'update_{update_date}')

    if not os.path.exists(data_dir_path):
        os.makedirs(data_dir_path)

    download_data_tasks = {}

    for symbol in tqdm(contract_symbols):

        print(f'当前下载{symbol}合约数据')

        data_save_file_name = f"{symbol}_data.csv"
        data_save_file_path = os.path.join(data_dir_path, data_save_file_name)

        download_data_tasks[symbol] = DataDownloader(api=self.api,
                                                     symbol_list=symbol,
                                                     dur_sec=duration,
                                                     start_dt=start_date,
                                                     end_dt=end_date,
                                                     csv_file_name=data_save_file_path,
                                                     )

    with closing(self.api):
        while not all([v.is_finished() for v in download_data_tasks.values()]):
            self.api.wait_update()
        print("progress: ", {k: ("%.2f%%" % v.get_progress()) for k, v in download_data_tasks.items()})

        self.logger.log(logging.INFO, f'{start_date}-----{end_date}数据全部下载完成')
        print(f'{start_date}-----{end_date}数据全部下载完成')

    time.sleep(10)
    # 检查是否有空的DataDownloader,如果有则删除相应文件

    for symbol, downloader in download_data_tasks.items():
        # 构建文件路径
        file_csv_path = os.path.join(data_dir_path, f"{symbol}_data.csv")

        if os.path.exists(file_csv_path):
            df = pd.read_csv(file_csv_path)

            if not df.empty:
                df = pd.read_csv(file_csv_path)
                df_copy = df.copy()
                # df_copy.columns = ['datetime', 'open', 'high', 'low', 'close', 'volume', 'open_oi', 'close_oi']
                df_copy['instrument_id'] = symbol
                df_copy.to_csv(file_csv_path)

            else:
                os.remove(file_csv_path)
                self.logger.log(logging.INFO, f"删除空文件: {symbol}")
                print(f"删除空文件: {symbol}")