search

soap117 / DeepRule

BSD 3-Clause "New" or "Revised" License
128 stars 44 forks source link

KeyError: Pretrain #37

Open Cxmorifolium opened 3 months ago

Cxmorifolium commented 3 months ago

Hello. I'm hoping I'm overlooking something obvious from this repo. I'm trying to run:

python train_chart.py --cfg_file CornerNetLine --data_dir "linedata(1028)"

But the error I'm receiving is:

(DeepRule) PS C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR> python train_chart.py --cfg_file CornerNetLine --data_dir "linedata(1028)" {'system': {'snapshot_name': 'CornerNetLine', 'dataset': 'Line', 'batch_size': 26, 'sampling_function': 'kp_detection', 'train_split': 'trainchart', 'val_split': 'valchart', 'test_split': 'testchart', 'learning_rate': 0.00025, 'decay_rate': 10, 'val_iter': 100, 'opt_algo': 'adam', 'prefetch_size': 5, 'max_iter': 50000, 'stepsize': 45000, 'snapshot': 5000, 'chunk_sizes': [5, 7, 7, 7], 'data_dir': '$AZUREML_DATAREFERENCE_workspaceblobstore/chartdata(0610)/', 'cache_dir': './cache_pure/', 'display': 5}, 'db': {'rand_scale_min': 0.6, 'rand_scale_max': 1.4, 'rand_scale_step': 0.1, 'rand_scales': None, 'rand_crop': True, 'rand_color': True, 'border': 128, 'gaussian_bump': True, 'gaussian_iou': 0.3, 'input_size': [511, 511], 'output_sizes': [[128, 128]], 'test_scales': [1], 'top_k': 200, 'categories': 1, 'ae_threshold': 0.5, 'nms_threshold': 0.5, 'max_per_image': 100, 'annotations': 'line/annotations/instancesLine(1023)_train2019.json'}} ['cache', 'line'] loading all datasets... using 1 threads loading from cache file: linedata(1028)\cache\line_train2019.pkl loading annotations into memory... Path to file: C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\linedata(1028)\line\annotations\instancesLine(1023)_train2019.json Done (t=3.71s) creating index... index created! loading from cache file: linedata(1028)\cache\line_val2019.pkl loading annotations into memory... Path to file: C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\linedata(1028)\line\annotations\instancesLine(1023)_val2019.json Done (t=0.12s) creating index... index created! system config... {'batch_size': 26, 'cache_dir': 'linedata(1028)\cache', 'chunk_sizes': [5, 7, 7, 7], 'config_dir': 'config', 'data_dir': 'linedata(1028)', 'data_rng': RandomState(MT19937) at 0x11108D6D140, 'dataset': 'Line', 'decay_rate': 10, 'display': 5, 'learning_rate': 0.00025, 'max_iter': 50000, 'nnet_rng': RandomState(MT19937) at 0x11108D6D240, 'opt_algo': 'adam', 'prefetch_size': 5, 'result_dir': 'results', 'sampling_function': 'kp_detection', 'snapshot': 5000, 'snapshot_name': 'CornerNetLine', 'stepsize': 45000, 'tar_data_dir': 'cls', 'test_split': 'testchart', 'train_split': 'trainchart', 'val_iter': 100, 'val_split': 'valchart'} db config... {'ae_threshold': 0.5, 'border': 128, 'categories': 1, 'data_aug': True, 'gaussian_bump': True, 'gaussian_iou': 0.3, 'gaussian_radius': -1, 'input_size': [511, 511], 'lighting': True, 'max_per_image': 100, 'merge_bbox': False, 'nms_algorithm': 'exp_soft_nms', 'nms_kernel': 3, 'nms_threshold': 0.5, 'output_sizes': [[128, 128]], 'rand_color': True, 'rand_crop': True, 'rand_pushes': False, 'rand_samples': False, 'rand_scale_max': 1.4, 'rand_scale_min': 0.6, 'rand_scale_step': 0.1, 'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]), 'special_crop': False, 'test_scales': [1], 'top_k': 200, 'weight_exp': 8} len of db: 116745 Traceback (most recent call last): File "C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\train_chart.py", line 206, in train(training_dbs, validation_db, args.start_iter) File "C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\train_chart.py", line 76, in train pretrained_model = system_configs.pretrain ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\config.py", line 99, in pretrain return self._configs["pretrain"]


KeyError: 'pretrain'

My inkling is that it's not loading a pretrained model in config, however if that's the case, I have no idea which file is the pretrained file in this repo. Or when I go to comment it out to run without the pretrained parameters, it'll give me the same error as well. I'm a bit new to all of this so any help is much appreciate it.
soap117 commented 3 months ago

See the dataset link, included.

Cxmorifolium commented 3 months ago

See the dataset link, included.

So I have that included in the directory. It's loading the line(1028) (or whatever the actual folder name is). Then to clarify, it's the one under chart pipeline model that I'm probably not loading correctly perhaps? Thanks.

soap117 commented 3 months ago

You can debug to track the system_config instance. I am not sure, the default setting will set that para to None. Probably it is because you loaded some cached data on training mode, which is not expected.

Junyu Luo, Informatics Ph.D. Student The Pennsylvania State University

From: TheNoliOliandPea @.> Sent: Sunday, June 9, 2024 1:43 To: soap117/DeepRule @.> Cc: Luo, Junyu @.>; Comment @.> Subject: Re: [soap117/DeepRule] KeyError: Pretrain (Issue #37)

See the dataset link, included.

So I have that included in the directory. It's loading the line(1028) (or whatever the actual folder name is). Then to clarify, it's the one under chart pipeline model that I'm probably not loading correctly perhaps? Thanks.

— Reply to this email directly, view it on GitHubhttps://github.com/soap117/DeepRule/issues/37#issuecomment-2156331788, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGXQFDWMQZWPMCNC3ULK65TZGPTR7AVCNFSM6AAAAABI5UGRNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJWGMZTCNZYHA. You are receiving this because you commented.Message ID: @.***>

Cxmorifolium commented 3 months ago

I see! I'll check there, thank you for pointing that out.