RuntimeError: Unable to create internal loader object

[pantherso48]

Hi guys,

I am trying to load training data to my model and I am getting a RuntimeError: Unable to create internal loader object. The config params that I am using are: config = { 'manifest_filename': 'data/cifar10/train-index.csv', # CSV manifest of data 'manifest_root': 'data/cifar10', # root data directory 'image': {'height': 224, 'width': 224, # output image size 'scale': [0.875, 0.875], # random scaling of image before cropping 'flip_enable': True}, # randomly flip image 'type': 'image,label', 'batch_size': be.bsz, # batch size 'etl': [ {'type': 'image', 'width': 224, 'height': 224, 'channels':9}, {'type': 'label', 'binary': False} ] }

When I run the Aeon data loader with this line of code: train_set = AeonDataLoader(config, be)

I get the following error: Unable to create internal loader object: config element {image: "data/cifar10/train-index.csv"} is not understood

The error is pointing to the aeon_shim.py file, is there a property in my config that is depricated or is missing?

Thanks!

[wei-v-wang]

If it is cifar10 data, can you please check whether that worked for cifar10_msra.py under examples/ directory? If "python cifar10_msra.py" does not work, it might be data ingestion issue. Otherwise, can you incrementally transform cifar10_msra.py to your model (leaving the data loading part intact)?

[pantherso48]

Which cifar example? I am not seeing a cifar10_msra.py file.

[wei-v-wang]

My bad. Under cifar10_msra folder, there is data.py, train.py, README.md

Please run "python data.py --out_dir ." (data ingestion, which produces csv files etc under "." current directory ) You will see output like: DISPLAY:neon:Downloading file: ./cifar-10-python.tar.gz Download Progress |██████████████████████████████████████████████████| DISPLAY:neon:Download Complete 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50000/50000 [00:51<00:00, 974.63it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:09<00:00, 1103.53it/s] Manifest files written to: ./train-index.csv ./val-index.csv

Then you can run the following: under the same directory

" python train.py -b mkl"

You will see output like these: Epoch 0 [Train |█ | 61/782 batches, 1.78 cost.....

[pantherso48]

Ok that is running now, so since this is training the model does that train.py script replace this line of code?

train_set = AeonDataLoader(config, be)

[wei-v-wang]

I think the data.py and several lines of train.py (see below)

train = make_train_loader(args.manifest['train'], args.manifest_root, model.be, args.subset_pct, random_seed) test = make_validation_loader(args.manifest['val'], args.manifest_root, model.be, args.subset_pct)

would achieve what you want "train_set = AeonDataLoader(config, be)" to achieve.

Actually you can open data.py and check how was make_train_loader implemented to load the dataset.

[pantherso48]

Okay that makes sense, is there a list of the datasets Neon provides? I see MNIST and cifar10 but was curious if there are others like cats and dogs?

[wei-v-wang]

There is imagenet folder under examples/imagenet There is also LSUN dataset for GANS