sassoftware / python-dlpy

The SAS Deep Learning Python (DLPy) package provides the high-level Python APIs to deep learning methods in SAS Visual Data Mining and Machine Learning. It allows users to build deep learning models using friendly Keras-like APIs.
Apache License 2.0
224 stars 131 forks source link

SimpleObjectDetection required column does not exist in table #403

Closed crack3n-collab closed 7 months ago

crack3n-collab commented 7 months ago

So in the simpleObjectDetection example when the Soccer_Images_416.sashdat is converted to a CAStable. It has this amount of columns image later the soccer data is then split into trainset and testset and there's no issues.

So I tried using my own data and these are the columns image it's drastically different from soccer data and so I had trouble splitting it into trainset and testset in the later part of the code. image I understand my data columns are incorrect but is there a correct way to generate those missing required columns? https://github.com/sassoftware/python-dlpy/blob/master/examples/object_detection/SimpleObjectDetection.ipynb

crack3n-collab commented 7 months ago

I generated my CASTable using code from https://github.com/michaelgorkow/SAS_DeepLearning/blob/master/Face_Mask_Detection/image_augmentation_object_detection.ipynb . image

I do know that in simpleOjectDetection, there's also a PrepareObjectDetectionTable.ipynb but it seems to require xml data which I don't have.

dxq77dxq commented 7 months ago

Please add "im_table=False" to your two_way_split().

crack3n-collab commented 7 months ago

im_table = false sets the CASTables to be returned with all columns so they're not imagetables. In the later part of the code, it says the image table is created from the training table. But it still able to create the table so how does that work? image

dxq77dxq commented 7 months ago

Cells 10-14 are for summary and visualization. You don't really need them for training and scoring. You can directly what you get from two_way_split() for traning.

crack3n-collab commented 7 months ago

What are you referring to when you say cells 10-14?

dxq77dxq commented 7 months ago

https://github.com/sassoftware/python-dlpy/blob/master/examples/object_detection/SimpleObjectDetection.ipynb

crack3n-collab commented 7 months ago

Oh ok, I understand what you're referring to now. It's indeed for summary and visualization. So error message won't affect much, would it? Since in the end the tables are still generated

dxq77dxq commented 7 months ago

The error is from here: https://github.com/sassoftware/python-dlpy/blob/master/dlpy/images.py#L207

I would suggest you skip it and move to training directly.

crack3n-collab commented 7 months ago

So the split of the table into trainset and testset can be skipped?

dxq77dxq commented 7 months ago

I'll suggest you keep it so you can use different datasets for training and scoring. Please let me know if you questions are resolved and I'll close the issue.

crack3n-collab commented 7 months ago

I have run through the whole code successfully and you can close this issue. Thank you very much for the help.