Brain Tumor MRI Classification | Step 1 Dataset Exploration #8

[theiturhs]

8 These tasks are implemented:

1. Exploring class, class distribution, image size distribution for both training and testing dataset.
2. Performing necessary dimension reduction (RGB to Grayscale conversion)
3. Performing some Pre-processing tasks using Image processing techniques
4. Performing augmentation
5. Dataset uploaded over Kaggle. Please find it over here

Thank you

[SrijanShovit]

Output of this one not visible, just check one: ## Distribution of Image sizes by class in testing dataset

Rest, good work.

[theiturhs]

I have used Plotly for this. So if we are viewing the notebook in an environment like GitHub or a Jupyter Notebook viewer, they don't support interactive Plotly figures, so it is not visible. But if we execute the code, the output will be visible below the cell. Thus, I have pasted those results in Brain_Tumor_MRI_Dataset_Generation.md file.

I wanted to confirm whether this task is labelled to level 1 or level 2?

Thank you

[SrijanShovit]

Okk got it. You can proceed with the next step. I will created issue in evening. Just post it here, what you will be working on as next step.

[theiturhs]

Since we are done with Data Preprocessing and Augmentation, now we can go for

Model Selection: Considering using CNNs and exploring architectures such as VGG, ResNet, and to find the best-performing model.

Training and Evaluation: Then training the model and evaluating it's performance using metrics such as accuracy, F1 score, and confusion matrix.

[SrijanShovit]

1. Method of training: LOSO, Holdout, k-fold CV
2. Hyperparameter tuning : any libraries?
3. Generic code for model training, loss and acc plots, metrics calculation and CM
4. Play on different optims and activ fns All these can be done for any kind of model. Look for some research papers what's the result best achieved and how. Think of way to beat that.

[theiturhs]

Sure! Let me go through all these and will get back to you!

[theiturhs]

@SrijanShovit

Dividing this model training part, it can go in this way:

1. Cross-validation: For this, k-fold will be the better option looking on the size of dataset compared to LOO, as it can be computationally expensive.

2. Hyperparameter tuning: RandomizedSearchCV is often preferred over GridSearchCV, so I will go with RandomizedSearchCV.

3. Model training: Using cnn_learner with ResNet16 or ResNet18 or ResNet34 architecture (taking care of training time) from FastAI, using hyperparameter tuning libraries like scikit-learn, Optuna, or TensorFlow Tuner to optimize hyperparameters.

4. Performance metrics: confusion matrix, accuracy, recall, precision, F1 score, ROC curve

5. Curve plots: Plotting accuracy vs. epochs and loss vs. epochs (detecting issues like overfitting or underfitting)

6. Early stopping will be implemented during model training to prevent overfitting and reduce training time.

It would be better if I start training with smaller image sizes and gradually increase the image size during training to ensure if there is any training time constraint or hardware constraint. If it takes too long, I will need to change this approach. If there is something to add, I will look into that also. You can specify that over here.

Thanks!

[SrijanShovit]

@SrijanShovit

Dividing this model training part, it can go in this way:

1. Cross-validation: For this, k-fold will be the better option looking on the size of dataset compared to LOO, as it can be computationally expensive.
2. Hyperparameter tuning: RandomizedSearchCV is often preferred over GridSearchCV, so I will go with RandomizedSearchCV.
3. Model training: Using cnn_learner with ResNet16 or ResNet18 or ResNet34 architecture (taking care of training time) from FastAI, using hyperparameter tuning libraries like scikit-learn, Optuna, or TensorFlow Tuner to optimize hyperparameters.
4. Performance metrics: confusion matrix, accuracy, recall, precision, F1 score, ROC curve
5. Curve plots: Plotting accuracy vs. epochs and loss vs. epochs (detecting issues like overfitting or underfitting)
6. Early stopping will be implemented during model training to prevent overfitting and reduce training time.

It would be better if I start training with smaller image sizes and gradually increase the image size during training to ensure if there is any training time constraint or hardware constraint. If it takes too long, I will need to change this approach. If there is something to add, I will look into that also. You can specify that over here.

Thanks!

I guess it's multi-class classification. So we need not plot ROC graph just for the sake of plotting.

[SrijanShovit]

@SrijanShovit

Dividing this model training part, it can go in this way:

1. Cross-validation: For this, k-fold will be the better option looking on the size of dataset compared to LOO, as it can be computationally expensive.
2. Hyperparameter tuning: RandomizedSearchCV is often preferred over GridSearchCV, so I will go with RandomizedSearchCV.
3. Model training: Using cnn_learner with ResNet16 or ResNet18 or ResNet34 architecture (taking care of training time) from FastAI, using hyperparameter tuning libraries like scikit-learn, Optuna, or TensorFlow Tuner to optimize hyperparameters.
4. Performance metrics: confusion matrix, accuracy, recall, precision, F1 score, ROC curve
5. Curve plots: Plotting accuracy vs. epochs and loss vs. epochs (detecting issues like overfitting or underfitting)
6. Early stopping will be implemented during model training to prevent overfitting and reduce training time.

It would be better if I start training with smaller image sizes and gradually increase the image size during training to ensure if there is any training time constraint or hardware constraint. If it takes too long, I will need to change this approach. If there is something to add, I will look into that also. You can specify that over here.

Thanks!

yes batch size, image size, ram optimisation have to be taken care of

[SrijanShovit]

Also, you forgot activation fns and optimizers with each model, make sure to check AF: relu, selu, elu, mish, rest on your wish Opt: Lion, Adam, RMSprop, rest on your wish

[SrijanShovit]

Prepare a proper chart of procedure with the different experiments of all things you wanna try out. Each aspect will then be assigned as issue. Not all at once.

[theiturhs]

Also, you forgot activation fns and optimizers with each model, make sure to check AF: relu, selu, elu, mish, rest on your wish Opt: Lion, Adam, RMSprop, rest on your wish

Yes I will check this out also

[theiturhs]

Prepare a proper chart of procedure with the different experiments of all things you wanna try out. Each aspect will then be assigned as issue. Not all at once.

This is just an example of how can I start with.

I can start with Hyperparameter tuning where I will check which technique gives better result.

1. GridSearchCV
2. RandomizedSearchCV
3. Bayesian Optimization
4. HalvingGridSearchCV

Analysing with different libraries available:

1. Scikit-learn
2. KerasTuner
3. Optuna
4. Talos

If this is fine, I will check out different methods for each step and will share the proper flow chart.

[SrijanShovit]

yupp go ahead

[theiturhs]

@SrijanShovit

Here is the approach that I think can work

Choosing Cross Validation technique (sklearn library)

• Stratified K-Fold,
• K-Fold
• Repeated K-Fold
• LOOCV

Model Selection

• Simple CNN architecture
• FastAI architecture (VGG, ResNet)

Hyperparamter Tuning (scikit-learn, kerasTuner, Optuna)

• GridSearchCV
• RandomizedSearchCV
• Bayesian Optimization
• HalvingGridSearchCV

Model Training

• Training with tuned parameter
• Selecting optimizers (SGD, Adam, RMSprop, AdamW)

Activation Functions

• FastAI defaults
• Customize (example LeakyReLU)

Model Evaluation

• Confusion Matrix
• Precision
• Accuracy
• Recall
• F1 Score

For each issue, each method can be evaluated using small dataset first. And if the results are promising, then we can proceed to apply the methods to the whole dataset.

Thanks

[SrijanShovit]

One more thing, you can play with callbacks like Learning Rate scheduler, epochs, early stopping as well. Go ahead. Good job. Since all this would take time and efforts, so open issues yourself for a particular combination. If I approve it, it will have points for you accordingly.