Open hjafar opened 2 years ago
MarvinLer
commented
2 years ago Hi Hussain,
Seems like no data were fetched at some point. Hard to tell without the whole stack of logs and errors, could you copy/paste the complete output from your second command please (the python -m code.training ...)? In particular, I would need the logs from the program (that includes the logs from the fetched data) and the full error stack (indicating where the ValueError was raised).
Best, ML
hjafar
commented
2 years ago Dear Marvin,
Thank you for your response. Kindly find the following command with the complete run output.
$ python -m code.training --source-slides-folder '/home/hussain/Documents/breast_cancer/manifest_test' --alpha 0.1 --beta 0. --max-bag-size 100 --no-download
INFO Hyper parameters
INFO { 'alpha': 0.1,
'beta': 0.0,
'cuda': False,
'dataset_max_size': None,
'do_download': False,
'early_stopping_patience': 10,
'gdc_executable': None,
'input_data_folder': './data',
'learning_rate': 0.0005,
'manifest_file': None,
'max_bag_size': 100,
'models_save_folder': './saved_models/',
'n_epochs': 20,
'n_timesteps_per_epoch': None,
'save_model': True,
'save_model_timesteps': 5,
'seed': 123,
'source_slides_folder': '/home/hussain/Documents/breast_cancer/manifest_test',
'test_size': 0.15,
'underlying_model_load_from': None,
'underlying_model_pretrained': False,
'underlying_model_type': 'resnet18',
'val_size': 0.1,
'verbose': False,
'weight_decay': 0.0001,
'with_data_augmentation': True,
'with_tensorboard': True}
INFO Data_processing control parameters:
INFO do_download: False
INFO gdc_executable: None
INFO manifest_filepath: None
INFO source_slides_folder: /home/hussain/Documents/breast_cancer/manifest_test
INFO Meta-parameters:
INFO desired_magnification 20
INFO tile_width 224
INFO expected_tile_shape (224, 224, 3)
INFO background_threshold 0.75
INFO background_pixel_value 220
INFO Performing no download from TCGA as requested; listing source slides from /home/hussain/Documents/breast_cancer/manifest_test
INFO Slides already tiled at magnification 20 -> skipping
INFO Extracting labels...
Traceback (most recent call last):
File "/home/hussain/anaconda3/envs/tcga/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/hussain/anaconda3/envs/tcga/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/hussain/Documents/breast_cancer/tools/tcga_segmentation/code/training.py", line 343, in
The manifest_test folder contains the following list files/folders: Folder: downloaded_slides Folder: filtered_tiles file: gdc_manifest_20211111_062451.txt file: has_been_downloaded file: has_been_moved_and_filtered file: has_been_tiled_mag20
Best, Hussain
hjafar
commented
2 years ago Dear Marvin, This is kind reminder that I am still awaiting for your response regarding the above error. Best regards, Hussain
RorryB
commented
9 months ago Hey @hjafar, if you're still looking at this. I also got that error and the issues was just that I had a newline at the end of the manifest that was reading in an empty line as the final entry
Dear Marvin,
I would like to use your tcga_segmentation tool for tumor tissue prediction of primary breast cancer. I have finished the preprocessing step without any issues. I used the following command in this step:
python -m code.data_processing.main --gdc-executable '/home/hussain/Documents/breast_cancer/gdc-client_v1.6.1_Ubuntu_x64/gdc-client' --manifest '/home/hussain/Documents/breast_cancer/manifest_test/gdc_manifest_20211111_062451.txt' '/home/hussain/Documents/breast_cancer/manifest_test'The manifest file above contains 10 tumor and 10 normal WSIs in TCGA portal. All images were downloaded and patches, labels and summary text files were created for each WSI as a result of the above command.
However, I then ran the training step as follows:
python -m code.training --source-slides-folder '/home/hussain/Documents/breast_cancer/manifest_test' --alpha 0.1 --beta 0. --max-bag-size 100 --no-downloadThe above command fails and throws the following error message:
ValueError: not enough values to unpack (expected 4, got 0)Would you please explain why I am getting this error and what could have gone wrong? Best, Hussain