jamesdolezal / slideflow

Deep learning library for digital pathology, with both Tensorflow and PyTorch support.
https://slideflow.dev
GNU General Public License v3.0
234 stars 39 forks source link

[BUG] Datasets do not pass `min_tiles` filter when training GANs #315

Closed siddhir closed 8 months ago

siddhir commented 10 months ago

Description

GAN training io issue likely because it takes in the filters dictionary but not the .filter() function when constructing this initial dataset for io. Filtering after this step doesn’t seem to work and neither does including min_tiles=21 within the initial P.dataset(….min_tiles=21, filters={….}) call. Given I couldn’t figure out a way to include the min_tiles within the dictionary, I basically duplicated the annotation file and included another filter called “MIN_TILE” which excluded all WSIs with <21 extracted tiles.

To Reproduce

Steps to reproduce the behavior:

  1. Use min_tiles argument when constructing any dataset for GAN

def main(): P = sf.Project('/mnt/data/PROJECTS/Title') P.annotations = './qc_plan.csv' full_dataset = P.dataset( tile_px=299, tile_um=302, filters={'site': ['XXXXXXXXXXX], 'progressor':[XXXXX], 'QC_Sid': ['QC_PASS'], }, min_tiles=21 )

train_df = full_dataset.filter({'site': [XXXXXXX]}) 

P.gan_train(
    dataset=train_df,
    model='stylegan3',
    cfg='stylegan2',
    gpus=4,
    batch=128,
    kimg=10000,
    resize=256,
    tile_labels= "label",
    exp_label="label"

)

if name == 'main': mp.freeze_support() main()

Expected behavior

Should take in the min_tiles argument when constructing dataset

Environment:

Additional context

jamesdolezal commented 9 months ago

This should be fixed in the latest dev update (f939885). Give it a try and see if it works - as a reminder, you'll need to update both the main slideflow repository as well as the GAN submodules:

git checkout dev 
git pull
git submodule update --recursive
jamesdolezal commented 9 months ago

For planning - this will be included in the 2.2 release. I can't put it in the next patch release due to somewhat complex dependencies on the StyleGAN submodules which are being updated for 2.2.

jamesdolezal commented 8 months ago

Released in 2.2 - thanks!