Issue with json config file

[prash-p]

I hope this is the correct place to ask for assistance. I'm trying to run the following json config, but the process fails and I'm not sure what's wrong. Could someone point me in the right direction? Thanks :)

{
  "General": {
    "Hyperparameters": [
      {
        "type": ".NumWorker",
        "params": {
          "value": 1
        }
      },
      {
        "type": ".BatchSize",
        "params": {
          "value": 1
        }
      }
    ],
    "Models": [
      {
        "type": "eisen.models.segmentation.vnet.VNet",
        "params": {
          "input_names": [
            "image"
          ],
          "output_names": [
            "prediction"
          ],
          "input_channels": 1,
          "output_channels": 7,
          "n_filters": 16,
          "filter_size": 3,
          "outputs_activation": "sigmoid",
          "normalization": "groupnorm"
        }
      }
    ]
  },
  "Training": {
    "Datasets": [],
    "Readers": [],
    "Transforms": [
      {
        "type": "eisen.transforms.imaging.ResampleNiftiVolumes",
        "params": {
          "fields": [
            "image"
          ],
          "resolution": [
            4,
            4,
            2
          ],
          "interpolation": "linear"
        }
      },
      {
        "type": "eisen.transforms.imaging.ResampleNiftiVolumes",
        "params": {
          "fields": [
            "label"
          ],
          "resolution": [
            4,
            4,
            2
          ],
          "interpolation": "nearest"
        }
      },
      {
        "type": "eisen.transforms.imaging.NiftiToNumpy",
        "params": {
          "fields": [
            "image",
            "label"
          ],
          "multichannel": false
        }
      },
      {
        "type": "eisen.transforms.imaging.CropCenteredSubVolumes",
        "params": {
          "fields": [
            "image",
            "label"
          ],
          "size": [
            128,
            128,
            128
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.MapValues",
        "params": {
          "fields": [
            "image"
          ],
          "min_value": 0,
          "max_value": 1,
          "channelwise": false
        }
      },
      {
        "type": "eisen.transforms.imaging.AddChannelDimension",
        "params": {
          "fields": [
            "image"
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.LabelMapToOneHot",
        "params": {
          "fields": [
            "label"
          ],
          "classes": [
            1,
            2,
            3,
            4,
            5,
            6,
            7
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.FilterFields",
        "params": {
          "fields": [
            "image",
            "label"
          ]
        }
      }
    ],
    "Losses": [
      {
        "type": "eisen.ops.losses.dice.DiceLoss",
        "params": {
          "input_names": [
            "prediction",
            "label"
          ],
          "output_names": [
            "dice_loss"
          ],
          "weight": 1,
          "dim": [
            2,
            3,
            4
          ]
        }
      }
    ],
    "Metrics": [
      {
        "type": "eisen.ops.metrics.dice.DiceMetric",
        "params": {
          "input_names": [
            "prediction",
            "label"
          ],
          "output_names": [
            "dice_metric"
          ],
          "weight": 1,
          "dim": [
            2,
            3,
            4
          ]
        }
      }
    ],
    "Optimizer": [
      {
        "type": "torch.optim.Adam",
        "params": {
          "lr": 0.001,
          "betas": [
            0.9,
            0.999
          ],
          "eps": 1e-8,
          "weight_decay": 0,
          "amsgrad": false
        }
      }
    ],
    "Workflow": [
      {
        "type": "eisen.utils.workflows.training.Training",
        "params": {
          "gpu": true
        }
      }
    ],
    "Hooks": [
      {
        "type": "eisen.utils.logging.summaries.TensorboardSummaryHook",
        "params": {
          "comparison_pairs": [],
          "show_all_axes": false
        }
      },
      {
        "type": "eisen.utils.logging.logs.LoggingHook",
        "params": {}
      }
    ]
  },  
  "Validation": {
    "Datasets": [],
    "Readers": [],
    "Transforms": [
      {
        "type": "eisen.transforms.imaging.ResampleNiftiVolumes",
        "params": {
          "fields": [
            "image"
          ],
          "resolution": [
            4,
            4,
            2
          ],
          "interpolation": "linear"
        }
      },
      {
        "type": "eisen.transforms.imaging.ResampleNiftiVolumes",
        "params": {
          "fields": [
            "label"
          ],
          "resolution": [
            4,
            4,
            2
          ],
          "interpolation": "nearest"
        }
      },
      {
        "type": "eisen.transforms.imaging.NiftiToNumpy",
        "params": {
          "fields": [
            "image",
            "label"
          ],
          "multichannel": false
        }
      },
      {
        "type": "eisen.transforms.imaging.CropCenteredSubVolumes",
        "params": {
          "fields": [
            "image",
            "label"
          ],
          "size": [
            128,
            128,
            128
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.MapValues",
        "params": {
          "fields": [
            "image"
          ],
          "min_value": 0,
          "max_value": 1,
          "channelwise": false
        }
      },
      {
        "type": "eisen.transforms.imaging.AddChannelDimension",
        "params": {
          "fields": [
            "image"
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.LabelMapToOneHot",
        "params": {
          "fields": [
            "label"
          ],
          "classes": [
            1,
            2,
            3,
            4,
            5,
            6,
            7
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.FilterFields",
        "params": {
          "fields": [
            "image",
            "label"
          ]
        }
      }
    ],
    "Metrics": [
      {
        "type": "eisen.ops.metrics.dice.DiceMetric",
        "params": {
          "input_names": [
            "prediction",
            "label"
          ],
          "output_names": [
            "dice_metric"
          ],
          "weight": 1,
          "dim": [
            2,
            3,
            4
          ]
        }
      }
    ],
    "Workflow": [
      {
        "type": "eisen.utils.workflows.validation.Validation",
        "params": {
          "gpu": true
        }
      }
    ],
    "Hooks": [
      {
        "type": "eisen.utils.logging.summaries.TensorboardSummaryHook",
        "params": {
          "comparison_pairs": [],
          "show_all_axes": false
        }
      },
      {
        "type": "eisen.utils.logging.logs.LoggingHook",
        "params": {}
      }
    ]
  }
}

[eisen-ai]

Hello,

I believe that you are trying to manipulate the field “label” of the data. If you are working with covid data, you don’t have that field as far as I know. You may try this other config file (read it and change things that need to be changed).

On 15. Sep 2020, at 19:40, prash-p notifications@github.com wrote:

I hope this is the correct place to ask for assistance. I'm trying to run the following json config, but the process fails and I'm not sure what's wrong. Could someone point me in the right direction? Thanks :)

{ "General": { "Hyperparameters": [ { "type": ".NumWorker", "params": { "value": 1 } }, { "type": ".BatchSize", "params": { "value": 1 } } ], "Models": [ { "type": "eisen.models.segmentation.vnet.VNet", "params": { "input_names": [ "image" ], "output_names": [ "prediction" ], "input_channels": 1, "output_channels": 7, "n_filters": 16, "filter_size": 3, "outputs_activation": "sigmoid", "normalization": "groupnorm" } } ] }, "Training": { "Datasets": [], "Readers": [], "Transforms": [ { "type": "eisen.transforms.imaging.ResampleNiftiVolumes", "params": { "fields": [ "image" ], "resolution": [ 4, 4, 2 ], "interpolation": "linear" } }, { "type": "eisen.transforms.imaging.ResampleNiftiVolumes", "params": { "fields": [ "label" ], "resolution": [ 4, 4, 2 ], "interpolation": "nearest" } }, { "type": "eisen.transforms.imaging.NiftiToNumpy", "params": { "fields": [ "image", "label" ], "multichannel": false } }, { "type": "eisen.transforms.imaging.CropCenteredSubVolumes", "params": { "fields": [ "image", "label" ], "size": [ 128, 128, 128 ] } }, { "type": "eisen.transforms.imaging.MapValues", "params": { "fields": [ "image" ], "min_value": 0, "max_value": 1, "channelwise": false } }, { "type": "eisen.transforms.imaging.AddChannelDimension", "params": { "fields": [ "image" ] } }, { "type": "eisen.transforms.imaging.LabelMapToOneHot", "params": { "fields": [ "label" ], "classes": [ 1, 2, 3, 4, 5, 6, 7 ] } }, { "type": "eisen.transforms.imaging.FilterFields", "params": { "fields": [ "image", "label" ] } } ], "Losses": [ { "type": "eisen.ops.losses.dice.DiceLoss", "params": { "input_names": [ "prediction", "label" ], "output_names": [ "dice_loss" ], "weight": 1, "dim": [ 2, 3, 4 ] } } ], "Metrics": [ { "type": "eisen.ops.metrics.dice.DiceMetric", "params": { "input_names": [ "prediction", "label" ], "output_names": [ "dice_metric" ], "weight": 1, "dim": [ 2, 3, 4 ] } } ], "Optimizer": [ { "type": "torch.optim.Adam", "params": { "lr": 0.001, "betas": [ 0.9, 0.999 ], "eps": 1e-8, "weight_decay": 0, "amsgrad": false } } ], "Workflow": [ { "type": "eisen.utils.workflows.training.Training", "params": { "gpu": true } } ], "Hooks": [ { "type": "eisen.utils.logging.summaries.TensorboardSummaryHook", "params": { "comparison_pairs": [], "show_all_axes": false } }, { "type": "eisen.utils.logging.logs.LoggingHook", "params": {} } ] },
"Validation": { "Datasets": [], "Readers": [], "Transforms": [ { "type": "eisen.transforms.imaging.ResampleNiftiVolumes", "params": { "fields": [ "image" ], "resolution": [ 4, 4, 2 ], "interpolation": "linear" } }, { "type": "eisen.transforms.imaging.ResampleNiftiVolumes", "params": { "fields": [ "label" ], "resolution": [ 4, 4, 2 ], "interpolation": "nearest" } }, { "type": "eisen.transforms.imaging.NiftiToNumpy", "params": { "fields": [ "image", "label" ], "multichannel": false } }, { "type": "eisen.transforms.imaging.CropCenteredSubVolumes", "params": { "fields": [ "image", "label" ], "size": [ 128, 128, 128 ] } }, { "type": "eisen.transforms.imaging.MapValues", "params": { "fields": [ "image" ], "min_value": 0, "max_value": 1, "channelwise": false } }, { "type": "eisen.transforms.imaging.AddChannelDimension", "params": { "fields": [ "image" ] } }, { "type": "eisen.transforms.imaging.LabelMapToOneHot", "params": { "fields": [ "label" ], "classes": [ 1, 2, 3, 4, 5, 6, 7 ] } }, { "type": "eisen.transforms.imaging.FilterFields", "params": { "fields": [ "image", "label" ] } } ], "Metrics": [ { "type": "eisen.ops.metrics.dice.DiceMetric", "params": { "input_names": [ "prediction", "label" ], "output_names": [ "dice_metric" ], "weight": 1, "dim": [ 2, 3, 4 ] } } ], "Workflow": [ { "type": "eisen.utils.workflows.validation.Validation", "params": { "gpu": true } } ], "Hooks": [ { "type": "eisen.utils.logging.summaries.TensorboardSummaryHook", "params": { "comparison_pairs": [], "show_all_axes": false } }, { "type": "eisen.utils.logging.logs.LoggingHook", "params": {} } ] } } — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/eisen-ai/covid19-challenge/issues/9, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOERTVDG43P7EUK32IUSZ6DSF6RH7ANCNFSM4RNPNXAQ.

[prash-p]

Hey, I don't see the "other config file" that you mentioned? With the covid data, is the label field called something else or are there no labels at all (sorry for the confusion, I must have missed something in the latest updates to the challenge),

[faustomilletari]

{
  "General": {
    "Hyperparameters": [
      {
        "type": ".NumWorker",
        "params": {
          "value": 1
        }
      },
      {
        "type": ".BatchSize",
        "params": {
          "value": 2
        }
      }
    ],
    "Models": [
      {
        "type": "eisen.models.segmentation.vnet.VNet",
        "params": {
          "input_names": [
            "image"
          ],
          "output_names": [
            "prediction"
          ],
          "input_channels": 1,
          "output_channels": 7,
          "n_filters": 16,
          "filter_size": 3,
          "outputs_activation": "sigmoid",
          "normalization": "groupnorm"
        }
      }
    ]
  },
  "Training": {
    "Datasets": [],
    "Readers": [],
    "Transforms": [
      {
        "type": "eisen.transforms.imaging.ResampleNiftiVolumes",
        "params": {
          "fields": [
            "image"
          ],
          "resolution": [
            4,
            4,
            2
          ],
          "interpolation": "linear"
        }
      },
      {
        "type": "eisen.transforms.imaging.ResampleNiftiVolumes",
        "params": {
          "fields": [
            "con",
            "ggo",
            "ill",
            "ilr",
            "mlr",
            "ssl",
            "slr"
          ],
          "resolution": [
            4,
            4,
            2
          ],
          "interpolation": "nearest"
        }
      },
      {
        "type": "eisen.transforms.imaging.NiftiToNumpy",
        "params": {
          "fields": [
            "image",
            "con",
            "ggo",
            "ill",
            "ilr",
            "mlr",
            "ssl",
            "slr"
          ],
          "multichannel": false
        }
      },
      {
        "type": "eisen.transforms.imaging.CropCenteredSubVolumes",
        "params": {
          "fields": [
            "image",
            "con",
            "ggo",
            "ill",
            "ilr",
            "mlr",
            "ssl",
            "slr"
          ],
          "size": [
            128,
            128,
            128
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.MapValues",
        "params": {
          "fields": [
            "image"
          ],
          "min_value": 0,
          "max_value": 1,
          "channelwise": false
        }
      },
      {
        "type": "eisen.transforms.imaging.AddChannelDimension",
        "params": {
          "fields": [
            "image"
          ]
        }
      },
      {
        "type": "eisen.transforms.imaging.StackImagesChannelwise",
        "params": {
          "fields": [
            "con",
            "ggo",
            "ill",
            "ilr",
            "mlr",
            "ssl",
            "slr"
          ],
          "dst_field": "label"
        }
      },
      {
        "type": "eisen.transforms.imaging.FilterFields",
        "params": {
          "fields": [
            "image",
            "label"
          ]
        }
      }
    ],
    "Losses": [
      {
        "type": "eisen.ops.losses.dice.DiceLoss",
        "params": {
          "input_names": [
            "prediction",
            "label"
          ],
          "output_names": [
            "dice_loss"
          ],
          "weight": 1,
          "dim": [
            2,
            3,
            4
          ]
        }
      }
    ],
    "Metrics": [
      {
        "type": "eisen.ops.metrics.dice.DiceMetric",
        "params": {
          "input_names": [
            "prediction",
            "label"
          ],
          "output_names": [
            "dice_metric"
          ],
          "weight": 1,
          "dim": [
            2,
            3,
            4
          ]
        }
      }
    ],
    "Optimizer": [
      {
        "type": "torch.optim.Adam",
        "params": {
          "lr": 0.001,
          "betas": [
            0.9,
            0.999
          ],
          "eps": 1e-8,
          "weight_decay": 0,
          "amsgrad": false
        }
      }
    ],
    "Workflow": [
      {
        "type": "eisen.utils.workflows.training.Training",
        "params": {
          "gpu": true
        }
      }
    ],
    "Hooks": []
  }
}

[prash-p]

Thanks, that worked. Is it possible to have a validation workflow? I tried the same JSON code with a validation workflow (minus optimizer and loss) but it failed.

Also, when I click the tensorflow button, I get an empty tab instead of a tensorflow dashboard (tried with chrome and firefox).

[eisen-ai]

That’s really weird…

Let me investigate… In any case we will try to get the STDERR stream as part of the “logs” of the model so at least you can see why things fail

On 16. Sep 2020, at 18:15, prash-p notifications@github.com wrote:

Thanks, that worked. Is it possible to have a validation workflow? I tried the same code with a validation workflow but it failed.

Also, when I click the tensorflow button, I get an empty tab instead of a tensorflow dashboard (tried with chrome and firefox).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/eisen-ai/covid19-challenge/issues/9#issuecomment-693511516, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOERTVBY2TAHGCRJGITK64TSGDQBBANCNFSM4RNPNXAQ.