Random Ideas of things to implement

[Ivan-E-Johnson]

Ensure we are using caudate to its full potential

from lightning.pytorch.callbacks import DeviceStatsMonitor
trainer = Trainer(callbacks=[DeviceStatsMonitor()]) 

Find bottlenecks in the pipeline

trainer = Trainer(profiler="simple")

Visual bar to help track how things are going

self.log(..., prog_bar=True)

Save Images

def training_step(self):
    tensorboard = self.logger.experiment
    tensorboard.add_image()

More info here

Add Schedualer to optimizer

    def configure_optimizers(self):
        optimizer = torch.optim.SGD(
            self.parameters(),
            lr=self.hparams.lr,
            momentum=0.9,
            weight_decay=5e-4,
        )
        steps_per_epoch = 45000 // BATCH_SIZE
        scheduler_dict = {
            "scheduler": OneCycleLR(
                optimizer,
                0.1,
                epochs=self.trainer.max_epochs,
                steps_per_epoch=steps_per_epoch,
            ),
            "interval": "step",
        }
        return {"optimizer": optimizer, "lr_scheduler": scheduler_dict}

[Ivan-E-Johnson]

EarlyStopping

LearningRateFinder

BatchSizeFinder

[Ivan-E-Johnson]

UNETR

Hausdorffdtloss

generalizedwassersteindiceloss

Cool Other things