Ideas for model improvements

[MagnusOstertag]

• two different models, one for the open sea and one for near land areas (harder)
• problem with ice versus wind and rain areas, can we know when they occur from the meteorology?
• can we learn the arctic currents? Or do we know sth from the physics
• pre-train model on similar/denoising tasks (like BERT etc) to reconstruct -> multi-stage learning
  • introduce artificial noise and let the model reconstruct
  • masking of parts of the image
  • batch sizes
• seasonality and climate change should be a large factor, do we account for them?
• search the literature for ways to improve u-nets
  • increasing the depth
  • skip connections
• maybe use the model parameters from each easier model as initialization for the more complex models? We should somehow utilize transfer learning

[MagnusOstertag]

• [ ] oversample hard scenes image segmentation lessons
• [ ] and/or adversarial validation: see whether a simple classifier can easily tell the samples from the two different distributions apart post
• [ ] Squeeze-and-Excitation Networks: hierarchy in channels github, pytorch
• [ ] do we have residual connections?
• [ ] adaptive learning rate ml mastery
• [ ] additional data: ice chart provider (!), month (!), location/difficult location flag (!) - maybe as a feature in the last layer before the feature map