search

CIGLR-ai-lab / GreatLakes-TempSensors

Collaborative repository for optimizing the placement of temperature sensors in the Great Lakes using the DeepSensor machine learning framework. Aiming to enhance the quantitative understanding of surface temperature variability for better environmental monitoring and decision-making.
MIT License
0 stars 0 forks source link

Incorporate Bathymetry as a Context Set for DeepSensor #34

Closed DaniJonesOcean closed 1 month ago

DaniJonesOcean commented 1 month ago

Sure! Below is a GitHub issue outlining the task for incorporating bathymetry as a context set in the DeepSensor model. This includes getting the bathymetry data, formatting it for DeepSensor, and training the model with the new context set. The main supervisor will provide assistance with obtaining the bathymetry data.

Task Description:

Integrate bathymetry data as a new context set for the DeepSensor model. This task involves obtaining bathymetry data, formatting it appropriately for DeepSensor, and training the model with this new context set to enhance the spatial network design for environmental monitoring.

Checklist:

  1. Obtain Bathymetry Data:

    • [x] @DaniJonesOcean: obtain relevant bathymetry data for the Great Lakes.
    • [x] Ensure the data covers the necessary spatial and temporal resolutions required for the DeepSensor model.

  2. Format Bathymetry Data for DeepSensor:

    • [x] Preprocess the bathymetry data to align with the required input format for DeepSensor.
    • [x] Create a context set that includes bathymetry data, ensuring consistency with existing context sets used in the model.

  3. Train DeepSensor with the New Context Set:

    • [x] Configure the DeepSensor environment to include the new bathymetry context set.
    • [x] Train the DeepSensor model using the updated context set that includes bathymetry.
    • [x] Monitor the training process and document any improvements or challenges observed.
    • [x] Evaluate the model's performance with the new context set and compare it to the baseline model without bathymetry.

  4. Documentation and Reporting:

    • [x] Create a Jupyter notebook to document the entire process of incorporating bathymetry data, including code, preprocessing steps, and results.
    • [x] Save the trained model and relevant output files.
    • [x] Upload the Jupyter notebook, formatted bathymetry data, and trained model to the shared GitHub repository under an appropriate directory.
DaniJonesOcean commented 1 month ago

@eredding02 I've created a new bathymetry file that will hopefully work with DeepSensor:

https://github.com/CIGLR-ai-lab/GreatLakes-TempSensors/blob/main/notebooks/debugging/interpolated_bathymetry.nc

I'll also share it with you via Google Drive and Colab. When you get a chance, give it a try and let me know how it goes!

DaniJonesOcean commented 1 month ago

@eredding02 I pretty much consider this done, aside from the bit of extra plotting that we discussed yesterday.

The "documentation and reporting" steps are captured in our Milestone 5 tasks, so they are kind of duplicated here.

Once you have some comparison plots (spatial, time series) so we can see some differences between training with bathymetry and training without bathymetry, we can close this issue.