linaskerath
commented
1 year ago 2023 02 21
- [x] ozone layer/atmospheric gas data
- [x] weather station data
- [x] how to navigate on the websites aka which files represent what and which are relevant
- [x] file type / how to read file
- [x] outlier testing: is there a known upper limit of snow grain diameter mm ?
- [x] Python package for calculating slope and slope direction
- [x] how are the seasons in Greenland? Are there variations of length, are some more similar to each other than the seasons for us?
Slope feature:
- 359.9 and 0.1 degrees are technically the same but very different numerical values
- should be fine for Random Forest
- otherwise, transform to linear feature (Southerness vs. Northness as angle towards South and North)
Season/Month feature:
- do not consider seasons since they very much from South to North
- leave in Months for now until we find a Python package with daily sunlight duration per coordinate
Outlier Removal:
- we can use the Albedo which is the overall reflectivity of the surface (attribute from website of optical dataset)
- there might be arbitrary snow grain diameter values since optical remote sensing is not applicable to areas of bare ice
- if we want to filter out the areas of bare ice and only keep the snow-covered areas, we only keep pixels with Albedo > 0.565 (https://geusbulletin.org/index.php/geusb/article/view/5284)
- Do data quality checks based on the additional albedo data. Map and plot albedo data and opt data - grain size. If outliers correlate with ice areas then that could mean we should exclude ice data.
External data:
- no additional external data - ozone layer data will most likely not have a large impact.
Weather Station data: https://geus-my.sharepoint.com/personal/bav_geus_dk/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fbav%5Fgeus%5Fdk%2FDocuments%2FOnline%20files%2FSnow%20model%20runs&ga=1
-
What to use / focus on:
-
PROMICE_coordinates_2015.csv: list of all weather stations with longitude, latitude, elevation → only look at the ones above 1000 m as there is more likely snow instead of ice
-
{weather_station}_surface.nc: surface melt
- Features:
- meltflux = energy excess per cm^2 per timestep
- melt_mweq = how many m^3 of water produced per timestep ( divide by 1000 to convert to kg) → melt at surface in each time step of energy into surface and out of surface (phase change)
- Baptiste said the data should also include info about:
- incoming / outgoing shortwave radiation which, if taken the ratio, is the same as Albedo from optical dataset
- snowfall
- theta (?) 2 m temperature
- Should have high agreement with optical dataset
- Features:
-
{weather_station}_slwc.nc: snow liquid water content
- Features:
- slwc: firn liquid water content per layer and per point in time
- time: point in time
- level: level of layers 1 - 101 → look at level 1 below surface. start by only taking the water content and assuming that that amount should be visible to the optical method. Later, maybe adjust to layer depth.
- Depth: layers have different depths (do not consider for now)
- Should have high agreement with microwave dataset
- Features:
-
-
General info:
- melt water surface can be high but firn liquid water of top layer can be low (melt did not happen for a long time because underlying snow was cold) → maybe optical would say melt, microwave would say no melt
- weather stations measure per hour → probably take average of day
- weather station measurements will flicker because Albedo causes fine line between melting and not melting (hence very locally)
- Optical dataset is more robust indicator for snow cover, Weather station is more robust indicator for bare ice
-
Final thoughts:
-
DATA EXPLORATION
-
Baptiste suggested to look at the stations KAN_U, KPC_U, EGP and CEN for the years 2017, 2018, 2019 because he can assure high data quality for those (see the figure of his paper where gray background is bare ice)
-
Try to match both datasets to weather station data - take daily avg or date and time specific. Or take the past day’s data. Ideally weather station data (or melt threshold) should match the optical data (threshold) better → quality of the match decides whether we will use it as feature or validator
-
- we will probably need to use the weather station data as evaluation for our model outputs
- we might not be able to use all stations (Baptiste said a lot of them yield insufficient data quality)
- we would have a point observation for multiple pixels which would lead to a coarse resolution → how to put weather stations onto a map and map to raster data
- we could possibly split the trainings sets into one for snow-covered and one for bare ice areas → might consider an additional different model for ice area prediction. use different labels from weather station data?
- maybe post-processing of model output with weather station info?
-
ninazahn
2023 02 07
[x] How are day values collected - one value for the day or an average?
[x] Weather station data
[x] Biases:
There might be different results in the beginning and end of the season due to the grain diameter changing during melt and staying elevate for a period of time.
Data collection:
Optical data - the algorithm works best when sun is highest. The best scene is picked pixel by pixel. Lines or weird looking blobs could be due to two different frame bounds meeting, different zenith scenes meeting or cloud obstruction in some parts of the day and clear sky at the others. We could add a potential feature - name of frame with date and time of observation ( to know the exact time the value for the pixel was recorded). The satellite has the same orbit every time.
Microwave data collection - satellite flies 2x per day (morning and afternoon) over Greenland. The value is either the average or the value of one of the two passes.
Having weather station data we will have to create a surface energy balance model? - Energy transfer from air to surface. Difference between short wave radiation (solar radiation and surface emitted energy) vs only thermal energy. With clouds, energy bounces back and forth between surface and clouds which increases temperature. Calculated from features like thermal radiation, temperature etc.
Possibly post-processing is needed for clouds - adjust for cloudy vs not cloudy days. To check we'll need to calculate and compare the accuracy from cloudy vs non - cloudy days to weather station info.
Potential additional data sets: