search

ARGANS / shellfish_and_algae-MODEL

0 stars 0 forks source link

Symlinks to non-existent files #6

Closed nmaltsev closed 2 years ago

nmaltsev commented 2 years ago

After analyzing the results of the preprocessing scripts, I found that the output directory contains symlinks to non-existent files:

./12450795f303df1b8f76437e6a92486d/_pretreated/par:                                                                                total 0                                                                                                                                  lrwxrwxrwx 1 root root 56 Aug  4 16:15 parNWSmodelNetCDF2021-01to2022-01.nc -> '/media/share/data/12450795f303df1b8f76437e6a92486d/par/*'  
# This model uses the `NASA Ocean Colour 2021, filled` dataset
...
./123443ddd8ea5843e1c91407d9d777a5/_pretreated/par:                                                                                total 0                                                                                                                                  lrwxrwxrwx 1 root root 56 Aug  5 16:09 parBSmodelNetCDF2021-01to2022-01.nc -> '/media/share/data/123443ddd8ea5843e1c91407d9d777a5/par/*'  
# This model uses the `NASA Ocean Colour 2020, filled` dataset

In both cases, the par directory was empty, so I suspect the dataimport scripts didn't load the data set for the par parameter. I cannot estimate the impact of this issue on the results of the dataread scripts.

qjutard commented 2 years ago

There is currently no live data download and pretreatment script for PAR (these take too long too run live as well). The 2020 dataset should point to '/media/global/PAR_ref_OC_2020.nc' and the 2021 dataset to another file that I have yet to create.

I will add the necessary switch option to the pretreatment for the 2020 dataset. The 2021 dataset will remain not to be used for now. This should also allow us to treat PAR as any other variable in dataread/start.py

nmaltsev commented 2 years ago

Thank you for the clarification

qjutard commented 2 years ago

This should be solved as of https://github.com/ARGANS/shellfish_and_algae-MODEL/commit/b1ecab77fcc3bd9aa163ade57903925759f689ed

However it is still not actually used for simulations, I need to adapt dataread/start.py for this.