CliMA / ClimaLand.jl

Clima's Land Model
Apache License 2.0
35 stars 8 forks source link

reduce dependence on external data servers #625

Open juliasloan25 opened 3 months ago

juliasloan25 commented 3 months ago

Describe the bug

We download some of our data directly from external servers every time we run a simulation. This causes problems, e.g. when those servers are down we can't download data and can't run our simulations. We can address this by fully switching to use ClimaArtifacts.

SNOTEL data bug description

Suddenly I get a docs failure on two separate branches that don't make any changes in docs/. It comes from docs/tutorials/standalone/Snow/data_tutorial.jl and seems to be due to a dimension mismatch when creating a dataframe.

ERROR: LoadError: LoadError: DimensionMismatch: Length of nms doesn't match length of x.
in expression starting at /home/runner/work/ClimaLand.jl/ClimaLand.jl/docs/src/generated/standalone/Snow/data_tutorial.md:2
when executing the following code block from inputfile `~/work/ClimaLand.jl/ClimaLand.jl/docs/tutorials/standalone/Snow/data_tutorial.jl`

metadata = DataTools.snotel_metadata();
metacols = ["id", "state", "elev", "lat", "lon"]
DataFrames.rename!(metadata, Symbol.(metacols));

Failing Github actions:

To Reproduce

Ideally a minimal reproducible example:

# some code here

Or even better, add it as a unit test, and open pull request.

Project

If not using the `examples` project: ``` paste your Project.toml here. ``` ``` paste your Manifest.toml here. ```

System details

Any relevant system information:

Related issues / PRs

Please add any relevant links.

Sbozzolo commented 3 months ago

There were probably server issues. I meged the PR.

I think that the real issue is that we should try to avoid depending on an external server if the data we use is always the same.

juliasloan25 commented 3 months ago

There were probably server issues. I meged the PR.

I think that the real issue is that we should try to avoid depending on an external server if the data we use is always the same.

Agreed, I updated this issue to highlight that