CliMA / ClimaArtifacts

Pre-processing pipelines for the input data used by the CliMA project
Apache License 2.0
4 stars 0 forks source link

Add FAQs #31

Closed Sbozzolo closed 3 weeks ago

Sbozzolo commented 4 weeks ago

Start collecting common complaints.

Sbozzolo commented 3 weeks ago

This is great! One suggestion for the next FAQ: I ended up copying the template for my download from another folder but it might be nice to point users to an easy first example, i.e., simply saving a file as an artifact. It would look something like this

using Downloads 
using ClimaArtifactsHelper

# file url should link directly to the object you want to download. 
const FILE_URL = "https://caltech.box.com/shared/static/cryptic_hash_path.nc"
const FILE_PATH = "ERA5_DATA.nc" # name of the file after downloading, e.g., data from ERA5

output_dir = "era5_data" # descriptive name of the folder where the artifact will be stored
if isdir(output_dir)
    @warn "$output_dir already exists. Content will end up in the artifact and may be overwritten."
    @warn "Abort this calculation, unless you know what you are doing."
else
    mkdir(output_dir)
end

if !isfile(FILE_PATH)
    @info "$FILE_PATH not found, downloading it (might take a while)"
    data_file = Downloads.download(FILE_URL)
    Base.mv(data_file, FILE_PATH)
    Base.cp(FILE_PATH, joinpath(output_dir, basename(FILE_PATH)))
end

@info "Data file generated!"
create_artifact_guided(output_dir; artifact_name = basename(@__DIR__))

I implemented a simpler version here: https://github.com/CliMA/ClimaArtifacts/pull/9

Could you try this and see if it works for your artifact? If it does, I'll merge it.

Julians42 commented 3 weeks ago

@Sbozzolo Tested this and can confirm it works!

Sbozzolo commented 3 weeks ago

Updated, thank you!