blaylockbk / Herbie

Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.
https://herbie.readthedocs.io/
MIT License
424 stars 70 forks source link

Can H.download() add a new parameter which can help users define the download file names? #321

Open GloriaYY opened 1 month ago

GloriaYY commented 1 month ago

the issue is as the title said.

The problems is when I download different subsets of the same model on the same date, it is hard to distinguish between them, manually renaming files can be a painful job. If H.download() can help solve this problem, herbie will just become more powerful.

Thanks for your nice work! Herbie has saved me a lot of time.

blaylockbk commented 1 month ago

Hi @GloriaYY, Glad you like using Herbie. Thanks for the poke; this is something I've been meaning to do for some time. For now, there is a workaround...take a look at #58.


Duplicate of #60

blaylockbk commented 1 month ago

Note to self...It shouldn't be too hard to add an argument name to download that modifies the self.LOCALFILE attribute. Users could set name='my-name' or even use Herbie attributes name=f"{H.model}_{H.DATE:%Y%M%D%H}_my-name".

GloriaYY commented 1 month ago

Thanks for the timely response. 

I have just noticed there was a same issue.

Well, the optimization point should be something around the way of generating the filename. Now, it seems Herbie is  using hash label as ordering tool, maybe we can use "target query time + subset name"  as label, for full set, default subset name is "all". 

The subset name is easy to generate, just extract only the numbers and  alphabets  from subset query name, about "target query time' we just extract only the numerical characters from query time.say, from subeset ":10[u|v]",  we take only "10uv" for the subset name in downloaded file; from query time "2022-06-01 00:00", we take only "202206010000" as target query time. 

Those labels should be enough to make files in order and meanwhile friendly for selection. Looks like there is no need to change any attributes of Herbie at the moment, maybe we don't need a new parameter, what we need is just a better file naming rule. :)

------------------ 原始邮件 ------------------ 发件人: "blaylockbk/Herbie" @.>; 发送时间: 2024年5月23日(星期四) 晚上11:05 @.>; @.**@.>; 主题: Re: [blaylockbk/Herbie] Can H.download() add a new parameter which can help users define the download file names? (Issue #321)

Note to self...It shouldn't be too hard to add an argument name to download that modifies the self.LOCALFILE attribute. Users could set name='my-name' or even use Herbie attributes name=f"{H.model}_{H.DATE:%Y%M%D%H}_my-name".

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

GloriaYY commented 1 month ago

OK, "subset name" and "target query time" alone is not enough to order all subset files, there should be a  "product" part also

------------------ 原始邮件 ------------------ 发件人: "blaylockbk/Herbie" @.>; 发送时间: 2024年5月23日(星期四) 晚上11:05 @.>; @.**@.>; 主题: Re: [blaylockbk/Herbie] Can H.download() add a new parameter which can help users define the download file names? (Issue #321)

Note to self...It shouldn't be too hard to add an argument name to download that modifies the self.LOCALFILE attribute. Users could set name='my-name' or even use Herbie attributes name=f"{H.model}_{H.DATE:%Y%M%D%H}_my-name".

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>