lhenneman / hyspdisp

4 stars 4 forks source link

ZCTA shape files + Monthly global planetary boundary layer heights + reanalysis meteorology files #35

Closed schoolAccountMajaG closed 4 years ago

schoolAccountMajaG commented 5 years ago

If the DESCRIPTION contains LazyData: true, then datasets will be lazily loaded. This means that they won’t occupy any memory until you use them. (...). You can see that memory usage doesn’t change until you inspect the flights dataset stored inside the package.

http://r-pkgs.had.co.nz/data.html

Not sure how feasible it would be to attach these files to the package. They are huge. What do you think @cchoirat

cchoirat commented 5 years ago

https://git-lfs.github.com/ might be a possibility, but easy local downloads might be an easier way to move forward.

On Fri, Jun 14, 2019 at 10:45 AM Maja notifications@github.com wrote:

If the DESCRIPTION contains LazyData: true, then datasets will be lazily loaded. This means that they won’t occupy any memory until you use them. (...). You can see that memory usage doesn’t change until you inspect the flights dataset stored inside the package.

http://r-pkgs.had.co.nz/data.html

Not sure how feasible it would be to attach these files to the package. They are huge. What do you think @cchoirat https://github.com/cchoirat

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/35?email_source=notifications&email_token=AA73AZISL5UHNQGDXW7MI6LP2NLDFA5CNFSM4HYFY3C2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GZQAOGA, or mute the thread https://github.com/notifications/unsubscribe-auth/AA73AZNXW5QOQXD2TAKDR4DP2NLDFANCNFSM4HYFY3CQ .

lhenneman commented 5 years ago

At the least, I think it would be easy to incorporate downloading the large files files into R/hyspdisp_fac_model_parallel.R if they do not already exist. I took the downloading steps outside of that function originally because there were issues with parallel calls to download the same file.

Maybe this could link up with #18 and we could define a "project_setup" function that creates the directory structure and optionally downloads all the appropriate input files.

schoolAccountMajaG commented 5 years ago

@lhenneman I think it is a good idea, but we need another vignette in case someone wants to see how we did it. This would be the case, for example, for users who want to do the analysis for a different country.

cchoirat commented 5 years ago

great idea

perfectly appropriate to have several theme-specific vignettes

On Fri, Jun 14, 2019 at 4:26 PM Maja notifications@github.com wrote:

@lhenneman https://github.com/lhenneman I think it is a good idea, but we need another vignette in case someone wants to see how we did it. This would be the case, for example, for users who want to do the analysis for a different country.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/35?email_source=notifications&email_token=AA73AZLGYX6L4R65IHDC7CTP2OTB7A5CNFSM4HYFY3C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXW6L6I#issuecomment-502130169, or mute the thread https://github.com/notifications/unsubscribe-auth/AA73AZOEGTV3X5JWLBPXEELP2OTB7ANCNFSM4HYFY3CQ .

schoolAccountMajaG commented 5 years ago

https://github.com/lhenneman/hyspdisp/issues/27 - Sometimes servers fail to work. I will close this issue then if this is not a big problem. We should at least tell the user to try again later, I guess.

cchoirat commented 5 years ago

servers?

On Fri, Jun 14, 2019 at 4:43 PM Maja notifications@github.com wrote:

27 https://github.com/lhenneman/hyspdisp/issues/27 - Sometimes servers

fail to work. I will close this issue then if this is not a big problem. We should at least tell the user to try again later, I guess.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/35?email_source=notifications&email_token=AA73AZOUYJZPHJ37TL4RMGLP2OU7LA5CNFSM4HYFY3C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXW73MA#issuecomment-502136240, or mute the thread https://github.com/notifications/unsubscribe-auth/AA73AZL3PEW6ZYLRVJ2JPNLP2OU7LANCNFSM4HYFY3CQ .

schoolAccountMajaG commented 5 years ago

I mean, the website just does not work, so it is not possible to download. That happened to me.

lhenneman commented 5 years ago

I've run into that before. Should we have the new project_setup function detect if that is the case and return a warning with a suggestion to try again later?

On Fri, Jun 14, 2019 at 10:45 AM Maja notifications@github.com wrote:

I mean, the website just does not work, so it is not possible to download. That happened to me.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/35?email_source=notifications&email_token=AHMZYW4R7QBN6TAMLIVXCSLP2OVKHA5CNFSM4HYFY3C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXXAECQ#issuecomment-502137354, or mute the thread https://github.com/notifications/unsubscribe-auth/AHMZYW2I6D5YLYYIJWYH5OTP2OVKHANCNFSM4HYFY3CQ .

-- Lucas R.F. Henneman, Ph.D. (404) 788-2161 lhenneman@gmail.com

schoolAccountMajaG commented 5 years ago

I like this idea, simple!

schoolAccountMajaG commented 4 years ago

Implemented in disperseR