search

lhenneman / hyspdisp

4 stars 4 forks source link

Project Structure #18

Closed schoolAccountMajaG closed 5 years ago

schoolAccountMajaG commented 5 years ago

Now, in the Vignette creates different folders are created at different points. This might be a bit confusing for the user. I was thinking about creating all the folders at the beg. Something like this:

cchoirat commented 5 years ago

Precise structure TBD, but I'm on board with the idea.

On Wed, Jun 12, 2019 at 2:13 PM Maja notifications@github.com wrote:

Now, in the Vignette different folders are created at different points. This might be a bit confusing for the user. I was thinking about creating all the folders at the beg. Something like this:

  • Project Main_dir:
    • Input
      • zcta_dir: ZCTA (A Zip Code Tabulation Area) shapefiles
      • hpbl_dir : Monthly global planetary boundary layer files.
    • Output
      • proc_dir: the overarching directory containing each of the subdirectories, and the working directory for HYSPLIT
      • hysraw_dir: raw hyspdisp output (one file for each emissions event)
      • ziplink_dir: files containing ZIP code linkages
      • meteo_dir: (reanalysis) meteorology files
      • rdata_dir: RData files containing HyADS source-receptor matrices

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/18?email_source=notifications&email_token=AA73AZIYP5GYYJHVTYGTQZDP2DR6HA5CNFSM4HXIKLNKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GZBO2BA, or mute the thread https://github.com/notifications/unsubscribe-auth/AA73AZO6WNZI52BRFSGRIPTP2DR6HANCNFSM4HXIKLNA .

lhenneman commented 5 years ago

Yes, also on board. Would we still check if the directories exist and create them if not in the functions?

schoolAccountMajaG commented 5 years ago

I do not think you have to check if they exist. I checked with 


main_dir<-"/Users/majagarbulinska/Dropbox/EPFL/EnvProject"
  input_dir <- file.path(main_dir, 'input')
    zcta_dir <- file.path(input_dir, 'zcta_500k')
    hpbl_dir <- file.path(input_dir, 'hpbl')
  output_dir <- file.path(main_dir, 'output')
    hysraw_dir <- file.path(output_dir, 'output_hysplit')
    ziplink_dir <- file.path(output_dir, 'output_ziplinks')
    meteo_dir <- file.path(output_dir, 'meteo')
    rdata_dir <- file.path(output_dir, 'output_rdata')
    exposure_dir <- file.path(output_dir, 'exposure_data')

dir.create(main_dir, showWarnings = F)
dir.create(input_dir, showWarnings = F)
dir.create(output_dir, showWarnings = F)
dir.create(zcta_dir, showWarnings = F)
dir.create(hpbl_dir, showWarnings = F)
dir.create(hysraw_dir, showWarnings = F)
dir.create(ziplink_dir, showWarnings = F)
dir.create(meteo_dir, showWarnings = F)
dir.create(rdata_dir, showWarnings = F)
dir.create(exposure_dir, showWarnings = F)

And it works fine. We could loop that, but in my opinion, this is fine.

cchoirat commented 5 years ago

  1. I suggest you create a function that handles file structure creation.

  2. Can you (all) use FALSE, not F?

On Fri, Jun 14, 2019 at 3:57 PM Maja notifications@github.com wrote:

I do not think you have to check if they exist. I checked with

main_dir<-"/Users/majagarbulinska/Dropbox/EPFL/EnvProject" input_dir <- file.path(main_dir, 'input') zcta_dir <- file.path(input_dir, 'zcta_500k') hpbl_dir <- file.path(input_dir, 'hpbl') output_dir <- file.path(main_dir, 'output') hysraw_dir <- file.path(output_dir, 'output_hysplit') ziplink_dir <- file.path(output_dir, 'output_ziplinks') meteo_dir <- file.path(output_dir, 'meteo') rdata_dir <- file.path(output_dir, 'output_rdata') exposure_dir <- file.path(output_dir, 'exposure_data')

dir.create(main_dir, showWarnings = F) dir.create(input_dir, showWarnings = F) dir.create(output_dir, showWarnings = F) dir.create(zcta_dir, showWarnings = F) dir.create(hpbl_dir, showWarnings = F) dir.create(hysraw_dir, showWarnings = F) dir.create(ziplink_dir, showWarnings = F) dir.create(meteo_dir, showWarnings = F) dir.create(rdata_dir, showWarnings = F) dir.create(exposure_dir, showWarnings = F)

And it works fine. We could loop that, but in my opinion, this is fine.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/18?email_source=notifications&email_token=AA73AZJTMG2BTDEKKUXGDWDP2OPTXA5CNFSM4HXIKLNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXW3W4Q#issuecomment-502119282, or mute the thread https://github.com/notifications/unsubscribe-auth/AA73AZL5FOR7RCFPGNOCB5DP2OPTXANCNFSM4HXIKLNA .

schoolAccountMajaG commented 5 years ago
  1. I suggest you create a function that handles file structure creation.

What kind of function?

cchoirat commented 5 years ago

a skeleton type of function to handle create/checks of

main_dir<-"/Users/majagarbulinska/Dropbox/EPFL/EnvProject" input_dir <- file.path(main_dir, 'input') zcta_dir <- file.path(input_dir, 'zcta_500k') hpbl_dir <- file.path(input_dir, 'hpbl') output_dir <- file.path(main_dir, 'output') hysraw_dir <- file.path(output_dir, 'output_hysplit') ziplink_dir <- file.path(output_dir, 'output_ziplinks') meteo_dir <- file.path(output_dir, 'meteo') rdata_dir <- file.path(output_dir, 'output_rdata') exposure_dir <- file.path(output_dir, 'exposure_data')

dir.create(main_dir, showWarnings = F) dir.create(input_dir, showWarnings = F) dir.create(output_dir, showWarnings = F) dir.create(zcta_dir, showWarnings = F) dir.create(hpbl_dir, showWarnings = F) dir.create(hysraw_dir, showWarnings = F) dir.create(ziplink_dir, showWarnings = F) dir.create(meteo_dir, showWarnings = F) dir.create(rdata_dir, showWarnings = F) dir.create(exposure_dir, showWarnings = F)

On Fri, Jun 14, 2019 at 4:01 PM Maja notifications@github.com wrote:

  1. I suggest you create a function that handles file structure creation.

What kind of function?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/18?email_source=notifications&email_token=AA73AZL7TZ32DGQ52H7GV5DP2OQELA5CNFSM4HXIKLNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXW4DIY#issuecomment-502120867, or mute the thread https://github.com/notifications/unsubscribe-auth/AA73AZPDJSOZXWYUMPXLCNLP2OQELANCNFSM4HXIKLNA .

schoolAccountMajaG commented 5 years ago

Not sure if I understand. Like that:

createdirs<-function(main_dir) { dir.create(main_dir, showWarnings = F) dir.create(input_dir, showWarnings = F) ... } ?

cchoirat commented 5 years ago

yes, along these lines (as clean and flexible as possible)

@lhenneman, thoughts?

lhenneman commented 5 years ago

Love it, and also like the idea of packaging the function with a catch-all setup function that downloads everything you need for a run. Thoughts?

cchoirat commented 5 years ago

Same!

lhenneman commented 5 years ago

Still brainstorming on this and taking notes here. I think the new project_setup function should return a list of input objects for functions R/hyspdisp_fac_model_parallel and R/hyspdisp_zip_link. This would help with #24, so a primary input would be the output from project_setup.

schoolAccountMajaG commented 5 years ago

@lhenneman How do you feel about doing two functions. _projsetup - which will create all the folders and define paths, and another _downloaddata function?

lhenneman commented 5 years ago

Great, sounds good to me.

On Tue, Jun 18, 2019 at 8:43 AM Maja notifications@github.com wrote:

@lhenneman https://github.com/lhenneman How do you feel about doing two functions. proj_setup - which will create all the folders and define paths, and another download_data function?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lhenneman/hyspdisp/issues/18?email_source=notifications&email_token=AHMZYWZGTA42M6FOWRC7CKTP3DJ5NA5CNFSM4HXIKLNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX6JL6I#issuecomment-503092729, or mute the thread https://github.com/notifications/unsubscribe-auth/AHMZYW32DK6ZANZKGLIZE4DP3DJ5NANCNFSM4HXIKLNA .

-- Lucas R.F. Henneman, Ph.D. (404) 788-2161 lhenneman@gmail.com

cchoirat commented 5 years ago

Fine with me too, let's try to stick with good function naming conventions, say:

create_project
download_data

Sounds good?

schoolAccountMajaG commented 5 years ago

Sounds good to me!

schoolAccountMajaG commented 5 years ago

implemented in dipserseR