search

DOI-USGS / mda.streams

backend tools for powstreams
Creative Commons Zero v1.0 Universal
3 stars 8 forks source link

API changes #67

Closed appling closed 9 years ago

appling commented 9 years ago

Proposed structure for mda.streams

MANAGE PROJECT/SESSION CONFIGURATION

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#86) get_scheme get_scheme write and change all other functions to use it
x (#75) set_scheme set_scheme good to go

ACQUIRE & PREPARE DATA FROM OTHER SOURCES

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#92) init_nwis_sites stage_nwis_sitelist link to pcode list in doc
x (#83) get_nwis_df stage_nwis_ts should read to df, then call write_tsv. link to pcode list in doc
x (#83) get_nldas_df stage_nldas_ts should read to df, then call write_tsv. make sure gets tz right
x (#83) [unwritten] write_ts should accept df, check tz, gzip, write to file. internal function. add or require units info in tsv
get_shed stage_watershed add to package. should read to file in prep for posting
[unwritten] stage_calc_ts add calculated values to SB
[unwritten] e.g., stage_uwy_depth, stage_nawqa_ts, stage_metab as needed, add additional functions to stage other data types

PUSH SITES & DATA TO SCIENCEBASE

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#92) create_site post_sites good to go
x (#83) post_nwis_ts [remove] remove; put loop in makefile
p (#83) post_ts post_ts test that it really doesn’t write duplicates on SB. add option to append
post_watershed post_watershed good to go (but check duplicates)

LOCATE ITEMS ON SCIENCEBASE (INTERNAL)

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#92) get_project_id locate_item get_project_root should find the continental stream metabolism folder, not the sites folder.
x (#92) get_publication_id [remove] merge with locate_folder
x (#92) get_presentation_id [remove] merge with locate_folder
x (#92) [unwritten] locate_site wrapper to locate_item
x (#92) [unwritten] locate_ts wrapper to locate_item
x (#92) [unwritten] locate_folder wrapper to locate_item

READ USER-FRIENDLY INFO FROM SCIENCEBASE

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#91) get_sites list_sites look up available sites from SB
get_ts_variables list_data look up available ts (and other?) data from SB. don't replace get_var_codes
[unwritten] list_presentations returns list of presentation names/titles (user-friendly IDs)
[unwritten] list_publications returns list of publication names/titles

READ DATA FROM SCIENCEBASE, DATARETRIEVAL, ETC

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#91) [unwritten] lookup_site_coords should get lat & long using dataRetrieval
get_watershed_WFS lookup_watershed_url ok

RETRIEVE DATA FROM SCIENCEBASE

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x download_ts download_ts remove session arg, use sbtools instead?
get_watershed_WMS download_watershed_wms implement (still looks like get_watershed_WFS)
[unwritten] download_presentation implement
[unwritten] download_publication implement

PULL DOWNLOADED DATA INTO R

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#86) read_ts read_ts modify to expect & read units. reads from file. also move "UTC" from units into POSIXct tz field
[unwritten] read_watershed read into R-useful format, e.g., for ggmaps?

MANAGE ITEM KEYS

STATUS FUNCTION NEW NAME PROPOSED CHANGES
x (#85) make_ts_variable make_ts_name make it use pkg.env$ts_prefix
x (#85) [unwritten] parse_ts_name make it use pkg.env$ts_prefix
x (#85) [unwritten] make_site_name opposite of parse_site_name
x (#85) split_site parse_site_name option to return source (eg NWIS), id (eg 0388819), or both
x (#85) get_ts_prefix [remove] move to pkg.env$ts_prefix
x (#85) get_ts_extension [remove] move to pkg.env$ts_extension
x (#85) get_ts_delim [remove] move to pkg.env$ts_delim
x (#74) [unwritten] get_var_codes lookup table to relate variable name to pcode. include units, streamMetabolizer names, English names, dataset types. assist with filters and selects.
x (#92) ts_variables_superset [remove] remove; instead, use get_var_codes(out="shortname")
### PUSH INFO TO SCIENCEBASE ? ?

NEW SBTOOLS FUNCTIONS

STATUS FUNCTION NEW NAME PROPOSED CHANGES
[unwritten] item_get_timestamp write in sbtools? tell when a file was last updated
get_title item_get_title move this to sbtools

other stuff

jordansread commented 9 years ago

@appling updated your comment

jordansread commented 9 years ago

oh, you did too.

jordansread commented 9 years ago
CATEGORY FUNCTION NEW NAME PROPOSED CHANGES
get_nwis_df stage_nwis_ts should read to df, then call write_unitted. link to pcode list in doc
get_nldas_df stage_nldas_ts should read to df, then call write_unitted. make sure gets tz right
aappling-usgs commented 9 years ago

CATEGORY: MANAGE ITEM KEYS

in PR #85

aappling-usgs commented 9 years ago

Still a few loose threads, especially with respect to watersheds, but I'm pretty satisfied and ready for this task to be checked off.