This branch is built off of "faster_taxaCheck" branch (PR).
Some scripts did more than they suggested (specifically, taxaImport would initialise the dateAccess working directory, and also define regionGeometry), which could cause issues depending on the order in which scripts are sourced. sourced scripts assumed the location of various files (eg location of focalTaxa, external covariates, and location where external covariates are to be downloaded); these have now been specified in in masterScript. In addition, numerous user-specified parameters were defined within sourced scripts; these have all been moved up front in the masterScript, and are now saved/accessed from the dateAccessed folder.
Main changes: move all user-defined "control parameters" up front in masterScript. Divided taxaImport into initialiseRepository + defineRegionGeometry + taxaImport. initialiseRepository initialises dateAccessed folder and saves copy of control parameters, and files from external folder. Improve saving/loading of files and use of control parameters across files sourced from main (there may be additional sourced files/functions that require modification).
pipeline/import/initialiseRepository.R - script initialises the repository based on dateAccessed (create folder and subfolders), saves/loads all control parameters, pre-precesses focalTaxa (filters selected taxa & assign missing usageKeys), and saves a copy of focalTaxa, polyphyleticSpecies, metadataSummary, and focalCovariates from the external folder into the dateAccessed folder.
pipeline/import/defineRegionGeometry.R - Script imports defines region geometry using 'defineRegion' and saves it the working folder
functions/defineRegion.R - assign level and region attributes to the regionGeometry sf object
masterScript.R - define and describe all control parameters up front (where relevant, these parameters have been removed from scripts in which they were defined). Source initialiseRepository.R & defineRegionGeometry.R scripts. Updated some of the text descriptions of sourced scripts.
pipeline/import/taxaImport.R - Remove folder construction, focalTaxa pre-processing, region definition, and saving, which are handled by initialiseRepository.R and defineRegionGeometry.R. Loads control parameters, focalTaxa, polyphyleticSpecies, regionGeometry, and metadataSummary from dateAccessed folder. assign level and region attributes to dataList from attributes now stored in regionGeometry
pipeline/import/environmentalImport.R - Update import/messaging of parameters and regionGeometry. Move filtering of selected parameters and check of externally sourced parameters under Dataset Import section. Use myMesh instead of mesh for defining download extent. use folderName when saving environmentalDataImported at the end.
pipeline/integration/speciesDataProcessing.R - import custom functions up front and update import/messaging of regionGeometry and polyphyleticSpecies.
pipeline/models/speciesModelRuns.R - import custom functions up front. import metadataSummary from dateAccessed folder
Why have changes been made?
This branch is built off of "faster_taxaCheck" branch (PR).
Some scripts did more than they suggested (specifically, taxaImport would initialise the dateAccess working directory, and also define regionGeometry), which could cause issues depending on the order in which scripts are sourced. sourced scripts assumed the location of various files (eg location of focalTaxa, external covariates, and location where external covariates are to be downloaded); these have now been specified in in masterScript. In addition, numerous user-specified parameters were defined within sourced scripts; these have all been moved up front in the masterScript, and are now saved/accessed from the dateAccessed folder.
Main changes: move all user-defined "control parameters" up front in masterScript. Divided taxaImport into initialiseRepository + defineRegionGeometry + taxaImport. initialiseRepository initialises dateAccessed folder and saves copy of control parameters, and files from external folder. Improve saving/loading of files and use of control parameters across files sourced from main (there may be additional sourced files/functions that require modification).
PR should address issues regarding dateAccessed specification and saving/loading of polyphyleticspecies & metadataSummary.
What changes have been made?
pipeline/import/initialiseRepository.R - script initialises the repository based on dateAccessed (create folder and subfolders), saves/loads all control parameters, pre-precesses focalTaxa (filters selected taxa & assign missing usageKeys), and saves a copy of focalTaxa, polyphyleticSpecies, metadataSummary, and focalCovariates from the external folder into the dateAccessed folder.
pipeline/import/defineRegionGeometry.R - Script imports defines region geometry using 'defineRegion' and saves it the working folder
functions/defineRegion.R - assign level and region attributes to the regionGeometry sf object
masterScript.R - define and describe all control parameters up front (where relevant, these parameters have been removed from scripts in which they were defined). Source initialiseRepository.R & defineRegionGeometry.R scripts. Updated some of the text descriptions of sourced scripts.
pipeline/import/taxaImport.R - Remove folder construction, focalTaxa pre-processing, region definition, and saving, which are handled by initialiseRepository.R and defineRegionGeometry.R. Loads control parameters, focalTaxa, polyphyleticSpecies, regionGeometry, and metadataSummary from dateAccessed folder. assign level and region attributes to dataList from attributes now stored in regionGeometry
pipeline/import/environmentalImport.R - Update import/messaging of parameters and regionGeometry. Move filtering of selected parameters and check of externally sourced parameters under Dataset Import section. Use myMesh instead of mesh for defining download extent. use folderName when saving environmentalDataImported at the end.
pipeline/integration/speciesDataProcessing.R - import custom functions up front and update import/messaging of regionGeometry and polyphyleticSpecies.
pipeline/models/speciesModelRuns.R - import custom functions up front. import metadataSummary from dateAccessed folder