d3m - config file

[raprasad]

• Allow admin entry/upload of a basic config file
  • Assess whether files indicated are reachable
  • Use variation of TA2 config specs below
• Pass config variables to TwoRavens UI via url params

---

Configuration variable	Description	Input/Output
dataset_schema	Path to the current dataset schema	Input
problem_schema	Path to the current problem schema	Input
training_data_root	Path to the root directory of the dataset described by dataset_schema	Input
pipeline_list	Path at which performers should write the pipeline list, output described in Section 4.1.3	Output
executables_root	Directory in which to write the Test Executables.	Output
temp_storage_root	Temporary storage root for performers to use.	N/A

---

Configuration variable	Description	Input/Output
test_data_root	Path to the root directory of the test data.	Input
solution	Path at which performers should write the Test Executable output, i.e. the solution to the challenge problem given the test data.	Output
temp_storage_root	Temporary storage root for performers to use.	N/A

[raprasad]

• [x] config branch models
• [x] config branch logic to get look for/retrieve files
• [x] pass info to rook either by direct url (to start) or with callback info
• [x] django management command to receive config file from command line and load to db
• [x] add size of dataset in bytes

[raprasad]

So far: The master branch has the ability to retrieve config information from django. if you pull the master, run the following afterwards: (assuming you're in a Terminal and have already done workon 2ravens)

1. fab init_db
   • adds new table with the D3M search config attributes here (as of 9/18): https://datadrivendiscovery.org/wiki/display/gov/TA2+Configuration+file+syntax
2. fab make_d3m_config
   • creates config settings with fully qualified paths based on your local system for the o_196 and o_4550 test data
3. fab run
4. go to: http://127.0.0.1:8080/config/d3m-config/list
   • if the above worked, you will see info on the config settings available in the database

[raprasad]

outcome:

• this url: http://127.0.0.1:8080/config/d3m-config/json/latest
• should give the following but with your local paths and a different id:

{
  "id": 6,
  "name": "o_196seed",
  "is_default": true,
  "dataset_schema": "/Users/yourname/Documents/github/TwoRavens/data/d3m/o_196seed/data/dataSchema.json",
  "problem_schema": "/Users/yourname/Documents/github/TwoRavens/data/d3m/o_196seed/problemSchema.json",
  "training_data_root": "/Users/yourname/Documents/github/TwoRavens/data/d3m/o_196seed/data",
  "executables_root": "/Users/yourname/Documents/github/TwoRavens/test_setup_local/d3m_output_o_196seed/temp",
  "pipeline_logs_root": "/Users/yourname/Documents/github/TwoRavens/test_setup_local/d3m_output_o_196seed/temp",
  "temp_storage_root": "/Users/yourname/Documents/github/TwoRavens/test_setup_local/d3m_output_o_196seed/temp",
  "created": "2017-09-19 01:50:11.257608+00:00",
  "modified": "2017-09-19 01:50:11.258682+00:00",
  "config_url": "/config/d3m-config/details/json/6"
}