update TA32 api to v2020.6.2 + core package - Githubissues

TwoRavens / TwoRavens

A web application for data exploration, machine learning and statistical analysis, model construction and meta analysis tools, that integrates with data repositories including Dataverse.

Apache License 2.0

22 stars 1 forks source link

update TA32 api to v2020.6.2 + core package #868

Closed raprasad closed 4 years ago

raprasad commented 4 years ago

[x] API update
- https://gitlab.com/datadrivendiscovery/ta3ta2-api/-/blob/devel/HISTORY.md
[x] update core package version to 2020.5.18
- https://gitlab.com/datadrivendiscovery/d3m/-/tags
[x] add CMU as a TA2
[x] update output directory: https://datadrivendiscovery.org/wiki/display/work/Evaluation+Workflow

raprasad commented 4 years ago

upgrade numpy to numpy>=1.16.4 in tworavens-preprocess
- mlbox: ERROR: tworavens-preprocess 1.1.3 has requirement numpy==1.16.4, but you'll have numpy 1.18.2 which is incompatible.
tensorflow incompatibility with ludwig and mlbox. keeping mlbox for now
- mlbox: tensorflow==2.0.1
- ludwig: tensorflow==1.15.3
- mljar-supervised: tensorflow==2.2.0

raprasad commented 4 years ago

Directory notes:

[ ] 'solvers' directory
- How does the tworavens_solver know where to write files?
- where is 'solvers' coming from?
- base this off of the D3MOUTPUTDIR/temp/ravens-solvers or similar
- remove hardcoding in wrapped.js
[x] EventData shared directory: EVTDATA_2_TWORAVENS_DIR
- base this off of the D3MOUTPUTDIR/temp/evtdata_user_datasets
[x] TWORAVENS_USER_DATASETS_DIR
- base this off of the D3MOUTPUTDIR/temp/TwoRavens_user_datasets

raprasad commented 4 years ago

/ravens_volume now split into two directories which may be set via env variables:

RAVENS_TEST_DATA_READONLY_DIR
- Points to the location of the https://github.com/TwoRavens/tworavens-test-datasets
  - Default: /ravens_volume/test_data
RAVENS_VOLUME_DIR
- Used to share data between TA3 docker containers
- Default: {TwoRavens repo dir}/RAVENS_VOLUME_TEST

Impacts:

[ ] Eventdata across dataset sharing

raprasad commented 4 years ago

[x] replace usages of d3m_config.additional_inputs