Currently the branch updates_july2022 contains the newest version v.14.4. It contains all the commits from @drotheram's reformat_add_data branch to breakout dsra, psra and exposure ancillary pieces.
DSRA path is referenced at 4 points in add_data.sh.
The current workflow works fine without issue if the dsra path is in a branch other than master.
If the branch is a master branch there is a function in DSRA_outputs2postgres.py in model factory do a replace between tree/master/contents that breaks the run.
Below is the log highlighting the error
python-opendrr_1 | 2022-10-31T23:58:25.400231447Z File "DSRA_outputs2postgres_lfs.py", line 213, in <module> python-opendrr_1 | 2022-10-31T23:58:25.400363240Z main() python-opendrr_1 | 2022-10-31T23:58:25.400392826Z File "DSRA_outputs2postgres_lfs.py", line 66, in main python-opendrr_1 | 2022-10-31T23:58:25.400399308Z dfsr[retrofit] = GetDataframeForScenario(url, branch, repo_list, retrofitPrefix, eqscenario, columnConfigParser, auth, args) python-opendrr_1 | 2022-10-31T23:58:25.400401732Z File "DSRA_outputs2postgres_lfs.py", line 154, in GetDataframeForScenario python-opendrr_1 | 2022-10-31T23:58:25.400429214Z segment = response[response.index('gsim_logic_tree_file'):] python-opendrr_1 | 2022-10-31T23:58:25.400457737Z ValueError: substring not found python-opendrr_1 | 2022-10-31T23:58:25.477103393Z Command exited with non-zero status 1
There are a few replace in DSRA_outputs2postgres.py
The 2nd screenshot above will run successfully but if you notice the dsraModelDir has to be written https://github.com/OpenDRR/earthquake-scenarios/contents/FINISHED for it to work. The link doesn't work, but in one of the replace command somwhere it is replacing /contents/ to tree/master which will then work.
So I think there are 2 fixes to consider? Fix the replace so that we dont need to hack the path for /contents/FINISHED if it the branch is master so we can put in directly tree/master/FINISHED?
Reduce the amount of DSRA path reference?
Currently the branch updates_july2022 contains the newest version v.14.4. It contains all the commits from @drotheram's reformat_add_data branch to breakout dsra, psra and exposure ancillary pieces.
DSRA path is referenced at 4 points in add_data.sh.
The current workflow works fine without issue if the dsra path is in a branch other than master. If the branch is a master branch there is a function in DSRA_outputs2postgres.py in model factory do a replace between tree/master/contents that breaks the run.
Below is the log highlighting the error
python-opendrr_1 | 2022-10-31T23:58:25.400231447Z File "DSRA_outputs2postgres_lfs.py", line 213, in <module> python-opendrr_1 | 2022-10-31T23:58:25.400363240Z main() python-opendrr_1 | 2022-10-31T23:58:25.400392826Z File "DSRA_outputs2postgres_lfs.py", line 66, in main python-opendrr_1 | 2022-10-31T23:58:25.400399308Z dfsr[retrofit] = GetDataframeForScenario(url, branch, repo_list, retrofitPrefix, eqscenario, columnConfigParser, auth, args) python-opendrr_1 | 2022-10-31T23:58:25.400401732Z File "DSRA_outputs2postgres_lfs.py", line 154, in GetDataframeForScenario python-opendrr_1 | 2022-10-31T23:58:25.400429214Z segment = response[response.index('gsim_logic_tree_file'):] python-opendrr_1 | 2022-10-31T23:58:25.400457737Z ValueError: substring not found python-opendrr_1 | 2022-10-31T23:58:25.477103393Z Command exited with non-zero status 1There are a few replace in DSRA_outputs2postgres.py
The 2nd screenshot above will run successfully but if you notice the dsraModelDir has to be written https://github.com/OpenDRR/earthquake-scenarios/contents/FINISHED for it to work. The link doesn't work, but in one of the replace command somwhere it is replacing /contents/ to tree/master which will then work.
So I think there are 2 fixes to consider? Fix the replace so that we dont need to hack the path for /contents/FINISHED if it the branch is master so we can put in directly tree/master/FINISHED? Reduce the amount of DSRA path reference?