brendanjmeade
commented
2 years ago @tbenthompson Thanks for bringing this up. This important now and moving forward. Responses below:
The runs/ folder didn't exist so I needed to create it. Is this something that we should just be creating if it doesn't exist?
Yes, and will do.
The saved elastic operator didn't exist so I had to go into command/japan_command.json to turn off loading. Can we just fall back to creating the elastic operator if doesn't exist? Make a big warning message that says "The command file requested loading the elastic operators but the operators cannot be found at FILEPATH_HERE. Recomputing the elastic operators instead." I might even add a similar warning for situations where the elastic operators are reloaded just so users aren't surprised.
That's a really good idea and I'll do it!
The operator saving errors because the data/operators folder doesn't exist. Can we create this automatically if it doesn't exist?
Yes, and will do!
- If I re-run the first cell that loads the command file, I get an error because the runs/2022-03-07-13-45-12/ already exists. This is because the RUN_NAME variable is being set globally on import. It would be nice to move it to be set when the command file is loaded. The current design means that you would not be able to run two separate block models in the same Python process. This is almost certainly going to be something you or others will want to do at some point and will be more painful to change the longer we wait to support it. It'll also be nice because I can re-run cells without restarting the notebook kernel.
I've run into this problem too and this deserves a fix too. Will do.
As a more general point, it could be nice to split the input and output folders apart. Currently, the operators are being saved back into the input data/ folder. Maybe they could be saved into an output folder instead. I could imagine the structure could look something like:
With the format we have now (see below) I think the input and output folders are separated. All output goes to a subfolder in runs and all input files live subfolders in data. What do you think?
project_name/
├── README.md
├── notebooks/
│ ├── block_model.ipynb
│ ├── visualize_results.ipynb
│ └── resolution_tests.ipynb
├── data/
| ├── command/
│ | ├── command_001.json
│ | └── command_NNN.json
│ ├── segment/
│ │ ├── segment_001.csv
│ │ └── segment_NNN.csv
│ ├── block/
│ │ ├── block_001.csv
│ │ └── block_NNN.csv
│ ├── station/
│ │ ├── station_001.csv
│ │ └── station_NNN.csv
│ ├── mesh/
│ | ├── mesh_params_001.json
│ | ├── mesh_params_NNN.json
│ | ├── mesh_001.msh
│ | └── mesh_NNN.msh
| └── operators/
│ ├── elastic_001.hdf5
│ └── elastic_NNN.hdf5
└── runs/
├── 2022-02-20-17-01-39/
│ ├── 2022-02-20-17-01-39.log
│ ├── elastic_operators.hdf5
│ ├── model_segment.csv
│ ├── model_block.csv
│ └── model_station.csv
└── NNNN-NN-NN-NN-NN-NN/
├── NNNN-NN-NN-NN-NN-NN.log
├── elastic_operators.hdf5
├── model_segment.csv
├── model_block.csv
└── model_station.csv
tbenthompson
jploveless
While getting set up to debug the H-matrix issue, I ran into a few things that might or might not be bugs depending on preferences:
runs/folder didn't exist so I needed to create it. Is this something that we should just be creating if it doesn't exist?command/japan_command.jsonto turn off loading. Can we just fall back to creating the elastic operator if doesn't exist? Make a big warning message that says "The command file requested loading the elastic operators but the operators cannot be found at FILEPATH_HERE. Recomputing the elastic operators instead." I might even add a similar warning for situations where the elastic operators are reloaded just so users aren't surprised.data/operatorsfolder doesn't exist. Can we create this automatically if it doesn't exist?runs/2022-03-07-13-45-12/already exists. This is because theRUN_NAMEvariable is being set globally on import. It would be nice to move it to be set when the command file is loaded. The current design means that you would not be able to run two separate block models in the same Python process. This is almost certainly going to be something you or others will want to do at some point and will be more painful to change the longer we wait to support it. It'll also be nice because I can re-run cells without restarting the notebook kernel.Folders
As a more general point, it could be nice to split the input and output folders apart. Currently, the operators are being saved back into the input data/ folder. Maybe they could be saved into an output folder instead. I could imagine the structure could look something like: