DARMA-tasking / LB-analysis-framework

Analysis framework for exploring, testing, and comparing load balancing strategies
Other
3 stars 1 forks source link

#521: Enrich synthetic data set with shared blocks #522

Closed ppebay closed 1 month ago

ppebay commented 2 months ago

Resolves #521

DO NOT MERGE BEFORE #509 (PR #516)

ppebay commented 1 month ago

Now ready for review @lifflander @cwschilly

ppebay commented 1 month ago

@tlamonthezie I rebased this PR on top of develop, following the merger of PR #519 and I am now getting the following runtime error:

$ python LBAF_app.py  -c conf.yaml 
[LBAF_app] Found configuration file at path /Users/pppebay/Documents/Git/LB-analysis-framework/config/global.yaml
[LBAF_app] Found configuration file at path /Users/pppebay/Documents/Git/LB-analysis-framework/config/conf.yaml
[LBAF_app] Logging level: info
[lbsConfigurationValidator] Skeleton schema is valid
[lbsConfigurationValidator] Reading from data was chosen
[lbsConfigurationValidator] from_data schema is valid
[lbsConfigurationValidator] Checking algorithm schema of: {'name': 'InformAndTransfer', 'phase_id': 0, 'parameters': {'n_iterations': 8, 'n_rounds': 2, 'fanout': 2, 'order_strategy': 'arbitrary', 'transfer_strategy': 'Recursive', 'criterion': 'Tempered', 'max_objects_per_transfer': 8, 'deterministic_transfer': True}}
[lbsConfigurationValidator] Algorithm: {'name': 'InformAndTransfer', 'phase_id': 0, 'parameters': {'n_iterations': 8, 'n_rounds': 2, 'fanout': 2, 'order_strategy': 'arbitrary', 'transfer_strategy': 'Recursive', 'criterion': 'Tempered', 'max_objects_per_transfer': 8, 'deterministic_transfer': True}} schema is valid
[LBAF_app] Data stem: /Users/pppebay/Documents/Git/LB-analysis-framework/data/synthetic_lb_data/data
[LBAF_app] Executing LBAF version 1.0.2
[LBAF_app] Executing with Python 3.9.19
Traceback (most recent call last):
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/Applications/LBAF_app.py", line 622, in <module>
    LBAFApplication().run()
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/Applications/LBAF_app.py", line 471, in run
    reader = LoadReader(
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/IO/lbsVTDataReader.py", line 59, in __init__
    from ..imported.JSON_data_files_validator import \
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/imported/JSON_data_files_validator.py", line 19, in <module>
    import LBDatafile_schema
ModuleNotFoundError: No module named 'LBDatafile_schema'

Do you know what is wrong here? Thanks.

ppebay commented 1 month ago

Update: the same error happens for me with develop, following the merger of #519

ppebay commented 1 month ago

I confirm @cwschilly that your change fixed the problem with the LB data file schema file.

PR can be reviewed @cwschilly @lifflander, in the default setting that comes with this PR, any other load-balancer (whether it is vt's temperedLB or the MILP approach) must find a 0-imbalance as LBAF's InformAndTransfer does in a couple of iterations:

synthetic-dataset-blocks0 synthetic-dataset-blocks1

N.B.: the rendered edges in the figures above must be disregarded as they correspond to the VTK problem on certain versions of macOS.