Optimize the case of source specific logic tree

[micheles]

This happens for instance in the demo LogicTreeCase2ClassicalPSHA and in the model for South Africa. We should be able to recognize such cases (there is an applyToSources for each source) and to change the calculation logic. For instance in the case of the demo there are 3**4 = 81 combinations, but we should be able to reduce the computation weight to 3 * 4 = 12 combinations, with a ~7 times speedup. For the South Africa model there are 2**24 combinations that should be reduced to 2*24 computations, with a potential speedup of 2**24/2*24 ~ 350,000 times!

[micheles]

Closing because it will not be solved any soon in the general case, while in practice it is already solved by the change in https://github.com/gem/oq-engine/pull/4846. Here is the speedup in LogicTreeCase2ClassicalPSHA (10x):

   $ oq show performance 38754
   ============================ ======== ========= =======
   operation                    time_sec memory_mb counts 
   ============================ ======== ========= =======
   total classical_split_filter 335      4.05469   23     
   make_contexts                127      0.0       129,492
   get_poes                     41       0.0       129,492
    $ oq show performance 38755  # 10x faster, 9x less counts!!
   ============================ ======== ========= ======
   operation                    time_sec memory_mb counts
   ============================ ======== ========= ======
   total classical_split_filter 34       6.53516   3     
   make_contexts                13       0.0       14,388
   get_poes                     4.39047  0.0       14,388

[micheles]

Reopening it because now we have (or will have) the funding for working on it. It is true that we do not duplicate calculations anymore, but we are storing in source_info a terrible amount of duplicate information (for LogicTreeCase2 162 sources instead of 18) which will not scale for large logic trees. The idea is to attach to each source a small logic tree object. For instance in a case with two sources "1" and "2" depending on 2 parameters a and b, with a taking values a1 a2 a3 and b taking values b1 b2 b3, when sending the source "1" we will do 3x3 calculations and when sending the source "2" other 3x3 calculations, recomposing the PoEs at the end. It is nearly what we are already doing, but sending the sources once and not nine times (remember that in the past we were sending the same source 81 times!). This will solve the ugliness of the multiplication of the source models and source groups that we have now: in this case we will keep having 1 source model with 2 source groups, not 81 source models with 162 source groups.

This will be a large and difficult project, which will likely require a refactoring of logictree.py and related code. My suggestions to start are

1. ~prototype the new approach outside of the engine~
2. ~consider only the case of sampling by making full enumeration an error in the new system (in practice the logic trees will be so large that only sampling makes sense)~
3. do not change at all the input files (source_model.xml and source_model_logic_tree.xml)

In a second phase we could do more changes (or not).

[micheles]

NB: the event based calculator is still fully inefficient for full enumeration: for N sources with C choices each, we are killed by a factor C*N/ (CN which becomes immediately huge. We ignored that until now since event based in practice always uses sampling, but the problem is there, making impossible even calculations that should be possible. Fixing the problem will require several steps:

• [x] simplify the seed algorithm that we are currently using: fix the seed only once in src.sample_ruptures and loop on the group IDs
• [x] avoid transferring C**N copies of the sources
• [x] avoid reading C**N copies of the sources
• [x] avoid calling .count_ruptures too many times

[micheles]

I am closing the ticket now that even complex logic trees like the one from Guillaume Daniel's are manageable. Further work will be done in new tickets,