arnaudon
commented
7 hours ago nice! do you want to make a PR for this fix? Otherwise I can do it, np! Thanks for spotting this one!
Open paul019 opened 8 hours ago
arnaudon
commented
7 hours ago nice! do you want to make a PR for this fix? Otherwise I can do it, np! Thanks for spotting this one!
The issue
When I set the luigi config parameter
CreateQuantumGraph.edge_sizeto a small value (for instance0.1), I get the following error message:Error message
```python RemoteTraceback: """ Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) ^^^^^^^^^^^^^^^^^^^ File "/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py", line 923, in _get_new_D0 increment = lasing_threshold_linear(new_mode, graph, D0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py", line 1084, in lasing_threshold_linear * np.real(compute_overlapping_factor(mode, graph)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py", line 292, in compute_overlapping_factor in_mask, pump_mask = _get_mask_matrices(graph.graph["params"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py", line 226, in _get_mask_matrices pump_mask = sc.sparse.diags(_convert_edges(params["pump"])).dot(in_mask) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/.venv/lib/python3.11/site-packages/scipy/sparse/_base.py", line 450, in dot return self @ other ~~~~~^~~~~~~ File "/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/.venv/lib/python3.11/site-packages/scipy/sparse/_base.py", line 669, in __matmul__ return self._matmul_dispatch(other) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/.venv/lib/python3.11/site-packages/scipy/sparse/_base.py", line 579, in _matmul_dispatch raise ValueError('dimension mismatch') ValueError: dimension mismatch """ The above exception was the direct cause of the following exception: ValueError Traceback (most recent call last) Cell In[2], [line 10](vscode-notebook-cell:?execution_count=2&line=10) [6](vscode-notebook-cell:?execution_count=2&line=6) config.pump_config.D0_steps = 6 [8](vscode-notebook-cell:?execution_count=2&line=8) config.create_quantum_graph.edge_size = 0.1 ---> [10](vscode-notebook-cell:?execution_count=2&line=10) lasing_modes, analysis_data = compute_lasing_modes( [11](vscode-notebook-cell:?execution_count=2&line=11) network, passive_modes, pattern, config=default_config_1, debug=True, n_workers=8 [12](vscode-notebook-cell:?execution_count=2&line=12) ) [14](vscode-notebook-cell:?execution_count=2&line=14) with open(f"lasing_modes_{pattern_id_}{no_str}.pkl", "wb") as f: [15](vscode-notebook-cell:?execution_count=2&line=15) pickle.dump(lasing_modes, f) File ~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:205, in compute_lasing_modes(input_graph, passive_modes, pattern, refraction_index, config, n_workers, debug) [203](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:203) print("FindThresholdModes") [204](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:204) start_time = time.time() --> [205](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:205) threshold_modes, analysis_data = netsalt.find_threshold_lasing_modes( [206](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:206) mode_trajectories, [207](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:207) quantum_graph_with_pump, [208](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:208) quality_method=config.mode_search_config.quality_method, [209](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:209) ) [210](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:210) if debug: [211](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/network-optimizer/netopt/netsalt/netsalt.py:211) print(f"Finished in {(time.time() - start_time):.1f} seconds") File ~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:998, in find_threshold_lasing_modes(modes_df, graph, quality_method) [996](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:996) else: [997](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:997) with multiprocessing.Pool(graph.graph["params"]["n_workers"]) as pool: --> [998](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:998) for mode_id, new_D0, new_mode_approx in pool.imap( [999](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:999) partial(_get_new_D0, graph=graph, D0_steps=D0_steps), args [1000](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:1000) ): [1001](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:1001) analysis_data["modes"][mode_id]["D0s"].append(new_D0) [1002](https://file+.vscode-resource.vscode-cdn.net/Users/Paul_Obernolte/P/ml_with_nanophotonic_lasers/network-optimizer/notebooks/playground/~/P/ml_with_nanophotonic_lasers/netSALT/netsalt/modes.py:1002) new_D0s[mode_id] = new_D0 File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py:873, in IMapIterator.next(self, timeout) [871](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py:871) if success: [872](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py:872) return value --> [873](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py:873) raise value ValueError: dimension mismatch ```The reason
The reason for this issue lies in the following code snippet:
https://github.com/arnaudon/netSALT/blob/fa0a2aed237f134f80aa88dd6a8afb5f7d0707ca/netsalt/tasks/passive.py#L49-L87
Here is a step-by-step breakdown of what is happening:
create_quantum_graph,params["inner"]is set to an array whose length is equal to the initial number of edges in the graph.oversample_graph,quantum_graph.graph["params"]["inner"]is (correctly) set to an array whose length is equal to the number of edges in the oversampled graph.quantum_graph.graph["params"]["inner"]is overwritten with the valueparams["inner"]that was set in line 70! This causes the problem.The solution
The solution is very simple: Just switch line 85 and 86! I tested it, and it works like a charm.