search

wmayner / pyphi

A toolbox for integrated information theory.
https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006343
Other
374 stars 97 forks source link

ExplicitTPM: Numpy universal funcs and NDArrayOperatorsMixin #79

Closed isacdaavid closed 1 year ago

isacdaavid commented 1 year ago

Now it's safe to merge, the second regression after fixing TPM validation has also been sorted out (normal array methods such as tpm.sum(axis=0) were not being handled after the last refactoring). The important fixes are bc0193 and 80def5. 90f9f5 is unrelated, but improves the config option I recently added. Rest of commits are aimed at fixing a few tests along the way.

For validation, this is the system from the 4.0 demo, values look good:

>>> network
Network(ExplicitTPM([[[[1. 0. 0.]
   [0. 0. 0.]]

  [[1. 1. 1.]
   [0. 0. 1.]]]

 [[[0. 1. 0.]
   [1. 1. 0.]]

  [[0. 1. 1.]
   [1. 0. 1.]]]]), cm=[[1 1 0]
 [0 1 1]
 [1 1 1]])
>>> candidate_system
Subsystem(A, B, C)
>>> pyphi.new_big_phi.sia(candidate_system)
┌──────────────────────────────────┐
│ SystemIrreducibilityAnalysis     │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━      │
│      Subsystem:  A,B,C           │
│  Current state:  1,0,0           │
│            φ_s: 2.0              │
│ Normalized φ_s: 0.4              │
│          CAUSE:  (0, 0, 0)       │
│           II_c: 3.0              │
│         EFFECT:  (0, 1, 0)       │
│           II_e: 3.0              │
│   #(tied MIPs): 2                │
│      Partition:                  │
│                 3 parts: {A,B,C} │
│                 [[0 0 1]         │
│                  [1 0 1]         │
│                  [1 1 0]]        │
└──────────────────────────────────┘
>>> pyphi.new_big_phi.phi_structure(candidate_system)
2023-01-02 16:53:36,229 INFO services.py:1470 -- View the Ray dashboard at http://127.0.0.1:8265
┌──────────────────────────────────────┐
│             PhiStructure             │
│ ════════════════════════════════════ │
│               Φ:  5.410149997115375  │
│ #(distinctions):  4                  │
│           Σ φ_d:  1.792481250360578  │
│    #(relations): 14                  │
│           Σ φ_r:  3.6176687467547968 │
│ ┌──────────────────────────────────┐ │
│ │ SystemIrreducibilityAnalysis     │ │
│ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━      │ │
│ │      Subsystem:  A,B,C           │ │
│ │  Current state:  1,0,0           │ │
│ │            φ_s: 2.0              │ │
│ │ Normalized φ_s: 0.4              │ │
│ │          CAUSE:  (0, 0, 0)       │ │
│ │           II_c: 3.0              │ │
│ │         EFFECT:  (0, 1, 0)       │ │
│ │           II_e: 3.0              │ │
│ │   #(tied MIPs): 2                │ │
│ │      Partition:                  │ │
│ │                 3 parts: {A,B,C} │ │
│ │                 [[0 0 1]         │ │
│ │                  [1 0 1]         │ │
│ │                  [1 1 0]]        │ │
│ └──────────────────────────────────┘ │
└──────────────────────────────────────┘

And what we wanted to achieve in the first place:

>>> import pickle
>>> network.tpm
ExplicitTPM([[[[1. 0. 0.]
   [0. 0. 0.]]

  [[1. 1. 1.]
   [0. 0. 1.]]]

 [[[0. 1. 0.]
   [1. 1. 0.]]

  [[0. 1. 1.]
   [1. 0. 1.]]]])
>>> pickle.loads(pickle.dumps(network.tpm))
ExplicitTPM([[[[1. 0. 0.]
   [0. 0. 0.]]

  [[1. 1. 1.]
   [0. 0. 1.]]]

 [[[0. 1. 0.]
   [1. 1. 0.]]

  [[0. 1. 1.]
   [1. 0. 1.]]]])
>>>

State of the art of the test suite. The errors/failures I sampled come from outdated coverage, which is to say, I couldn't find extra issues with this PR.

$  py.test -k 'not test_relations' -W ignore::UserWarning

======================================== short test summary info ========================================
FAILED test/test_actual.py::test_background_3_node[before_state0-purview0-0.0] - ValueError: must prov...
FAILED test/test_actual.py::test_background_3_node[before_state1-purview1-1.0] - ValueError: must prov...
FAILED test/test_actual.py::test_ac_ex1_transition - ValueError: must provide purview state for genera...
FAILED test/test_actual.py::test_sia - ValueError: must provide purview state for generalized intrinsi...
FAILED test/test_actual.py::test_causal_nexus - ValueError: must provide purview state for generalized...
FAILED test/test_actual.py::test_true_ces - ValueError: must provide purview state for generalized int...
FAILED test/test_big_phi.py::test_conceptual_info - NotImplementedError
FAILED test/test_big_phi.py::test_sia_single_micro_node_selfloops_have_phi - NotImplementedError
FAILED test/test_big_phi.py::test_sia_big_network_0_thru_3 - ray.exceptions.RayTaskError(NotImplementedError): ray::_map_reduce_tree() (pid=433024, ip=10.140.118.148)
FAILED test/test_big_phi.py::test_sia_bipartitions - pyphi.conf.ConfigurationError: CUT_ONE_APPROXIMAT...
FAILED test/test_big_phi.py::test_ces_concepts_share_the_same_subsystem[False] - NotImplementedError
FAILED test/test_big_phi.py::test_ces_concepts_share_the_same_subsystem[True] - NotImplementedError
FAILED test/test_cache.py::test_split_mechanism_mice_is_not_reusable[False] - assert 0 == 1
FAILED test/test_compute_network.py::test_complexes_standard - ray.exceptions.RayTaskError(NotImplementedError): ray::_map_reduce_tree() (pid=433350, ip=10.140.118.148)
FAILED test/test_concept_style_cuts.py::test_unpickle - ValueError: if no total is given, chunksize mu...
FAILED test/test_concept_style_cuts.py::test_concept_style_phi - ValueError: if no total is given, chu...
FAILED test/test_distribution.py::test_purview_max_entropy_distribution - TypeError: argument of type ...
FAILED test/test_distribution.py::test_repertoire_shape - assert [] == [1, 1, 1]
FAILED test/test_macro_subsystem.py::test_macro_cut_is_for_micro_indices - TypeError: 'int' object is ...
FAILED test/test_metrics.py::test_default_distribution_measures - AssertionError: assert {'AID',\n 'AP...
FAILED test/test_metrics.py::test_default_asymmetric_distribution_measures - AssertionError: assert {'...
FAILED test/test_metrics_ces.py::test_ces_distances - NotImplementedError
FAILED test/test_metrics_ces.py::test_sia_uses_ces_distances - NotImplementedError
FAILED test/test_models.py::test_ria_ordering_and_equality - assert <[AttributeError("'NoneType' objec...
FAILED test/test_models.py::test_null_ria - assert  is None
FAILED test/test_models.py::test_ria_repr_str - AttributeError: 'NoneType' object has no attribute 'ties'
FAILED test/test_models.py::test_mice_ordering_by_purview - assert <[AttributeError("'NoneType' object...
FAILED test/test_models.py::test_mice_repr_str - AttributeError: 'NoneType' object has no attribute 't...
FAILED test/test_models.py::test_specified_states_and_indices - NotImplementedError
FAILED test/test_models.py::test_concept_repr_str - AttributeError: 'NoneType' object has no attribute...
FAILED test/test_models.py::test_ces_repr_str - AttributeError: 'NoneType' object has no attribute 'ties'
FAILED test/test_new_big_phi.py::test_compute_subsystem_ces[fig4] - AssertionError: assert {'__class__...
FAILED test/test_partition.py::test_partitioned_repertoire_with_tripartition - ValueError: must provid...
FAILED test/test_partition.py::test_tripartitions_choses_smallest_purview - pyphi.conf.ConfigurationEr...
FAILED test/test_subsystem.py::test_validate_cut_nodes_equal_subsystem_nodes - Failed: DID NOT RAISE <...
FAILED test/test_subsystem_cause_effect_info.py::test_cause_info - TypeError: generalized_intrinsic_di...
FAILED test/test_subsystem_cause_effect_info.py::test_effect_info - TypeError: generalized_intrinsic_d...
FAILED test/test_subsystem_cause_effect_info.py::test_cause_effect_info - TypeError: generalized_intri...
FAILED test/test_subsystem_expand.py::test_expand_effect_repertoire - ray.exceptions.RayTaskError(NotImplementedError): ray::_map_reduce_tree() (pid=433410, ip=10.140.118.148)
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-CAUSE-expected0] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-CAUSE-expected1] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-CAUSE-expected2] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-CAUSE-expected3] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-EFFECT-expected4] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-EFFECT-expected5] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-EFFECT-expected6] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[None-EFFECT-expected7] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut8-CAUSE-expected8] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut9-CAUSE-expected9] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut10-CAUSE-expected10] - AttributeError: 'NoneT...
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut11-CAUSE-expected11] - AttributeError: 'NoneT...
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut12-EFFECT-expected12] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut13-EFFECT-expected13] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut14-EFFECT-expected14] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_find_mice[cut15-EFFECT-expected15] - AttributeError: 'None...
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-CAUSE-expected0] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-CAUSE-expected1] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-CAUSE-expected2] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-CAUSE-expected3] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-EFFECT-expected4] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-EFFECT-expected5] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-EFFECT-expected6] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[None-EFFECT-expected7] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[cut8-CAUSE-expected8] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[cut9-CAUSE-expected9] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[cut12-EFFECT-expected12] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[cut13-EFFECT-expected13] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_mic_or_mie[cut14-EFFECT-expected14] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_phi_max[None-mechanism0-0.25] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_phi_max[None-mechanism1-0.5] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_phi_max[None-mechanism2-0.333333] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_phi_max[None-mechanism3-0.4999995] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_phi_max[cut4-mechanism4-0.0] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_phi_max[cut5-mechanism5-0.5] - NotImplementedError
FAILED test/test_subsystem_phi_max.py::test_phi_max[cut6-mechanism6-0.0] - NotImplementedError
FAILED test/test_subsystem_small_phi.py::test_find_mip[CAUSE-subsystem0-None-mechanism0-purview0-expected0]
FAILED test/test_subsystem_small_phi.py::test_find_mip[CAUSE-subsystem1-cut1-mechanism1-purview1-expected1]
FAILED test/test_subsystem_small_phi.py::test_find_mip[EFFECT-subsystem2-None-mechanism2-purview2-expected2]
FAILED test/test_subsystem_small_phi.py::test_find_mip[EFFECT-subsystem3-cut3-mechanism3-purview3-expected3]
FAILED test/test_subsystem_small_phi.py::test_find_mip[EFFECT-subsystem4-cut4-mechanism4-purview4-expected4]
FAILED test/test_subsystem_small_phi.py::test_find_mip[EFFECT-subsystem5-cut5-mechanism5-purview5-expected5]
FAILED test/test_subsystem_small_phi.py::test_find_mip[EFFECT-subsystem6-cut6-mechanism6-purview6-expected6]
FAILED test/test_subsystem_small_phi.py::test_cause_mip - NotImplementedError
FAILED test/test_subsystem_small_phi.py::test_effect_mip - NotImplementedError
FAILED test/test_subsystem_small_phi.py::test_phi_cause_mip - NotImplementedError
FAILED test/test_subsystem_small_phi.py::test_phi_cause_mip_reducible - NotImplementedError
FAILED test/test_subsystem_small_phi.py::test_phi_effect_mip - NotImplementedError
FAILED test/test_subsystem_small_phi.py::test_phi_effect_mip_reducible - NotImplementedError
FAILED test/test_subsystem_small_phi.py::test_phi - NotImplementedError
FAILED docs/conventions.rst::conventions.rst
FAILED docs/examples/2014paper.rst::2014paper.rst
FAILED docs/examples/actual_causation.rst::actual_causation.rst
FAILED docs/examples/conditional_independence.rst::conditional_independence.rst
FAILED docs/examples/emergence.rst::emergence.rst
FAILED docs/examples/index.rst::index.rst
FAILED docs/examples/magic_cut.rst::magic_cut.rst
FAILED docs/examples/residue.rst::residue.rst
FAILED docs/examples/xor.rst::xor.rst
ERROR test/test_actual.py::test_prevention - AttributeError: module 'pyphi.examples' has no attribute ...
ERROR test/test_big_phi.py::test_clear_subsystem_caches_after_computing_sia_config_option - pyphi.conf...
ERROR test/test_big_phi.py::test_find_sia_sequential_standard_example - AttributeError: module 'pyphi....
ERROR test/test_big_phi.py::test_find_sia_parallel_standard_example - AttributeError: module 'pyphi.co...
ERROR test/test_big_phi.py::test_find_sia_sequential_noised_example - AttributeError: module 'pyphi.co...
ERROR test/test_big_phi.py::test_find_sia_parallel_noised_example - AttributeError: module 'pyphi.comp...
ERROR test/test_big_phi.py::test_find_sia_parallel_micro - AttributeError: module 'pyphi.compute.subsy...
ERROR test/test_big_phi.py::test_find_sia_sequential_micro - AttributeError: module 'pyphi.compute.sub...
ERROR test/test_big_phi.py::test_sia_complete_graph_standard_example - pyphi.conf.ConfigurationError: ...
ERROR test/test_big_phi.py::test_sia_complete_graph_s_noised - pyphi.conf.ConfigurationError: MICE_TIE...
ERROR test/test_big_phi.py::test_sia_macro - ValueError: Invalid TPM: probabilities must sum to 1.
ERROR test/test_big_phi.py::test_system_cut_styles - pyphi.conf.ConfigurationError: MICE_TIE_RESOLUTIO...
ERROR test/test_big_phi.py::test_parallel_and_sequential_ces_are_equal - ValueError: Invalid TPM: prob...
ERROR test/test_compute_network.py::test_all_complexes_parallelization - pyphi.conf.ConfigurationError...
ERROR test/test_json.py::test_json_deserialization - pyphi.conf.ConfigurationError: MICE_TIE_RESOLUTIO...
ERROR test/test_json.py::test_deserialization_memoizes_duplicate_objects - pyphi.conf.ConfigurationErr...
ERROR test/test_macro_subsystem.py::test_cut_indices - ValueError: Invalid TPM: probabilities must sum...
ERROR test/test_macro_subsystem.py::test_cut_mechanisms - ValueError: Invalid TPM: probabilities must ...
ERROR test/test_macro_subsystem.py::test_cut_node_labels_are_for_micro_elements - ValueError: Invalid ...
ERROR test/test_macro_subsystem.py::test_concept_str_uses_macro_node_labels - ValueError: Invalid TPM:...
ERROR test/test_macro_subsystem.py::test_node_labels - ValueError: Invalid TPM: probabilities must sum...
ERROR test/test_macro_subsystem.py::test_macro_subsystem - ValueError: Invalid TPM: probabilities must...
ERROR test/test_macro_subsystem.py::test_macro_cut_subsystem - ValueError: Invalid TPM: probabilities ...
ERROR test/test_parallel.py::test_as_completed - RuntimeError: Maybe you called ray.init twice by acci...
ERROR test/test_parallel.py::test_get_remote - RuntimeError: Maybe you called ray.init twice by accide...
ERROR test/test_parallel.py::test_map_with_iterator_no_chunksize - RuntimeError: Maybe you called ray....
ERROR test/test_parallel.py::test_map_with_lambda - RuntimeError: Maybe you called ray.init twice by a...
ERROR test/test_parallel.py::test_map_with_iterators - RuntimeError: Maybe you called ray.init twice b...
ERROR test/test_parallel.py::test_map_with_shortcircuit - RuntimeError: Maybe you called ray.init twic...
ERROR test/test_parallel.py::test_map_reduce - RuntimeError: Maybe you called ray.init twice by accide...
ERROR test/test_subsystem_expand.py::test_expand_cause_repertoire - pyphi.conf.ConfigurationError: MIC...
=========== 98 failed, 580 passed, 36 skipped, 27 deselected, 2 xfailed, 31 errors in 40.94s ============