mmcdermott / EventStreamGPT

Dataset and modelling infrastructure for modelling "event streams": sequences of continuous time, multivariate events with complex internal dependencies.
https://eventstreamml.readthedocs.io/en/latest/
MIT License
102 stars 16 forks source link

Removed event type specificity in measurements interface, both during modeling and generation. #22

Closed mmcdermott closed 1 year ago

mmcdermott commented 1 year ago

Does two things: (1) Removes the ability for measurements to explicitly be configured to occur only on select event types. (2) Removes support for dynamic single-label classification tasks (aside from event_type, which is a special case and always mandated to be single label).

These are necessary for the following reasons: (1) For event type specificity to be meaningful during generation, we need to have post processing that removes measurements not allowed after event types have been generated, which we currently don't have. As a result, this current functionality is broken. It should be re-added (correctly) at some point, with tests to validate it, but until then it should be removed so as to avoid broken portions of the code. (2) Without the ability to state that a measurement is specific to an event type, dynamic, single-label classification tasks don't actually make that much sense, as there is an inherent assumption in them that they will always be measured exactly one time (on their allowed event types). As we support event aggregation, many "single label" tasks in reality end up being multi-label (as multiple valid events get aggregated together) and without event type support others end up being only partially observed single label tasks, which we don't currently support (as we'd need to predict (a) whether the task is measured at all and if so then (b) what label is observed, whereas now we just predict (b) by virtue of the assumption it is universally observed on valid event types.)

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 86.36% and project coverage change: -0.28 :warning:

Comparison is base (943086f) 69.35% compared to head (84e9e03) 69.07%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## dev #22 +/- ## ========================================== - Coverage 69.35% 69.07% -0.28% ========================================== Files 30 30 Lines 5279 5180 -99 ========================================== - Hits 3661 3578 -83 + Misses 1618 1602 -16 ``` | [Impacted Files](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott) | Coverage Δ | | |---|---|---| | [EventStream/data/dataset\_polars.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vZGF0YS9kYXRhc2V0X3BvbGFycy5weQ==) | `72.21% <ø> (+0.35%)` | :arrow_up: | | [...eam/transformer/conditionally\_independent\_model.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vdHJhbnNmb3JtZXIvY29uZGl0aW9uYWxseV9pbmRlcGVuZGVudF9tb2RlbC5weQ==) | `98.71% <ø> (-0.07%)` | :arrow_down: | | [EventStream/transformer/config.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vdHJhbnNmb3JtZXIvY29uZmlnLnB5) | `84.23% <ø> (-0.17%)` | :arrow_down: | | [...ansformer/lightning\_modules/generative\_modeling.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vdHJhbnNmb3JtZXIvbGlnaHRuaW5nX21vZHVsZXMvZ2VuZXJhdGl2ZV9tb2RlbGluZy5weQ==) | `44.65% <0.00%> (+0.33%)` | :arrow_up: | | [EventStream/transformer/nested\_attention\_model.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vdHJhbnNmb3JtZXIvbmVzdGVkX2F0dGVudGlvbl9tb2RlbC5weQ==) | `100.00% <ø> (ø)` | | | [EventStream/transformer/model\_output.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vdHJhbnNmb3JtZXIvbW9kZWxfb3V0cHV0LnB5) | `84.10% <88.23%> (-0.17%)` | :arrow_down: | | [EventStream/data/config.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vZGF0YS9jb25maWcucHk=) | `54.22% <100.00%> (-0.14%)` | :arrow_down: | | [EventStream/data/dataset\_base.py](https://app.codecov.io/gh/mmcdermott/EventStreamML/pull/22?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Matthew+McDermott#diff-RXZlbnRTdHJlYW0vZGF0YS9kYXRhc2V0X2Jhc2UucHk=) | `74.62% <100.00%> (-0.65%)` | :arrow_down: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.