keras-team / keras-core

A multi-backend implementation of the Keras API, with support for TensorFlow, JAX, and PyTorch.
Apache License 2.0
1.27k stars 115 forks source link

Add optimizer weights to the model summary. #859

Closed qlzh727 closed 12 months ago

qlzh727 commented 12 months ago

The sample output with optimizer weights is like below:

Model: "language_model"
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ Layer (type)        ┃ Output Shape      ┃ Param # ┃ Connected to         ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ padding_mask        │ (None, None)      │       0 │ -                    │
│ (InputLayer)        │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ token_ids           │ (None, None)      │       0 │ -                    │
│ (InputLayer)        │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ opt_backbone        │ (None, None,      │ 1,315,… │ padding_mask[0][0],  │
│ (OPTBackbone)       │ 2048)             │         │ token_ids[0][0]      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ token_embedding     │ (None, None,      │ 102,95… │ opt_backbone[0][0]   │
│ (ReversibleEmbeddi… │ 50272)            │         │                      │
└─────────────────────┴───────────────────┴─────────┴──────────────────────┘
 Total params: 3,947,261,954 (14.70 GB)
 Trainable params: 1,315,753,984 (4.90 GB)
 Non-trainable params: 0 (0.00 B)
 Optimizer params: 2,631,507,970 (9.80 GB)
codecov[bot] commented 12 months ago

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.14% :tada:

Comparison is base (d0b53fd) 76.10% compared to head (1689287) 76.25%. Report is 4 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #859 +/- ## ========================================== + Coverage 76.10% 76.25% +0.14% ========================================== Files 329 329 Lines 31270 31276 +6 Branches 6083 6084 +1 ========================================== + Hits 23799 23850 +51 + Misses 5871 5834 -37 + Partials 1600 1592 -8 ``` | [Flag](https://app.codecov.io/gh/keras-team/keras-core/pull/859/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) | Coverage Δ | | |---|---|---| | [keras_core](https://app.codecov.io/gh/keras-team/keras-core/pull/859/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) | `76.16% <100.00%> (+0.14%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files Changed](https://app.codecov.io/gh/keras-team/keras-core/pull/859?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) | Coverage Δ | | |---|---|---| | [keras\_core/utils/summary\_utils.py](https://app.codecov.io/gh/keras-team/keras-core/pull/859?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfY29yZS91dGlscy9zdW1tYXJ5X3V0aWxzLnB5) | `72.72% <100.00%> (+5.23%)` | :arrow_up: | ... and [4 files with indirect coverage changes](https://app.codecov.io/gh/keras-team/keras-core/pull/859/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

qlzh727 commented 12 months ago

Thanks for the PR -- looks good! Please add a unit test.

Sure. Done.