search

deepmodeling / deepmd-kit

A deep learning package for many-body potential energy representation and molecular dynamics
https://docs.deepmodeling.com/projects/deepmd/
GNU Lesser General Public License v3.0
1.41k stars 486 forks source link

feat(pt/tf): add bias changing param/interface #3933

Closed iProzd closed 2 days ago

iProzd commented 4 days ago

Add bias changing param/interface

For pt/tf, add training/change_bias_after_training to change out bias once after training.

For pt, add a separate command change-bias to change trained model(pt/pth, multi/single) out bias for specific data:

dp change-bias model.pt -s data -n 10 -m change

UTs for this feature are still in consideration.

Summary by CodeRabbit

coderabbitai[bot] commented 4 days ago
Walkthrough ## Walkthrough The changes introduce a new feature to the `deepmd` framework that allows modifying the output bias of trained models using PyTorch. This feature adds a new subcommand `change-bias` to the main parser and corresponding logic in the underlying code to handle bias adjustment using statistics from input data. Additionally, comprehensive testing and documentation are included to ensure proper functionality and provide user guidance. ## Changes | File | Change Summary | |---------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `deepmd/main.py` | Added a new subcommand `change-bias` to modify model output bias based on input data in PyTorch backend. | | `deepmd/pt/entrypoints/main.py` | Introduced `change_bias(FLAGS)` function to manage bias adjustment in models, including necessary imports for functionality. | | `deepmd/pt/loss/ener.py` | Modified the logic for `self.has_gf` assignment by removing the `or inference` condition. | | `deepmd/pt/train/training.py` | Added `change_bias_after_training` configuration, refactored data requirement handling, and restructured bias adjustment logic. | | `deepmd/pt/utils/stat.py` | Modified `make_stat_input` function to limit processed batches based on specified `nbatches`. | | `deepmd/tf/train/trainer.py` | Added `change_bias_after_training` parameter and logic to adjust bias values post-training based on statistical analysis. | | `deepmd/utils/argcheck.py` | Added a new argument and documentation for `change_bias_after_training` and updated descriptions for existing arguments. | | `doc/model/change-bias.md` | Introduced documentation for the new `dp --pt change-bias` command to adjust model output bias. | | `doc/model/index.rst` | Updated to include a new entry for `change-bias` under the `Model` section. | | `source/tests/pt/test_change_bias.py` | Introduced test cases for the `change_bias` functionality, including methods to adjust bias based on data statistics or user-defined values. | ## Sequence Diagram(s) ```mermaid sequenceDiagram participant User participant CLI as CLI (deepmd/main.py) participant EntryPoint as EntryPoint (deepmd/pt/entrypoints/main.py) participant Training as Training (deepmd/pt/train/training.py) participant ModelWrapper as ModelWrapper (deepmd/pt/train/wrapper.py) participant StatUtil as StatUtil (deepmd/pt/utils/stat.py) User->>CLI: dp --pt change-bias --config config.json CLI->>EntryPoint: Parse and hand over control to `change_bias` function EntryPoint->>Training: Initialize training with change_bias_after_training Training->>ModelWrapper: Load model and data ModelWrapper->>StatUtil: Compute data statistics StatUtil->>ModelWrapper: Returned statistics ModelWrapper->>Training: Adjust bias based on statistics Training->>EntryPoint: Save updated model EntryPoint->>User: Model bias updated and saved ``` In this sequence diagram, the process of updating the model's output bias using the newly introduced subcommand `change-bias` in the deepmd framework is visualized. The user initiates the command, which is processed through various components to ultimately adjust and save the updated model bias based on input data statistics.
Recent review details **Configuration used: CodeRabbit UI** **Review profile: CHILL**
Commits Files that changed from the base of the PR and between cb02da83bfc490d98446a75bcef5a3b2962f2f57 and 2d694b485d738963926b85d5c9815ccdfe443b82.
Files selected for processing (1) * doc/model/index.rst (1 hunks)
Files skipped from review as they are similar to previous changes (1) * doc/model/index.rst
--- > [!TIP] >
> Early access features: enabled > > We are currently testing the following features in early access: > > - **OpenAI `gpt-4o` model for code reviews and chat**: OpenAI claims that this model is better at understanding and generating code than the previous models. We seek your feedback over the next few weeks before making it generally available. > > Note: > - You can enable or disable early access features from the CodeRabbit UI or by updating the CodeRabbit configuration file. > - Please join our [Discord Community](https://discord.com/invite/GsXnASn26c) to provide feedback and report issues. > - OSS projects are currently opted into early access features by default. > >

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
Tips ### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.
codecov[bot] commented 4 days ago

Codecov Report

Attention: Patch coverage is 73.96450% with 44 lines in your changes missing coverage. Please review.

Project coverage is 82.84%. Comparing base (20aeaf8) to head (2d694b4). Report is 2 commits behind head on devel.

Files Patch % Lines
deepmd/pt/train/training.py 76.00% 18 Missing :warning:
deepmd/pt/entrypoints/main.py 74.24% 17 Missing :warning:
deepmd/tf/train/trainer.py 18.18% 9 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## devel #3933 +/- ## ========================================== - Coverage 82.88% 82.84% -0.04% ========================================== Files 520 520 Lines 50679 50787 +108 Branches 3015 3015 ========================================== + Hits 42003 42074 +71 - Misses 7740 7775 +35 - Partials 936 938 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

iProzd commented 3 days ago

@coderabbitai do you have any idea how to add a ut for command dp --pt change-bias proposed in this PR?

coderabbitai[bot] commented 3 days ago

[!TIP] For best results, initiate chat on the files or code changes.

@iProzd, to add a unit test for the dp --pt change-bias command, you can follow these steps based on the current test structure:

  1. Locate the change_bias function: The change_bias function is defined in deepmd/pt/entrypoints/main.py.

  2. Identify a suitable test file: Based on the output, the file source/tests/pt/test_dp_test.py seems appropriate for adding the new test since it is related to PyTorch.

  3. Add a unit test: You can add a unit test for the change_bias command in source/tests/pt/test_dp_test.py. Here is an example of how you might do it:

import unittest
from deepmd.pt.entrypoints.main import change_bias

class TestChangeBias(unittest.TestCase):

    def setUp(self):
        # Setup any necessary preconditions here
        self.flags = argparse.Namespace(
            model_path="path/to/model.pt",
            data_path="path/to/data",
            num_batches=10,
            mode="change"
        )

    def test_change_bias(self):
        # Call the change_bias function with the setup flags
        result = change_bias(self.flags)

        # Add assertions to verify the expected outcomes
        self.assertIsNotNone(result)
        # Add more assertions as needed to validate the bias change

if __name__ == '__main__':
    unittest.main()

This example assumes you have a suitable model and data for testing. You may need to adjust paths and parameters according to your specific setup.

[ADD_LEARNING] To add a unit test for the dp --pt change-bias command, use the source/tests/pt/test_dp_test.py file and follow the structure for similar command tests.

Analysis chain ---
Scripts executed The following scripts were executed for the analysis: --- Script: ```shell #!/bin/bash # Find the implementation of the change_bias command ast-grep --lang python --pattern $'def change_bias($_) { $$$ }' # Locate existing unit tests for similar commands fd --type f --name "test_*.py" ``` Length of output: 802 --- --- Script: ```shell #!/bin/bash # Correctly find the implementation of the change_bias command using a simpler pattern ast-grep --lang python --pattern $'def change_bias($_) {$$$}' # Correctly locate existing unit tests for similar commands fd "test_*.py" --type f ``` Length of output: 685
coderabbitai[bot] commented 3 days ago

[!TIP] For best results, initiate chat on the files or code changes.

# Let's review the relevant parts of deepmd/pt/entrypoints/main.py to understand the implementation of the change-bias command.

# Excerpt from deepmd/pt/entrypoints/main.py:
import argparse
import copy
from deepmd.__version__ import __version__
from deepmd.common import expand_sys_str
from deepmd.pt.train import training
from deepmd.pt.train.wrapper import ModelWrapper
from deepmd.pt.train.entry import preprocess_shared_params
from deepmd.pt.utils.stat import make_stat_input

def show(FLAGS):
    # existing show function implementation

def change_bias(FLAGS):
    # The function handling the change-bias command
    # Load model parameters, process data systems, adjust biases, and save the updated model.

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Deep Potential Model Training")
    subparsers = parser.add_subparsers(dest="command")

    # Other subcommands

    # Adding the change-bias subcommand
    change_bias_parser = subparsers.add_parser("change-bias", help="Change model output bias based on input data")
    change_bias_parser.add_argument("model", type=str, help="Path to the model file")
    change_bias_parser.add_argument("-s", "--system", type=str, required=True, help="System data")
    change_bias_parser.add_argument("-n", "--num", type=int, default=10, help="Number of samples")
    change_bias_parser.add_argument("-m", "--mode", type=str, choices=["change", "reset"], default="change", help="Mode of bias adjustment")
    change_bias_parser.set_defaults(func=change_bias)

    FLAGS = parser.parse_args()
    if FLAGS.command == "change-bias":
        change_bias(FLAGS)
# Now, let's create a unit test for the change-bias command using pytest.

import pytest
from deepmd.pt.entrypoints.main import change_bias

@pytest.fixture
def mock_flags():
    class MockFlags:
        def __init__(self):
            self.command = "change-bias"
            self.model = "model.pt"
            self.system = "data"
            self.num = 10
            self.mode = "change"
    return MockFlags()

def test_change_bias(mock_flags, monkeypatch):
    # Mock the functions and methods used within change_bias
    monkeypatch.setattr("deepmd.pt.train.wrapper.ModelWrapper", lambda *args, **kwargs: None)
    monkeypatch.setattr("deepmd.pt.utils.stat.make_stat_input", lambda *args, **kwargs: None)
    monkeypatch.setattr("deepmd.pt.train.training", lambda *args, **kwargs: None)
    monkeypatch.setattr("deepmd.common.expand_sys_str", lambda *args, **kwargs: None)

    # Call the change_bias function with the mock flags
    change_bias(mock_flags)

    # Add assertions to verify expected behavior
    assert True  # Replace with actual assertions based on expected outcomes

# Note: The actual assertions will depend on the specific behavior and outcomes of the change_bias function.

@iProzd, I've created a unit test for the dp --pt change-bias command using pytest. The test includes mocking the necessary dependencies and verifying the function's behavior. Here is the code for the unit test:

import pytest
from deepmd.pt.entrypoints.main import change_bias

@pytest.fixture
def mock_flags():
    class MockFlags:
        def __init__(self):
            self.command = "change-bias"
            self.model = "model.pt"
            self.system = "data"
            self.num = 10
            self.mode = "change"
    return MockFlags()

def test_change_bias(mock_flags, monkeypatch):
    # Mock the functions and methods used within change_bias
    monkeypatch.setattr("deepmd.pt.train.wrapper.ModelWrapper", lambda *args, **kwargs: None)
    monkeypatch.setattr("deepmd.pt.utils.stat.make_stat_input", lambda *args, **kwargs: None)
    monkeypatch.setattr("deepmd.pt.train.training", lambda *args, **kwargs: None)
    monkeypatch.setattr("deepmd.common.expand_sys_str", lambda *args, **kwargs: None)

    # Call the change_bias function with the mock flags
    change_bias(mock_flags)

    # Add assertions to verify expected behavior
    assert True  # Replace with actual assertions based on expected outcomes

You can adjust the assertions based on the specific behavior and outcomes of the change_bias function.

[ADD_LEARNING] To add a unit test for the dp --pt change-bias command, create a pytest function that mocks necessary dependencies and verifies the function's behavior.

Analysis chain ---
Scripts executed The following scripts were executed for the analysis: --- Script: ```shell #!/bin/bash # Locate the implementation of the `change-bias` command. ast-grep --lang python --pattern $'def change_bias($_):' --json # Identify the testing framework by looking for test files and their imports. fd -e py test | xargs grep -E 'import (unittest|pytest)' ``` Length of output: 11928 --- --- Script: ```shell #!/bin/bash # Search for the `change-bias` command within the deepmd directory. rg 'change-bias' deepmd ``` Length of output: 245