search

pytorch / captum

Model interpretability and understanding for PyTorch
https://captum.ai
BSD 3-Clause "New" or "Revised" License
4.93k stars 499 forks source link

Add unit test verifying compatibility with huggingface models #1352

Closed DianjingLiu closed 1 month ago

DianjingLiu commented 2 months ago

Summary: Our current unit tests for LLM Attribution use mocked models which are similar to huggingface transformer models (e.g. Llama, Llama2), but may have some unexpected differences such as this. To validate coverage and ensure compatibility with future changes to models, we would like to add tests using huggingface models directly and validate compatibility with LLM Attribution, which will help us quickly catch any breaking changes.

So far we only test for model type LlamaForCausalLM

Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 1 month ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 1 month ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 1 month ago

This pull request was exported from Phabricator. Differential Revision: D62894898

facebook-github-bot commented 1 month ago

This pull request has been merged in pytorch/captum@4f8caeb920486cff6d6dc949cae8d9d74a82bd81.