search

pytorch / captum

Model interpretability and understanding for PyTorch
https://captum.ai
BSD 3-Clause "New" or "Revised" License
4.96k stars 499 forks source link

Support Cache Class for New Versions of Transformers Library #1341

Closed craymichael closed 2 months ago

craymichael commented 2 months ago

Summary: Fixes D62210529 (now reverted by D62262760). Transformers library is now an optional dependency. We do not depend on it, however, we have some logic for transformers models here. The library will only be imported if a model already has the library in the corresponding environment. This TARGETS configuration prevents transformers version conflicts which e.g. caused T200877742.

Add support for new transformers Cache objects. This may need changes in the future as it seems that LLMs handle Caching differently. Some handle Caching themselves, however, some of them do not and some of them don't support Caches yet. Llama models seem to have a _supports_cache_class flag that indicates whether this new Cache object is supported. If it isn't marked as supported, we assume it takes legacy format (tuple past values). Multiple checks added to ensure compatibility.

(minor) Also, changed the defaults for LLM generation to dismiss warnings (does not change generation behavior).

Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request was exported from Phabricator. Differential Revision: D62408520

facebook-github-bot commented 2 months ago

This pull request has been merged in pytorch/captum@7b2255059d58b7c4e28a967242b877b5acad1860.