Patch OpenLLaMa tokenizer imports for transformers 4.29 compatibility

facebookresearch / mmf

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

https://mmf.sh/

Other

5.44k stars 925 forks source link

Patch OpenLLaMa tokenizer imports for transformers 4.29 compatibility #1315

Closed ebsmothers closed 1 year ago

ebsmothers commented 1 year ago

Summary: OpenLLaMa tokenizer maps the import path to read from LLaMa's tokenizer; this is incompatible with MMF's transformers patch. Wrap module imports in a try/except block to fix these import errors

Reviewed By: ankitade

Differential Revision: D46283948

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D46283948

facebook-github-bot commented 1 year ago

This pull request has been merged in facebookresearch/mmf@536f5039e80906b4f92df768b31399d40d2bd295.