mindspore-lab / mindone

one for all, Optimal generator with No Exception
Apache License 2.0
329 stars 63 forks source link

T5 Encoder Model and T5 Model: Graph Mode and Pynative Mode #440

Closed wcrzlh closed 2 months ago

wcrzlh commented 2 months ago

What does this PR do?

This pr is proposed to add T5 Encoder Model and T5Model. They are tested both in graph mode and pynative mode. Google/T5-small and DeepFloyd/T5-v1.1_xxl have been supported.

The T5Model could be utilized conveniently by the following command:

from mindone.transformers.models import T5Model, T5EncoderModel model = T5Model.from_pretrained("Google/T5-small") encoder_model = T5EncoderModel.from_pretrained("DeepFloyd/t5-v1_1-xxl")

Additionally, the feature of loading more than one safe tensors file has beed added.

Fixes # (issue)

Adds # (feature)

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

@xxx