Closed rvashurin closed 4 months ago
I suggest removing the max_length
parameter everywhere since we override it with max_new_tokens
. Also, I suggest removing min_length
in greedy_tokens.py or changing it to min_new_tokens
, since it also overrides with a value equal to the input length.
@ArtemVazh max_length removed, but I suggest we do min_length stuff separately to not overload this PR with concepts and code.
Several improvements here:
device_map=auto
feature ofaccelerate
by removing all.to(device)
calls to the model.use_auth_token
.Completes following Notion tasks: