Closed d-kleine closed 1 week ago
Thank you for this proposal and for offering to create a PR. I agree that checking the value should be an overall improvement. If you work on a PR, please don't hard code the possible values but take them from the enum:
Also, this applies to all PEFT methods, not just LoRA, so ideally the solution would solve this for all these methods.
Feature request
The current implementation of
LoraConfig
in the PEFT library does not validate whether the providedtask_type
is valid. This can lead to silent failures when users accidentally provide a misspelled or unsupported task type. To improve usability and error handling, I propose adding assertions or validation checks to ensure that thetask_type
parameter is one of the supported types.Supported task types include:
SEQ_CLS
: Text classificationSEQ_2_SEQ_LM
: Sequence-to-sequence language modelingCAUSAL_LM
: Causal language modelingTOKEN_CLS
: Token classificationQUESTION_ANS
: Question answeringFEATURE_EXTRACTION
: Feature extractionIf an invalid or misspelled task type is provided, the system should raise a clear error message instead of proceeding silently.
Add a validation check in the
LoraConfig
class to ensure that the providedtask_type
is one of the supported types. If not, raise aValueError
with a clear message indicating that the task type is invalid and listing the supported types.Example implementation:
Motivation
This feature request addresses a usability problem. Currently, if a user provides an invalid or misspelled
task_type
, no error is raised, and the model may behave unexpectedly without any indication of what went wrong. This can lead to confusion and wasted time during debugging. By adding assertions for valid task types, we can prevent such issues and provide immediate feedback to users, improving the overall developer experience.For example, if a user mistakenly sets
task_type="CASUAL_LM"
instead of the correctCAUSAL_LM
, no error is raised, and the model proceeds with incorrect configurations. This results in performance metrics of the model not being computed correctly and will be output asNone
(=empty).Your contribution
I am willing to contribute by submitting a Pull Request (PR) with this feature, but need a confirmation before that it is wanted.
Test code for verification of the issue: