Add support for only the final task-specific layers in eval script

[yongzx]

No longer makes sense to add task-adapters since we are comparing against different parameter-efficient FT methods.