search

XiangLi1999 / ContrastiveDecoding

contrastive decoding
173 stars 11 forks source link

Why change code in the whole Transformers library? #5

Closed ghosthamlet closed 1 year ago

ghosthamlet commented 1 year ago

Thanks for open source the Contrastive Decoding, it is really an interest and smart method. I found you have copied the whole Transformers library, and just changed around two files. Why not use python dynamic binding to patch Transformers generation methods? I think it will be easier to maintain and debug. e.g.

import torch
import transformers.generation_utils as gu

@torch.no_grad()
def generate(self, ...):
    ...

@torch.no_grad()
def sample(self, ...):
    ...

gu.GenerationMixin.generate = generate
gu.GenerationMixin.sample = sample
XiangLi1999 commented 1 year ago

Hi,

Thanks a lot for raising this suggestions! I totally agree. I will incorporate this in the next (cleaned up) version of my code release!

Best, Lisa

ghosthamlet commented 1 year ago

You are welcome. Looking forward to the new version.