Closed AshStuff closed 2 months ago
I see that there is a scatter_reduce function at https://github.com/dbolya/tomesd/blob/main/tomesd/merge.py#L105 which sums dst token and src token. However by default, the scatter_reduce function from pytorch https://pytorch.org/docs/stable/generated/torch.scatter_reduce.html has include_self=True. This tries to add the first r slices of dst as well. Do you think we need to set include_self=False?
include_self=True
r
dst
include_self=False
I see that there is a scatter_reduce function at https://github.com/dbolya/tomesd/blob/main/tomesd/merge.py#L105 which sums dst token and src token. However by default, the scatter_reduce function from pytorch https://pytorch.org/docs/stable/generated/torch.scatter_reduce.html has
include_self=True. This tries to add the firstrslices ofdstas well. Do you think we need to setinclude_self=False?