‘out’ is used in the _pulidattention, so out will be updated after out = out + callback (out, q, k, v, extra_options, ** self.kwargs [i]) . If connected to ipadapter, the order before and after will cause differences in effect. It is recommended to save the callback results to the list and add them up at the end, which is not affected by the order before and after.
‘out’ is used in the _pulidattention, so out will be updated after
out = out + callback (out, q, k, v, extra_options, ** self.kwargs [i])
. If connected to ipadapter, the order before and after will cause differences in effect. It is recommended to save the callback results to the list and add them up at the end, which is not affected by the order before and after.