fastpath inference is not supported

mlfoundations / open_clip

An open source implementation of CLIP.

Other

9.14k stars 908 forks source link

fastpath inference is not supported #901

Open bigtree2020 opened 1 week ago

bigtree2020 commented 1 week ago

I found the fastpath inference seems not supported, which is an optimization in https://github.com/mlfoundations/open_clip/blob/main/src/open_clip/transformer.py#L224.

May I know what is the reason to avoid using this in the current impl?

rwightman commented 1 week ago

we kept it in default batch_first=False only because the original was like that, could change it around so that fastpath could work in inference cases...

rwightman commented 5 days ago

I merged low risk changes, but haven't touched the masking which will prevent fastpath from being used for the text transformer still