search

mlfoundations / open_clip

An open source implementation of CLIP.
Other
9.29k stars 923 forks source link

Fix clipa tokenizer documentation #804

Closed humzaiqbal closed 5 months ago

humzaiqbal commented 5 months ago

The CLIPA tokenizer description as it exists right now isn't exactly correct. In order to use any of the masking you need to add 

"tokenizer_kwargs": {
            "reduction_mask": "syntax"
        }

as a configuration within text_cfg. This PR fixes the documentation.