search

SmilingWolf / JAX-CV

Repo for my JAX CV experiments. Mostly revolving around the Danbooru20xx dataset
24 stars 4 forks source link

Vit: Better init for pos_embed weight #6

Closed SmilingWolf closed 1 year ago

SmilingWolf commented 1 year ago

Benefits smaller models in particular. Taken from the implementation in Google's big_vision repo.