Add GeGLU activation and change activation type to activation config

google-ai-edge / ai-edge-torch

Supporting PyTorch models with the Google AI Edge TFLite runtime.

Apache License 2.0

199 stars 23 forks source link

Add GeGLU activation and change activation type to activation config #42

Closed yichunk closed 1 month ago

yichunk commented 1 month ago

Add GeGLU activation and change activation type to activation config for model_config.

GeGLU activation is used in diffusion model. And GeGLU has parameters: https://paperswithcode.com/method/geglu Change activation type to activation config in all model_configs class.

BUG=b/311216181

haozha111 commented 1 month ago

nice cleanup.