-
Your coatnet seems to only create 1 bias weight set when the paper says 1 per head
 in the paper, strides=2 is used in the first conv layer down-sampling in MBConv.
However, in [line 78-80 of coatnet.py](https://github.com/leondgarse/keras_cv_attention_models/blob/mai…
-
In the [timm](https://github.com/rwightman/pytorch-image-models#models) package, it provides some soft attention modules to building network blocks and I think it's a good fit here, for example:
- …
innat updated
2 years ago
-
Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper_gather)
/CoAtNet/nn/Transformer.py", line…
-
In [Line 44](https://github.com/leondgarse/keras_cv_attention_models/blob/2ed7ae66b55b25dcbc07995049a21eb620787aa0/keras_cv_attention_models/beit/beit.py#L44) of beit.py, you use `tf.meshgrid(range(he…
-
CoAtNet: Marrying Convolution and Attention for All Data Sizes
https://arxiv.org/pdf/2106.04803.pdf
-
Hello.I really aprreciate for your project.
However, The following error occurs when a 512-size image is input at `Attention` class.
> dots = dots + relative_bias
RuntimeError: The size of te…
-
최적 모델 도출을 위한 실험 페이지입니다
model= 사용한 모델
batch_size= 배치 사이즈
epchs= 에포크 횟수
--- 그 외에 새로 변경한 Parameter Set---
- 최종 train accuracy / valid accuracy
> 왜 해당 파라미터들을 변경했는지에 대한 이유와 개선점 등 코멘트
-
Hey, I tried with your implementation, and I found the calculated #param is a little bit different from the paper, and I am curious about the reason, could you please help me out?
Take coatnet_0 fo…
-
Hi, I can't found the code about stochastic depth in your implementation.
And I add the stochastic depth code and train a CoAtNet-Tiny on ImageNet 1k, but got 79.27%@top1.
Have you reproduce th…