search

robmarkcole / fastpages-blog

My blog
http://robmarkcole.com/
Apache License 2.0
2 stars 0 forks source link

Transformers in remote sensing | Blog of Robin Cole #5

Open utterances-bot opened 2 years ago

utterances-bot commented 2 years ago

Transformers in remote sensing | Blog of Robin Cole

Introduction to Transformers and why you should care

https://robmarkcole.com/markdown/2022/08/15/transformers.html

wgcban commented 2 years ago

Nice article & thanks. Here is the another recent work on transformers for remote sensing change detection: https://github.com/wgcban/ChangeFormer. We simplify the attention mechanism using sequence reduction process, so that it is much faster to train. Further, by design, it utilizes convolutions for positional embedding, which allows images with different resolution during testing.

robmarkcole commented 2 years ago

Thanks for sharing and providing a nice summary. You touch on an interesting technical point, the use of positional embeddings typically limits the range of resolutions that can be used with a transformer?