How do I set d_model, q, v, h, N, dropout, attention_size value?

maxjcohen / transformer

Implementation of Transformer model (originally from Attention is All You Need) applied to Time Series.

https://timeseriestransformer.readthedocs.io/en/latest/

GNU General Public License v3.0

852 stars 166 forks source link

How do I set d_model, q, v, h, N, dropout, attention_size value? #56

Closed mmsx closed 1 year ago

mmsx commented 2 years ago

Hi, I have a question. If I want to use the transformer part to deal with my image datasets, the datasets from high to 256 pixels and width to 128 pixels images, images in each group have 3000 pieces, how do I set d_model, q, v, h, N, dropout, attention_size value?

maxjcohen commented 1 year ago

Hi, this repo is focused on applying the Transformer for time series modelling. While you technically could use our implementation for computer vision, there are better Transformer implementations for this use case.