search

zhongkaifu / Seq2SeqSharp

Seq2SeqSharp is a tensor based fast & flexible deep neural network framework written by .NET (C#). It has many highlighted features, such as automatic differentiation, different network types (Transformer, LSTM, BiLSTM and so on), multi-GPUs supported, cross-platforms (Windows, Linux, x86, x64, ARM), multimodal model for text and images and so on.
Other
200 stars 37 forks source link

Question: How to use Seq2SeqSharp for image classification? #89

Open WinInsider opened 4 months ago

WinInsider commented 4 months ago

I am a bit a a lose, as to how to use Seq2SeqSharp for image classification... perhaps is not possible?

First step, as I understand for image classification is to extract features of the image (usually a float or a double array), for each set of image for given label (category). Then build model. Subsequently use new image against model, to obtain closest matching label/category.

Where should I look for an example.

zhongkaifu commented 4 months ago

Hi @WinInsider

You could check this tool: https://github.com/zhongkaifu/Seq2SeqSharp/tree/master/Tools/SeqClassificationConsole SeqClassificationConsole uses embeddings of tokens as input. For image, you could project the entire image as smaller block (by CNN, sliding window, or other methods) and each block could be considered as a token, and then send them to the tool as input.

Thanks Zhongkai Fu