Try using other network architectures

[iamvc7]

@marco-c I am planning to add, Resnet along with a siamese module as class now. Later I will proceed using pretrained models of these networks (Vgg, Resnet) and compare the accuracies. Then we will have a better picture about how to proceed (As I said in #2). If you have something specific in mind, please let me know.

[nok]

We can test pre-defined and later fine-tuned networks, but I'm concerned about the size of the input layer, because a small length could distort and skew the relevant visual representation of the bug.

[iamvc7]

@nok Yes, I understand. To detect differences, Y+D and N in a better way or even Y and D+N finding attention based ROIs and feeding those patches to the NN would be a good idea. Because, some times the advertisements act as blocks and also grab most of the visual representation. Although this can be considered as an add-on to improve accuracy after using pre-trained and fine tuned networks. I guess, this method can also act as a solution to the case you mentioned.

[ManasviKundalia]

@marco-c Do you have any specific architecture in mind? Or perhaps we could add support for attention based layers ?

[hardik124]

@marco-c Do you think a YOLO type network will be good to identify relevant ROIs and then classify them? I am new to this repository, so please correct me if I am wrong.

[marco-c]

@marco-c Do you have any specific architecture in mind? Or perhaps we could add support for attention based layers ?

@marco-c Do you think a YOLO type network will be good to identify relevant ROIs and then classify them? I am new to this repository, so please correct me if I am wrong.

I don't have specific ideas, we should just add whatever we think might work for us and then compare the results.

[hardik124]

@marco-c , Is there a documentation I can read? I have very little knowledge of this repository ,but if we target some rendering glitches or bugs, we can use YOLO type network.

[marco-c]

@marco-c , Is there a documentation I can read?

I'm going to improve the documentation (https://github.com/marco-c/autowebcompat/issues/77). At the moment, the documentation is the code (which is not too much anyway).

I have very little knowledge of this repository ,but if we target some rendering glitches or bugs, we can use YOLO type network.

Sounds good to me. I have never used them myself, but it's worth testing them and comparing them with other solutions.

[marco-c]

@marco-c , Is there a documentation I can read?

I've updated README to add more documentation. Feel free to ask if you still have some doubts.

[hardik124]

@marco-c . Thanks.

[Trion129]

I found this architecture Simnet by Amazon Development Services, it uses a variation of Siamese network, they use 2 extra shallower models trained on downsampled images alongside the ImageNet, The final results are better. https://arxiv.org/pdf/1709.08761.pdf

I can try implementing this, what are your thoughts?

[marco-c]

@Trion129 sounds good, please file an issue (Try using Simnet architecture) with the same description you put in your comment and add a reference to this issue.

[Shashi456]

@marco-c should we try out vgg19 , the microsoft resnet and other forms of inception later on ? i mean can we add them in networks.py?

[marco-c]

Sure, we can add them.

[marco-c]

Closing in favor of https://github.com/marco-c/autowebcompat/issues/194.