search

FluxML / model-zoo

Please do not feed the models
https://fluxml.ai/
Other
917 stars 335 forks source link

Project Proposal For Contribution to GSoC'19 #99

Closed shreyas-kowshik closed 2 years ago

shreyas-kowshik commented 5 years ago

Hi I am a GSoC'19 aspirant looking to contribute to the model-zoo of Flux. I have a few ideas in mind for my work and wanted to check the feasibility of it. I was planning to implement Generative models as in CycleGAN, pix2pix, neural image captioning, and an object detection pipeline as in YOLO-v3. I also see that the object detection modules as in Faster-RCNN and mobilenet-SSD have not been implemented and I can also work with them. I have prior experience with generative models and object detection using Deep Learning. @MikeInnes @ViralBShah

ViralBShah commented 5 years ago

Welcome! In general, it is good to have contributions made to model-zoo or Flux or some related repos. Also cc @staticfloat

shreyas-kowshik commented 5 years ago

@ViralBShah Yeah I would be contributing to model-zoo. However are all/any of the models above relevant? Or do have any of these models been implemented in some other repository/ are not of special interest at the moment?

ViralBShah commented 5 years ago

Let's wait for @MikeInnes to chime in. Also @avik-pal could perhaps chime in.

avik-pal commented 5 years ago

CycleGAN, Pix2Pix, and Neural Image Captioning are interesting additions. As for YOLO, someone recently posted about their implementation on slack. Also, someone is working on Mask RCNN, which is essentially an extension of Faster RCNN, for segmentation tasks. So it might not be ideal to spend time on Faster RCNN (we can simply build it using the layers of MaskRCNN).

ViralBShah commented 5 years ago

@dhairyagandhi96 seems to be looking into Mask RCNNs as well.

DhairyaLGandhi commented 5 years ago

Mask RCNN is simply building on top of Faster RCNN, but Mask RCNN can have different versions within itself. YOLO.jl is already a nice implementation of the same.

Welcome to Flux and Julia!

DhairyaLGandhi commented 5 years ago

There are some interesting multi-modal learning applications in the GAN-verse too, if you're interested

shreyas-kowshik commented 5 years ago

@dhairyagandhi96 Can you help me get started with an issue? I was thinking of adding tutorials for the already done work(As in the GAN models built last year) in a detailed jupyter notebook in a format similar to that in tensorflow by addressing #69.

Would that be an important issue to pick at the moment or should I first focus on working on the internals as in the issues in Flux/NNlib?

DhairyaLGandhi commented 5 years ago

Do feel free to take a look at issues as you wish.

69 is in fact something we've been working on. We have notebooks built around for the current models, but a lot of them need explanatory material to go along with them.

I would be interested to see some of those fleshed out as well. Maybe in conjunction with a couple gan implementations?

shreyas-kowshik commented 5 years ago

I'll get to working on the GAN implementation. As for the existing notebooks with explanatory material do you reference this and this repositories?

DhairyaLGandhi commented 5 years ago

Before moving full steam, it'd be useful and important to get Mike's and Elliot's thoughts on this.

shreyas-kowshik commented 5 years ago

@staticfloat I was thinking of adding tutorials for the already done work(As in the GAN models built last year) in a detailed jupyter notebook in a format similar to that in tensorflow and also to add explanatory material to these and these notebooks containing work on tutorials that has already been done. What's your opinion on this?

sidhantnagpal commented 5 years ago

I would like to contribute to CV and RL models as well. It is so nice to train models with Julia/Flux on Google Colab. I have implemented GANs with dense layers and DCGANs in Flux. I was waiting on ConvTranspose which just got merged in https://github.com/FluxML/Flux.jl/pull/311 (thanks to Tejank).

nikunj-taneja commented 5 years ago

Hi @dhairyagandhi96 @staticfloat ! I am interested in the Port ML Tutorials project as well. I was thinking of porting the tutorials from Udacity and Stanford 231n. I could start with simple neural networks and then move on to CNNs, RNNs/LSTMs and then maybe GANs. The notebooks will be theoretical explanation + Flux code, assuming basic knowledge of Julia and Flux. Existing notebooks could be incorporated in the same. What do you think about this?

shreyas-kowshik commented 5 years ago

I have been contributing to model-zoo for sometime now...These are the PR's I've made so far : https://github.com/FluxML/model-zoo/pull/111 https://github.com/FluxML/model-zoo/pull/98 https://github.com/FluxML/model-zoo/pull/82 I was thinking of starting to work upon the proposal along with solving issues parallelly. For that I wanted to know what all models are effectively required to be implemented in terms of priority for now.

I have the following in mind : pix2pix, cycleGAN, neural-image-captioning, generative-adversarial-text-to-image-synthesis, Proximal Policy Optimization For Policy Gradients, Mobilenet-Single-Shot-Detector, Deep Recurrent Q Networks...Some of these models like Mobilenet-SSD are part of tensorflow and have pretrained weights to be used on the fly. So I think adding them should be highly beneficial.

The image captioning model would utilise the Resnet layers that have been built and I see that even this is a basic module present in many frameworks by default.These ideas were somewhat inspired from : https://github.com/tensorflow/models/tree/master/research

What are your views on priority/variety or on the overall proposition...? @staticfloat @MikeInnes @ViralBShah @dhairyagandhi96 @avik-pal

MikeInnes commented 5 years ago

It would be nice to get one of these PRs polished and merged. I suggest focusing on one only and addressing feedback so that we can get it in. Do ask if you aren't sure what needs to be done.

shreyas-kowshik commented 5 years ago

Thanks for the reply. I am working on the PRs and have made the changes requested so far. I'll prioritize them one by one to get them polished and merged.

ToucheSir commented 2 years ago

Cleaning up this old issue for the same reason as https://github.com/FluxML/model-zoo/issues/114#issuecomment-1104604644.