Got Rejected

[loliverhennigh]

I, like 613 other people, was not accepted into this program. The main thing this camp really offers is mentorship and advice from intelligent and well experienced people. While I am certainly not as experienced as the mentors in this program I would like to offer my mentorship to others that have also been rejected. I am still planning on moving forward with the project I proposed and I suspect many others are as well. Maybe a few of you have projects in areas that I happen to be knowledgeable about (see bellow). If so, maybe I can help and give some advice. If you are interested in talking, send me a message. I also included my proposal in case anyone was interested.

Things I know about, Fast Object Detection. I worked for some time training models similar to YOLO on a variety of datasets. Running models on little computers like TX1 and raspberry pi. Reinforcement Learning. I haven't done anything in RL for about 6 months but I use to spend quite a bit of time on this. Applications of Deep learning to Physics simulations. See my proposal :). This is what I do a lot of now.

My proposal,

I propose to expand the work seen in the paper "Convolutional Neural Network for Steady Flow Approximation". In this paper they investigate predicting the steady state fluid flow around complex objects using neural networks. By doing so, they are able to accurately predict fluid flow with significantly less computation time then other flow solvers (100 times faster). This offers many possibilities including automated design and giving researchers with small computational resources access to high quality simulations. Because of the potential impact of this research area and the relatively little open source code available, I propose to work on this during my time at ML Jeju Camp.

Proposed Areas of Work

In the past month I have taken it upon myself to reimplement the original paper and open source the code (https://github.com/loliverhennigh/Steady-State-Flow-With-Neural-Nets). I have done this to familiarize myself with the subject manner and more accurately lay out modifications and extensions to the original work. The following contains a curated list of such modifications. I feel that tackling these with the assistance and resources provided at this camp would be more then possible.

• Heavily improved network architecture. In my implementation of this paper I applied residual U-networks and found that higher accuracy with faster compute times could be achieved. I feel that further improvements could be obtained by borrowing ideas seen in image segmentation and real time style transfer work.
• Much larger datasets. The original paper only shows results on 2D datasets of size 128x256 and 3D datasets of 32x32x32. I would like to test the method on much larger datasets as well as more complex geometries. A stretch goal is test this method on more complex partial differential equations such as multiphase flow or adding heat convection.
• In the paper "Accelerating Eulerian Fluid Simulations With Convolutional Networks" a neural network learn to predict time dependent flow. A significant aspect of this work is that the loss function is entirely unsupervised and just learns to minimize the divergence of the velocity vector field. There is no reason that such an approach could not be used when predicting the steady state flow and doing so would have the desirable effect of not needing a training set and learning more physically realistic flows. For these reasons, I would like to implement this capability.
• Condense project into simple and usable python package. The truly exciting thing about combining Deep Learning and Physics simulations is that it has the potential to give anyone with limited computational resources access to fast and accurate physics simulations. In a sense, democratizing this area of research. Because of this, I feel strongly motivated to consolidate this project into a powerful library with pretrained models and a usable interface. My goal is to create a demo with the library showing how a huge 3D 256x256x256 simulation can be run on complex objects with only a few lines of code.
• Optimize Boundary Conditions using loss from trained model. In the original style transfer work, the pixel values of an image are optimized to a loss produced by a neural network. A similar approach can be used where the boundary is optimized to a loss calculated by the flow predicting network. In this case, the loss can be horizontal drag or vertical lift of an object. I have proven this method to be successful on a few toy problems. Here is a little video demonstrating this (https://www.youtube.com/watch?v=AFAT0Izo1AM). This video shows the boundary being trained to produce low horizontal drag and high vertical lift. There are many improvements I can make to this that I will explore over the course of this project.
• Developing network architectures with low memory requirements. As I have experienced first hand in other work, memory usage becomes a key concern in 3D fluid simulations. This is because complex boundaries require high grid resolution to be expressed however the memory for such grids grow cubically. For example, a 1000x1000x1000 grid simulated with a typical Lattice Boltzmann solver requires around 150 gigabytes of working GPU memory. Because of this, I am working on neural network architectures that are memory efficient. The goal is to have a network that requires less memory to run then the memory of the flows velocity vector field. This might seem impossible however by breaking up the boundary into pieces and feeding them into the network sequentially as well as generating pieces of the predicted flow sequentially, a network can be constructed with significantly less working memory. I have yet to demonstrate whether this technique is really effective and so this is a very high risk aspect of my project. That being said, I have attempted a somewhat similar idea using Pixel-cnn to some success (https://github.com/loliverhennigh/pixel-cnn).

Bigger picture

The application of Deep Learning to Physics simulations is a fast growing area. Deep learning has proven a powerful surrogate model for simulations and offers a potential solution to the big data problems facing the scientific computing world today. I invite the reader to look at "CFD Vision 2030 Study: A Path to Revolutionary Computational Aerosciences" to get a sense of some of the current and future challenges. While my proposed work is only one small piece of a much bigger picture, I feel strongly that creating a well written open source code base that tackles this problem would provide both a powerful tool for researchers studying computational fluid dynamics and a good starting point for future applications of Deep Learning to Computational Physics.

[kaykim]

@loliverhennigh Sorry to hear the news. Your project seems interesting, though.

By the way, have your gotten the reject email? A friend of mine is one of the applicants, but he said he couldn't receive the notification. Even in Spam folder.

[loliverhennigh]

Thanks! I did receive the reject email. I have gmail and it showed up in the promotion folder.

[chg0901]

I think your proposal is much better than mine.

I am interested in the proposal of guys who are selected.