search

keras-team / keras

Deep Learning for humans
http://keras.io/
Apache License 2.0
61.98k stars 19.46k forks source link

Automatic new type of layer creation mechanism for Keras #11717

Closed andreiliphd closed 5 years ago

andreiliphd commented 5 years ago

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or on the Keras Slack channel instead of opening a GitHub issue.

Thank you!

IDEA: I want to offer you a method that put production of a new type of layers on an industrial scale.

PROBLEM: How to turn a neural network in mathematical formula suited for a creation of an absolutely new type of layer. It might be used in Keras, TensorFlow, and PyTorch or anything.

HOW: 1, You trained your network for particular industry application and invested a lot of computational power to make it work. And this neural network work for solving a particular problem in a particular industry very well.

  1. You freeze all layers in your neural network.
  2. Let's suppose we have an MSE loss for a simplicity.
  3. Then you start to training target(label) looking for INTEGRAL of a function of your neural network. It will produce a mathematical formula describing a neural network. Let's call ANTI GRAD algorithm.
  4. After it's done you have a mathematical formula that can easily be turned into a new type of layer for particular industry application with a simple(you can tune simplicity) formula for Chemistry, Economics, Medicine.

APPLICATION:

  1. Creation of new type of layers suited for particular industries and application.
  2. Research work can be enhanced. We are turning a neural network into the classic mathematical language.
gabrieldemarmiesse commented 5 years ago

It seems ambitious, I would recommend considering making a fork of keras if there are going to be many changes. If you are convinced that this should go in core keras, please submit an API design proposal as mentioned in the CONTRIBUTING.md. Thank you!

andreiliphd commented 5 years ago

I don't think that we should completely change an API for that. Let's make a class or two that handles this problem and work on it. So we should make an ANTI-GRAD first. This is an algorithm that searches integral for labels. The first question what is input for that? Out neural network architecture? We can try that. But I don't want to overcomplicate the formula Let's make it simple and suited for creation of new layer that doesn't take years of computational power. And we have to find a balance between a crazy formula that we will receive as an output of training labels looking for an integral and simple formula. We have to make it simple and suitable for production. This approach will create an absolutely new market: market of layers for neural networks for Oil and gas, Manufacturing, Retail, Investment and etc. So, the economic reason behind the implementation of this algorithm as a whole is pretty clear. So, the idea started from me thinking about layers market shop where you can buy on the website new layer suited for particular industry. But the main problem that I see

  1. Is to create an ANTI GRAD algorithm.
  2. Is to control simplicity of the formula.
  3. Is to make possible to train labels, not a neural network. So, maybe, it's a big task but the output is really worth it. So, the first thing is that we have to do is to define a plan for implementation of this algorithm.
    • Make it possible to train labels. So, Keras need a compilation of the model. We have to compile our training labels architecture to make a basis.
    • Methods that we will use to find an integral of a function. Some search algorithm.
    • Optimizer for that, what we are going to optimize and how.
farizrahman4u commented 5 years ago

I don't believe this is the most suitable place to have this discussion. I think you are better off implementing a prototype of your algorithm in numpy (as it doesn't seem you benefit much from auto grad) and maybe write a paper :)