I've been posting some ideas about evolution around the internet, so why not here!
“
Quantization is the enemy of evolution
It is fortunate that biological systems are heavily quantized, especially in bacteria and viruses. An atom is there or not, discrete point mutations are there or not, a plasmid is there or not . If the cost landscape where not so heavily quantized we simply wouldn’t exist. The crossover mechanism higher animals use is a weak optimizer but it does make the cost landscape less rough than what asexual microbes have to contend with.
Hence we can adapt to pathogens despite having a far longer time between generations and a far lower population count. It is also true (I think) that having a larger genome reduces the roughness of the cost landscape by giving more degrees of freedom.
In a non-quantized artificial system a perturbation in any of the basis directions gives a smoothly changing alteration in cost. A mutation in all dimensions gives a new cost that is a summary measure of multiple clues. Following mutations downhill in cost means following multiple clues about which way to go. If there were quantization in many basis directions a small movement in those directions would give you not information about whether such a movement was good or bad. You would get not clues in those directions, less clues overall, which is obviously detrimental.
A point here being that artificial evolution on digital computers can be far more efficient than biological evolutions. If you accept that back propagation is in some sense a form of evolution (at a slight stretch) then you can see that a GPU cluster can build in a few weeks the capacity to do vision that took biological evolution many millions of years to create.
I have some kind of code here:
https://github.com/S6Regen/Thunderbird ”
“Here’s a link for crossover being a weak optimizer:
https://youtu.be/WoamKUfisVM
It is actually to allow non-lethal mixability of traits. And that ends up implementing the multiplicative weights update algorithm, or so they say.
You might ask then, why are fungi not more lethal pathogens given what I said and that they reproduce by crossover. I don’t really know but I presume it has to do with crossover being a weak optimizer and maybe they have a smaller number of genes than a large animal.
A neural network can have squashing activation functions or non-squashing ones.
What I noticed from my experiments with associative memory is that squashing type activation functions result in attractor states/error correction/(soft) quantization. That seems to be difficult for evolution to deal with, especially if you use hard binary threshold activation functions (the ultimate squashing function.)
One the other hand nets with non-squashing activation are very easy to evolve. And result in reasoning in sparse patterns. Of course, just because evolution favors non-squashing activation functions does not mean they are the best possible ones to use. It could be squashing ones are if you had a suitable algorithm.”
There are also "growing" neural networks that that kind of side step some of the problems of evolution by adding additional neurons one at a time. It might be somewhat difficult to build a really big neural network that way.
I've been posting some ideas about evolution around the internet, so why not here!
“ Quantization is the enemy of evolution
It is fortunate that biological systems are heavily quantized, especially in bacteria and viruses. An atom is there or not, discrete point mutations are there or not, a plasmid is there or not . If the cost landscape where not so heavily quantized we simply wouldn’t exist. The crossover mechanism higher animals use is a weak optimizer but it does make the cost landscape less rough than what asexual microbes have to contend with.
Hence we can adapt to pathogens despite having a far longer time between generations and a far lower population count. It is also true (I think) that having a larger genome reduces the roughness of the cost landscape by giving more degrees of freedom.
In a non-quantized artificial system a perturbation in any of the basis directions gives a smoothly changing alteration in cost. A mutation in all dimensions gives a new cost that is a summary measure of multiple clues. Following mutations downhill in cost means following multiple clues about which way to go. If there were quantization in many basis directions a small movement in those directions would give you not information about whether such a movement was good or bad. You would get not clues in those directions, less clues overall, which is obviously detrimental.
A point here being that artificial evolution on digital computers can be far more efficient than biological evolutions. If you accept that back propagation is in some sense a form of evolution (at a slight stretch) then you can see that a GPU cluster can build in a few weeks the capacity to do vision that took biological evolution many millions of years to create. I have some kind of code here: https://github.com/S6Regen/Thunderbird ”
“Here’s a link for crossover being a weak optimizer: https://youtu.be/WoamKUfisVM It is actually to allow non-lethal mixability of traits. And that ends up implementing the multiplicative weights update algorithm, or so they say.
You might ask then, why are fungi not more lethal pathogens given what I said and that they reproduce by crossover. I don’t really know but I presume it has to do with crossover being a weak optimizer and maybe they have a smaller number of genes than a large animal.
A neural network can have squashing activation functions or non-squashing ones. What I noticed from my experiments with associative memory is that squashing type activation functions result in attractor states/error correction/(soft) quantization. That seems to be difficult for evolution to deal with, especially if you use hard binary threshold activation functions (the ultimate squashing function.)
One the other hand nets with non-squashing activation are very easy to evolve. And result in reasoning in sparse patterns. Of course, just because evolution favors non-squashing activation functions does not mean they are the best possible ones to use. It could be squashing ones are if you had a suitable algorithm.”
There are also "growing" neural networks that that kind of side step some of the problems of evolution by adding additional neurons one at a time. It might be somewhat difficult to build a really big neural network that way.