asksak commented 4 years ago

Good Day,

In the update of Karoo in 2017, it mentions the inclusion of >,<,<=,>=. However, the latest karoo GP does not seem to accept these functions.

Please excuse me if this question is a repeat, but I am finding it hard to run my project without such functions, will they be included?

kstaats commented 4 years ago

On 1/15/20 2:08 AM, asksak wrote:

Good Day,

In the update of Karoo in 2017, it mentions the inclusion of >,<,<=,>=. However, the latest karoo GP does not seem to accept these functions.

Please excuse me if this question is a repeat, but I am finding it hard to run my project without such functions, will they be included?

I appreciate your reaching out. I will run an experiment to test.

kai

asksak commented 4 years ago

As you asked in the email, here is the list of installed modules: (any commnets)

asksak commented 4 years ago

and here is the run error when i use classify with an added < function with arity 2

Select (c)lassification, (r)egression, (m)atching, or (p)lay (default m): c Select (f)ull, (g)row, or (r)amped 50/50 method (default r): g Enter depth of the initial population of Trees (default 3): Enter maximum Tree depth (default 3): Enter minimum number of nodes for any given Tree (default 3; max 15): Enter number of Trees in each population (default 100): Enter max number of generations (default 10): Display (i)nteractive, (g)eneration, (m)iminal, (s)ilent, or (d)e(b)ug (default m): s

We have constructed the first, stochastic population of 100 Trees

Evaluate the first generation of Trees ... Traceback (most recent call last): File "karoo_gp.py", line 251, in gp.fx_karoo_gp(kernel, tree_type, tree_depth_base, tree_depth_max, tree_depth_min, tree_pop_max, gen_max, tourn_size, filename, evolve_repro, evolve_point, evolve_branch, evolve_cross, display, precision, swim, mode) File "modules/karoo_gp_base_class.py", line 188, in fx_karoo_gp self.fx_fitness_gym(self.population_a) # generate expression, evaluate fitness, compare fitness File "modules/karoo_gp_base_class.py", line 1162, in fx_fitness_gym result = self.fx_fitness_eval(expr, self.data_train) File "modules/karoo_gp_base_class.py", line 1278, in fx_fitness_eval rule12 = tf.less_equal(result, 0 - skew) File "/Users/aymensaket/opt/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4700, in less_equal "LessEqual", x=x, y=y, name=name) File "/Users/aymensaket/opt/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 610, in _apply_op_helper param_name=input_name) File "/Users/aymensaket/opt/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 60, in _SatisfiesTypeConstraint ", ".join(dtypes.as_dtype(x).name for x in allowed_list))) TypeError: Value passed to parameter 'x' has DataType bool not in list of allowed values: float32, float64, int32, uint8, int16, int8, int64, bfloat16, uint16, float16, uint32, uint64 (tf) Mac-mini:karoo aymensaket$ conda list

asksak commented 4 years ago

A further comment, MACOSX catalina does not run in 32 bit. Is this a problem?

kstaats commented 4 years ago

I appreciate the full details delivered, and am eager to respond but a bit behind at this time. I appreciate your patience.

On 1/21/20 2:53 AM, asksak wrote:

A further comment, MACOSX catalina does not run in 32 bit. Is this a problem?

kstaats commented 4 years ago

Were you able to resolve this? I do not have Catalina so I am not able to test this. What does an on-line search tell you? Karoo GP is not explicitly 32-bit, so I do not see how this would be a problem.

What is the exact error you receive?

On 1/21/20 2:53 AM, asksak wrote:

A further comment, MACOSX catalina does not run in 32 bit. Is this a problem?

kstaats commented 4 years ago

Thank you for the detailed output. If we review the currently supported operators (see karoo_gp/files/templates/operators_list.txt) this is the list:

OPERATOR EXAMPLE add a + b subtract a - b multiply a b divide a / b pow a ** 2

logical_and a and b logical_or a or b logical_not not a

abs abs(a) log log(a) log1p log1p(a) sign sign(a) square square(a) sqrt sqrt(a)

cos cos(a) sin sin(a) tan tan(a) acos acos(a) asin asin(a) atan atan(a)

No, at this time ">" and "<" are not supported.

I will speak with my colleague Iurii to learn if we can add this, to the next revision. Thank you for your patience.

Cheers, kai

On 1/20/20 5:14 AM, asksak wrote:

and here is the run error when i use classify with an added < function with arity 2

Select (c)lassification, (r)egression, (m)atching, or (p)lay (default m): c Select (f)ull, (g)row, or (r)amped 50/50 method (default r): g Enter depth of the initial population of Trees (default 3): Enter maximum Tree depth (default 3): Enter minimum number of nodes for any given Tree (default 3; max 15): Enter number of Trees in each population (default 100): Enter max number of generations (default 10): Display (i)nteractive, (g)eneration, (m)iminal, (s)ilent, or (d)e(b)ug (default m): s

We have constructed the first, stochastic population of 100 Trees

Evaluate the first generation of Trees ... Traceback (most recent call last): File "karoo_gp.py", line 251, in gp.fx_karoo_gp(kernel, tree_type, tree_depth_base, tree_depth_max, tree_depth_min, tree_pop_max, gen_max, tourn_size, filename, evolve_repro, evolve_point, evolve_branch, evolve_cross, display, precision, swim, mode) File "modules/karoo_gp_base_class.py", line 188, in fx_karoo_gp self.fx_fitness_gym(self.population_a) # generate expression, evaluate fitness, compare fitness File "modules/karoo_gp_base_class.py", line 1162, in fx_fitness_gym result = self.fx_fitness_eval(expr, self.data_train) File "modules/karoo_gp_base_class.py", line 1278, in fx_fitness_eval rule12 = tf.less_equal(result, 0 - skew) File "/Users/aymensaket/opt/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4700, in less_equal "LessEqual", x=x, y=y, name=name) File "/Users/aymensaket/opt/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 610, in _apply_op_helper param_name=input_name) File "/Users/aymensaket/opt/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 60, in _SatisfiesTypeConstraint ", ".join(dtypes.as_dtype(x).name for x in allowed_list))) TypeError: Value passed to parameter 'x' has DataType bool not in list of allowed values: float32, float64, int32, uint8, int16, int8, int64, bfloat16, uint16, float16, uint32, uint64 (tf) Mac-mini:karoo aymensaket$ conda list

asksak commented 4 years ago

thank you

asksak commented 4 years ago

Dear Kai,

I am testing maximum and minimum to replace > and <. However, as arity is 3 I am losing a parameter when trees evolve. I am working on fixing it as in my opinion it is extremely important for better GP utilisation.

Now I wonder if anyone can help with the Null error I get after the first generation when I use regression. Classification does not give this error.

Karoo GP could be the next big thing if we can get enough people to participate in development on github.

kstaats commented 4 years ago

Good to hear from you again.

The arity of 3 does work, or at least it did, but I have not attempted an if/then statement for some time (the operator that requires this arity).

To be specific, what happens when you invoke this?

As for the Null error, please send a screenshot or paste the error.

kai

On 2/19/20 9:34 AM, asksak wrote:

Dear Kai,

I am testing maximum and minimum to replace > and <. However, as arity is 3 I am losing a parameter when trees evolve. I am working on fixing it as in my opinion it is extremely important for better GP utilisation.

Now I wonder if anyone can help with the Null error I get after the first generation when I use regression. Classification does not give this error.

Karoo GP could be the next big thing if we can get enough people to participate in development on github.

ducktalez commented 4 years ago

Might be irrelevant, but in earlier tests with expanding karoo, I found out that python's "max" and "min" function can get reduced by sympify to a higher arity. Example: min( min(a,b), min(b,c) ) -> sympify -> min(a, b, c)

The Ast-converter recognizes this as arity-3 function (or arity 4, 5, 6...) - which it is - but the conversion to tensors does not work as planned. Tricky feature interaction. There are many ideas to solve this, but none of them are very convenient. If this is related to the probmen, I may be able to provide some possible solutions, just let me know

asksak commented 4 years ago

Might be irrelevant, but in earlier tests with expanding karoo, I found out that python's "max" and "min" function can get reduced by sympify to a higher arity. Example: min( min(a,b), min(b,c) ) -> sympify -> min(a, b, c)

The Ast-converter recognizes this as arity-3 function (or arity 4, 5, 6...) - which it is - but the conversion to tensors does not work as planned. Tricky feature interaction. There are many ideas to solve this, but none of them are very convenient. If this is related to the probmen, I may be able to provide some possible solutions, just let me know

Thanks for this. What I have actually done is assign +min and +max to arity 3 +max(a,b). This works well until a crossover or mutation strips it from one node.

I figured therefore that all the functions should maintain an arity it 2 if I am to minimally interfere with the code, so I set it as max(a,b) and min(a,b) and modified the label and evaluation to fit each function, specifically min, max, pow.

This worked fine until I hit the NaN inf wall with sqrt and /. There was a thread somewhere that suggested removing /sin /cos /tan and that solves part of the problem, the / Nan issue is solved by using safe divide instead of divide in tf.

Another problem that pops up is that tournament lead is referenced before assignment and that is clear in the tournament functions in the Karoo Kernel. The tournament lead is a local variable and the if statements in the function will fail in particular cases of mutation.

I like Karoo GP, but my initial reaction is that Koza’s lil-gp is much more flexible, though Karoo is more advanced because of the adaption to TF even with the function limitation.

I’ll work on it in my free time as I can see great potential if I could place the fitness function in a separate file. Koza’s Lil-gp has all modules separate so one can adapt the fitness function to any application. Although that is possible in Karoo, it requires kernel intervention for a program that’s supposed to be plug and play.

Cheers,

asksak commented 4 years ago

On this note, the GP is amazing for many applications, but if the concept is made closer to DNA strand structures and behaviour, who knows!

kstaats commented 4 years ago

I apologize for the incredible delay in my response to your last messages. I am coming back to Karoo with a full update, soon.

Your application of 'max(a,b) and min(a,b)' is clever, and appreciated. Thank you. I will conduct my own tests to confirm and add this to the documentation. The 'nan' is a function of dividing by zero. I used to have a catch built-in, but sometimes it does not catch this. Something I need to look into again. As for "tournament lead is referenced before assignment" I have never seen this happen and wonder if it is due to the code changes you have invoked? If you can clearly state when and where this occurs, I can determine how this is happening.

I will be updating Karoo with the most recent libraries by the close of 2020. I encourage you to download this new release, when complete. Stay tuned!

kstaats / karoo_gp

function Inquiry #19

Name Version Build Channel