The purpose of the article seems fuzzy. In the abstract the authors indicate that the objective is to show how this new framework is able to beat the previous instance in accuracy with very competitive running times. But in experiments no mention is made of the time required by any of the frameworks to any of the problems. Even though in an experiment they indicate that "we are interested in finding out the speed with the whole operation could be performed, despite being Python a priori a slower language than C++". But nothing is provided about these results. Neither are details give on where the both frameworks has been executed as number of cores, memory, parallelization mechanisms,... The only references are:
evoMLP has been run on a laptop using ArchLinux.
Raw CPU power available now is much higher.
We will simple use a more powerful machine to run the experiments.
In conclusions section, the authors again point out that good results can be achieved with a limited budget and in a reasonable amount of time. In my opinion, such statements are not very enlightening. Both frameworks have the same stop conditions (number of generations, no improvement and find ideal solution) that are independent of the running time. Therefore, there is no evidence in the article to make such claims, related to the execution time.
About performance. Each algorithm has been executed just five times and given the similarity in some results, it may be insufficient.
The article lacks a clear methodology, being a series of experiments that is very difficult to follow, due to the variations that are made in each one (layers, operators, generations, ...), without being very clear what they are looking for, except to make comparisons.
Furthermore, the format of the tables is inconsistent, making them very difficult to interpret quickly. For example, Table 1 does not indicate the method inside the table (having to read the caption) or each table follows a different format, nothing consistent. The only figure on the paper lacks of axes lines, making it very difficult to analyze.
The results are not clearly presented. It lacks a methodology.
The conclusions refer to things that have not been demonstrated.
The purpose of the article seems fuzzy. In the abstract the authors indicate that the objective is to show how this new framework is able to beat the previous instance in accuracy with very competitive running times. But in experiments no mention is made of the time required by any of the frameworks to any of the problems. Even though in an experiment they indicate that "we are interested in finding out the speed with the whole operation could be performed, despite being Python a priori a slower language than C++". But nothing is provided about these results. Neither are details give on where the both frameworks has been executed as number of cores, memory, parallelization mechanisms,... The only references are:
In conclusions section, the authors again point out that good results can be achieved with a limited budget and in a reasonable amount of time. In my opinion, such statements are not very enlightening. Both frameworks have the same stop conditions (number of generations, no improvement and find ideal solution) that are independent of the running time. Therefore, there is no evidence in the article to make such claims, related to the execution time.
About performance. Each algorithm has been executed just five times and given the similarity in some results, it may be insufficient.
The article lacks a clear methodology, being a series of experiments that is very difficult to follow, due to the variations that are made in each one (layers, operators, generations, ...), without being very clear what they are looking for, except to make comparisons.
Furthermore, the format of the tables is inconsistent, making them very difficult to interpret quickly. For example, Table 1 does not indicate the method inside the table (having to read the caption) or each table follows a different format, nothing consistent. The only figure on the paper lacks of axes lines, making it very difficult to analyze.
The results are not clearly presented. It lacks a methodology.
The conclusions refer to things that have not been demonstrated.