search

hobbit-project / platform

HOBBIT benchmarking platform
GNU General Public License v2.0
23 stars 9 forks source link

Defaultevaluationstorage crashed #491

Closed sven-h closed 4 years ago

sven-h commented 4 years ago

Description

I run mutliple systems with multiple benchmarks and it results in different errors. The experiments are: https://master.project-hobbit.eu/experiments/1596540796784 https://master.project-hobbit.eu/experiments/1596540780017

In the experiments it looks like the defaultevaluationstorage crashed (see the benchmark log). I don't know if the system is not correctly implementing the data/RabbitMQ interface or if it is something else.

MichaelRoeder commented 4 years ago

1596540796784 crashed because the default evaluation storage waited 1 min. for another container being created by the platform. Unfortunately, the platform didn't manage to do it in time. So this problem is not related to the system implementation. I would suggest to use version 1.0.12 or newer (1.0.17 is the latest) of the hobbit.core library in the benchmark implementation. This should use a newer version of the eval storage that is more patient :wink:

1596540780017 has the issue that the image of the system does not exist.

2020-08-04 11:33:43,035 ERROR [org.hobbit.controller.docker.ContainerManagerImpl] - <Couldn't pull image git.project-hobbit.eu:4567/sven/simplejavamatcher:1.0 on node wu9wxozf6gwaqiziqc1nl2f75. No such image: git.project-hobbit.eu:4567/sven/simplejavamatcher:1.0>

After that, the platform still tries to start a Docker Swarm Service with this image but the service directly stops (because the image does not exist) and the platform recognizes this as a failing benchmarked system.

sven-h commented 4 years ago

Thank you very much for the quick response. Unfortunately I do not have access to the benchmark implementation but I will inform the owner to increase the version of the core library. I also noticed that some benchmarks (from OAEI) use the 1.0.9 version of hobbit.core. Last year I wanted to upgrade to a higher minor version ( 1.1.0 and 1.1.1) but it doesn't work because the communication protocol changed somewhere. So now the 1.0.xbranch will continue, right? And the different versions in 1.0.x branch should work together? This means that a benchmark using 1.0.9 can work with a system using 1.0.12 or 1.0.18?

Additionally, it would be very cool to have the core library in maven central (I can also give some hints because I recently released another library to central). One requirement is that you should own the domain http://hobbit.org.

MichaelRoeder commented 4 years ago

We are in contact with the developers as well :+1:

We are still working on improving the core library by a) extending its functionality and b) making its usage easier. However, since the HOBBIT project itself does not exist as a funded project anymore, it is difficult to find the resources to finalize the 1.1.x versions. We are aware of the confusion that these versions may create and apologize for that.

Yes, the different versions of the 1.0.x branch are compatible. :smiley:

I would be happy to have the library in maven central. Is there any guide how that can be done? We do not own the domain so we would have to change the name of the group id of our artifacts. But that is fine, I guess :thinking:

sven-h commented 4 years ago

For me it is fine if I can use the 1.0.x branch. :+1:

For maven central, I followed mainly this guide: https://dzone.com/articles/publish-your-artifacts-to-maven-central