Systems list contains not only my systems

smirnp commented 6 years ago

Original description

In MS Edge it looks, that the platform does not return the list of the systems either for registration into the challenge or for execution of the standalone experiment.

In Google Chrome it works fine

Problem

Several users define systems with the same URI (e.g., by copying the system.ttl file of another user). This leads to problems inside the platform which assumes that URIs are unique. We want to

block the execution of experiments with systems that have such a copied URI,
inform the user about the problem and
make sure that the original system can still be used (since the user of this system has not done anything wrong)

Solution

[x] make the backend identify systems with the same URI when crawling gitlab
[x] identify the "oldest" system (= the system that has been defined in the oldest gitlab project)
[x] mark "wrong" systems
[x] show a warning in the front end when the user is on the benchmark creation page or the system registration page (registration for a challenge)

Steps

InfAI will provide the information about the systems and adapt the beans in the UI backend.
USU will implement the displaying of the warning in the UI

smirnp commented 6 years ago

As we investigated via email the problem is not in the browser itself, but in the list of systems returned by the platform. For my one account (@gmail, Michael knows that) I see the list of 5-6 systems, while I have only one in git. For my second one (@agt) I see the same number of the systems, which I have in my git.

MichaelRoeder commented 6 years ago

I just checked this on the platform. To me, the situation looks like the following: The AGT user sees 1 system. That is correct since he defined this system in one of his projects. The gmail user sees 6 systems while he only has defined 2 systems.

The problem is caused be the reusage of the same URI for different systems. The gmail user defined a http://project-hobbit.eu/resources/debs2017/debsparrotsystemexample system. This appears already 5 times in our repository and causes the controller to list all 5 systems.

MichaelRoeder commented 6 years ago

@yamalight, @denkv We have the problem that several systems share the same URI (because unexperienced users might simply copy an example system without changing its URI). As it can be seen, this creates problems inside our system (not only in the UI but also in the platform itself since it is not clear which system will be benchmarked).

Do you have a good idea how to handle this? Should we block all systems that are sharing the same URI? But that might be bad when it comes to example systems that are not executable because someone copied its file...

yamalight commented 6 years ago

Can we enforce unique URIs for systems? Just rejecting creation with message "this URI is already used, please enter a different one" should solve this, right?

Upd: Considering users use gitlab to "create" their system, we can add a warning to user GUI when he has systems with conflicting URIs. Thoughts?

MichaelRoeder commented 6 years ago

Just rejecting creation with message "this URI is already used, please enter a different one" should solve this, right?

Yes, it would solve it but it is not possible since the "act of creating a system" is uploading / creating a file in gitlab.

we can add a warning to user GUI when he has systems with conflicting URIs. Thoughts?

Good idea. I would also "block" the execution of experiments containing these systems (maybe that is what you meant with "rejecting creation" above.

From my point of view this makes only sense, if we are able to check that the "original" system with this URI still can be used. Otherwise two users A and B are not able to benchmark their systems just because B made a mistake (i.e., copied the system.ttl from A without changing the system URI). I have to check whether we can get access to creation times of files/projects via the Gitlab API...

Update: getting the creation date of a project is no problem. I would suggest to "block" the execution of system A that is "reusing" a URI of another system B when the other system has been defined in a project P_B that is older than the project P_A.

yamalight commented 6 years ago

@MichaelRoeder yep, sounds good! :+1:

yamalight commented 6 years ago

@MichaelRoeder maybe move that outline to issue description instead?

denkv commented 6 years ago

not possible since the "act of creating a system" is uploading / creating a file in gitlab.

Looks like it's possible to set up global hooks in gitlab if we want to prevent uploading of systems with duplicate URIs in the first place.

MichaelRoeder commented 6 years ago

A global hook would be a solution but makes it harder to use something else than our Gitlab (makes it nearly impossible to implement #9). Since our platform should be able to handle different "sources" of benchmark and system definitions, it has to be able to handle URI conflicts, internally.

MichaelRoeder commented 6 years ago

@romankorf and @Ennosigaeon How do you would like to have the information, that a system has a problem and can not be chosen by the user? At the moment, the GUI Serverbackend returns List<SystemBean> encapsulated inside a response.

We can extend the class SystemBean to contain the error message
- Advantage: very simple to implement in the GUI Serverbackend
- Disadvantage: the GUI client will have to go through the beans, filter beans with an error message and show the message to the user.
We extend the Response to contain List<SystemBean> and a list of messages
- Advantage: might be easier to implement in the GUI client and might be reusable in the future (if we want to show other messages to the user)
- Disadvantage: a little bit more complicated in the GUI Serverbackend

What do you think?

Ennosigaeon commented 6 years ago

I think as long as the number of systems is limited (e.g. < 1000) it should be no problem to filter the systems in the frontend. We can simply remove all SystemBeans containing a error message from the dropdown and display an alert containing all faulty system including the error message.

This should be quite forward to implement.

MichaelRoeder commented 6 years ago

I added this information to the beans in the feature/UriCopyHandling136 branch.

Note that it has been added to benchmarks and systems (to their super class). I also created an example for both. The JSON that is received by the GUI client looks as follows.

Requesting benchmarks (faulty benchmark and another, normal benchmark):

[...
{
  "type":"benchmarkBean",
  "description":"This is a faulty benchmark used for testing!",
  "errorMessage":"This image has the same URI as HOBBIT Platform Benchmark (http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchmark) from gitadmin / platform-benchmark.",
  "id":"http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchmark",
  "name":"Faulty Benchmark (Do not use me!)","configurationParamNames":[]
},{
  "type":"benchmarkBean",
  "description":"DEBS GC 2017 parrot tutorial benchmark",
  "id":"http://project-hobbit.eu/resources/debs2017/debsparrotbenchmark",
  "name":"DEBS GC 2017 parrot tutorial benchmark","configurationParamNames":[]},
...]

(the complete example can be found here)

For benchmarks, it would be good to simply filter the faulty beans (this can also be done in the GUI-Serverbackend). We don't have to show an error for them because we would display the error to all the users without knowing whether the current user is responsible for the problem or not.

Requesting benchmark details (faulty system and another, normal system)):

{
  "type":"benchmarkBean",
  "description":"This benchmark emulates the traffic created by benchmarking a SPARQL store to determine the maximal throughput of the platform.",
  "id":"http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchmark",
  "name":"HOBBIT Platform Benchmark",
  "configurationParamNames": [...],
  "kpis":[...],
  "systems":[
    {
      "description":"I am a faulty system used for testing!",
      "errorMessage":"This image has the same URI as HOBBIT Platform Benchmark System 1 (http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchSystem_1) from gitadmin / platform-benchmark-system.",
      "id":"http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchSystem_1",
      "name":"Faulty System (Do not use me!)"
    },{
      "description":"The system needed to run the platform benchmark instantiating 1 system adapter.",
      "id":"http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchSystem_1",
      "name":"HOBBIT Platform Benchmark System 1"
    },...]}

(the complete example can be found here)

Ennosigaeon commented 6 years ago

@MichaelRoeder Currently we have a merge conflict in PlatformControllerClient (develop). Can you please fix that, so that I can incorporate #193 easily?

MichaelRoeder commented 6 years ago

@Ennosigaeon sure. Can you please create a pull request and assign me? Otherwise, I am not sure which branch I should merge :confused:

Ennosigaeon commented 6 years ago

Pull request #203 is created and assigned to you.

MichaelRoeder commented 6 years ago

merged.

Ennosigaeon commented 6 years ago

I have implemented the necessary UI changes.

smirnp commented 6 years ago

The problem still seems to be in the online platform. Under multiple accounts without the 'challenge organizer' role I'm able to execute all available systems compatible with selected benchmark: https://gyazo.com/b08515942c22de1808c51a880851052c
But do not see invalid systems warning, which I see under my 'challenge organizer' account (but 'duplicating' system is not included in the list)

smirnp commented 6 years ago

Also I may register/unregister not-my systems for challenge tasks.

hobbit-project / platform