search

openminted / Open-Call-Discussions

A central place for participants in the open calls to ask questions
2 stars 1 forks source link

Workflow editor shows old Docker components #21

Closed jakelever closed 6 years ago

jakelever commented 6 years ago

Hi, the Workflow editor seems to be showing Docker components from before the website was wiped and is not updated when new components are added to the platform.

galanisd commented 6 years ago

is not updated when new components are added to the platform.

What do mean? Can you provide an example?

jakelever commented 6 years ago

Sure. If I search for PubRunner in the workflow editor, I can see three components that I added a few weeks ago (Dummy dummy, Download Corpus with PubRunner, RunTool with PubRunner). These were deleted when the test platform was wiped. I cannot see either of my two current components: PubRunner Run v2 or Dummy ComponentX

oldcomponents

galanisd commented 6 years ago

Sure. If I search for PubRunner in the workflow editor, I can see three components that I added a few weeks ago (Dummy dummy, Download Corpus with PubRunner, RunTool with PubRunner). These were deleted when the test platform was wiped.

I do not know how/when these components disappeared. Probably they were deleted directly from metadata & database and/or metadata index. The issue is that OMTD Registry & Galaxy are not synced. @courado @antleb.

I cannot see either of my two current components: PubRunner Run v2 or Dummy ComponentX

PubRunner Run v2 appears now in Galaxy Editor (because I changed the version). See below... The problem is again (I think) that OMTD Registry & Galaxy are not synced. The same tool has been registered in Galaxy (same ID and version) multiple times; description does not matter if ID and version are the same, so, Galaxy is able to load only one.

galanisd commented 6 years ago

@jakelever

A question: How "Github Repo URL" parameter is used in the component you provided?

I think that it shouldn't be a parameter and that we have to move it to a different element/part of OMTD descriptor.

jakelever commented 6 years ago

Hey @galanisd , the PubRunner project loads another text mining tool from Github, builds it and runs it against a corpus. So the Github Repo is definitely a parameter.

There is a pubrunner.yml file in the root of the Github repo that gives instructions on how to build and execute the tool. An example would be https://github.com/jakelever/Ab3P which is an abbreviation detection tool built by a group at the NCBI.

reckart commented 6 years ago

@jakelever mind that it has been said the access of components to the internet will be limited at some point and access will only be granted to specific repositories. I imagine that GitHub will not be one of them.

Why do you build the tool at runtime instead of pre-building and shipping it in the container?

galanisd commented 6 years ago

Why do you build the tool at runtime instead of pre-building and shipping it in the container?

Same question with @reckart. In OMTD we adopted/use docker containers in order to avoid deployment, installation and interoperability issues, also for reproducability. If the image uses the latest commited code then the component might not work (build issues) or not produce the same results in two different invocations.

I think for all those reasons we should add in the guidelines that downloading source code and building it is not allowed.

greenwoodma commented 6 years ago

I'm with @reckart and @galanisd downloading, and compiling code from an external source should not be allowed as part of an OMTD component as it completely destroys any changes of maintaining reproducability.

galanisd commented 6 years ago

and also for security reasons...

jakelever commented 6 years ago

The main reason to pull the latest code is really to then pull the latest associated resources (e.g. versions of ontologies). I can see that this won't fit with the OMTD model so will work on static versions.