search

openminted / Open-Call-Discussions

A central place for participants in the open calls to ask questions
2 stars 1 forks source link

PubRunner Docker app execution #20

Closed jakelever closed 6 years ago

jakelever commented 6 years ago

Hi, I'm trying to add a Docker application/component. It takes an XMI corpus as an input and outputs a TSV of abbreviations/acronyms.

I've added it as an application and have tried running it. But it just stays at "Running..." as #13.

I've tried running the Docker directly with the XMI sample files (https://openminted.github.io/releases/xmiExamples/) on another machine and it seems to work okay. Can I confirm if the command below is equivalent of what OpenMinTeD will run?

docker run -v /home/jake_lever/input:/input -v /home/jake_lever/output:/output jakelever/pubrunner ./pubrunner/openminted/Ab3P.py --input /input --output /output

OMTD-Share file: pubrunner_Ab3P_app.zip

jakelever commented 6 years ago

Also +1 for #17 as it'd be good to see error messages.

galanisd commented 6 years ago

Can I confirm if the command below is equivalent of what OpenMinTeD will run?

Yes a command like this is executed from Galaxy engine @ OpenMinTeD.

In the metadata record the image location that you provided is this? https://hub.docker.com/r/jakelever/pubrunner/

If so, then the problem is that this location is copied to the respective wrapper and Galaxy runs a "docker pull https://hub.docker.com/r/jakelever/pubrunner/" which results in an error "invalid reference format".

A "jakelever/pubrunner" would be OK.

galanisd commented 6 years ago

I just noticed that you sent the OMTD-SHARE file and that the image location is

https://hub.docker.com/r/jakelever/pubrunner/
jakelever commented 6 years ago

Hi, thanks for the feedback. I've created a new test application which is just a pass-through (copies input corpus to output corpus and does nothing else). I've created a new Docker repo for this test and made sure the distributionLocation is correct. But I receive an undefined error.

So the command to execute this one looks like:

docker run -v /home/jake_lever/input:/input -v /home/jake_lever/output:/output jakelever/abcdefg ./pubrunner/openminted/passthrough.py --input /input --output /output

Here is the OMTD-SHARE XML file for this. passthrough.zip

And I get the undefined error: undefined_error

Is there obviously anything else wrong with the OMTD-SHARE file? Thanks for all your help.

ragerri commented 6 years ago

Hello,

Are we not supposed to add also the --input and --output parameters to the OMTD descriptor?

Thanks,

Rodrigo

On Thu, Apr 12, 2018 at 12:30 AM, Dimitrios Galanis < notifications@github.com> wrote:

I just noticed that you sent the OMTD-SHARE file and that the image location is ns0:distributionLocationhttps://hub.docker.com/r/jakelever/pubrunner/ </ns0:distributionLocation>

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openminted/Open-Call-Discussions/issues/20#issuecomment-380616346, or mute the thread https://github.com/notifications/unsubscribe-auth/AAZl_zgmY7-kw1QUWMOWCY8IeeTw3SMRks5tnoQNgaJpZM4TQz05 .

ragerri commented 6 years ago

As a follow-up,

I have been looking at the examples in

https://openminted.github.io/releases/omtd-share/3.0.2/examples/dockerComponentsv302.zip

and they do not add the --input --output parameters to the omtd descriptor.

But then in the https://openminted.github.io/releases/docker-spec/1.0.0/specification

"The component must specify inputs and outputs as parameters to the docker run command in its OMTD-SHARE descriptor."

It will be nice to clarify this.

Thanks,

Rodrigo

On Thu, Apr 12, 2018 at 9:22 AM, Rodrigo Agerri rodrigo.agerri@ehu.eus wrote:

Hello,

Are we not supposed to add also the --input and --output parameters to the OMTD descriptor?

Thanks,

Rodrigo

On Thu, Apr 12, 2018 at 12:30 AM, Dimitrios Galanis < notifications@github.com> wrote:

I just noticed that you sent the OMTD-SHARE file and that the image location is ns0:distributionLocationhttps://hub.docker.com/r/jakelever/pubrunner/ </ns0:distributionLocation>

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openminted/Open-Call-Discussions/issues/20#issuecomment-380616346, or mute the thread https://github.com/notifications/unsubscribe-auth/AAZl_zgmY7-kw1QUWMOWCY8IeeTw3SMRks5tnoQNgaJpZM4TQz05 .

galanisd commented 6 years ago

Are we not supposed to add also the --input and --output parameters to the OMTD descriptor?

In the OMTD descriptor you shouldn't add --input and --output parameters; just the command for calling your component. --input and --output parameters are generated/added automatically by OMTD Registry; i.e. there is a module in Registry that generates a Galaxy wrapper (XML tool file) from an OMTD descriptor and copies it to Galaxy server. This wrapper is used to call the component; actually a part of it contains a template command which has the following format

--input --output Please see also this thread for more info. https://groups.google.com/forum/#!searchin/openminted-user-forum/%22docker$20run%22%7Csort:date/openminted-user-forum/2zKTXE8zkXE/UaHL5b4LAAAJ > But then in the > https://openminted.github.io/releases/docker-spec/1.0.0/specification > > "The component must specify inputs and outputs as parameters to the *docker > run* command in its OMTD-SHARE descriptor." > Your image contains contains a TDM component that can be called with the command that will provide. The dockerized component should accept --input and --output parameters as it is specified in the Docker spec. (https://openminted.github.io/releases/docker-spec/1.0.0/specification).
galanisd commented 6 years ago

A correction ...

This wrapper is used to call the component; actually a part of it contains a template command which has the following format

|commandThatYouProvidedInOMTDDesc| --input |inputFolderSelectedByGalaxy| --output |outputFolderSelectedByGalaxy|

ragerri commented 6 years ago

OK,

Thanks!

R

On Thu, Apr 12, 2018 at 11:09 AM, Dimitrios Galanis < notifications@github.com> wrote:

A correction ...

This wrapper is used to call the component; actually a part of it contains a template command which has the following format

|commandThatYouProvidedInOMTDDesc| --input |inputFolderSelectedByGalaxy| --output |outputFolderSelectedByGalaxy|

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/openminted/Open-Call-Discussions/issues/20#issuecomment-380733195, or mute the thread https://github.com/notifications/unsubscribe-auth/AAZl_5M-e9MOFeKK6vW2VMLLMQdVZ_HCks5tnxm3gaJpZM4TQz05 .

jakelever commented 6 years ago

Hi @galanisd , I was wondering if there's anything I should change for the passthrough application example. I've tried running it again and it still stays stuck on "Running".

mhabsaoui commented 6 years ago

@jakelever I think @galanisd has fully took the measure of our common bug (jobs running endlessly...). It seems OMTD Tech-Team needs a bit more time to look into it...

@galanisd One useful thing to do ASAP, would be to log meaning errors (currently 'erros[].length=0') on browser console, so we, users, could grasp on something to step forward...

Cheers.

galanisd commented 6 years ago

It seems OMTD Tech-Team needs a bit more time to look into it...

Our plan is to check the metadata and test all components/apps which are coming from open call plus the ones that are being developed by OpenMinTeD partners. It seems that in several of them there are some details to take care of, and some bugs to be fixed. We are checking them one by one...

jakelever commented 6 years ago

Thanks @galanisd for looking into this and for all your work.