search

galaxyproject / training-material

A collection of Galaxy-related training material
https://training.galaxyproject.org
MIT License
294 stars 846 forks source link

Workflow Parameters #1393

Closed mvdbeek closed 4 years ago

mvdbeek commented 4 years ago

I think this is ready, if I can get a review that'd be great!

hexylena commented 4 years ago

@mvdbeek have a couple of fixes I'll push but I'm stuck at Compose text parameter value, where should this be found? Is that something from toolshed that's required? Or something in dev only?

Edit: this requires your new expression tools or?

hexylena commented 4 years ago

Update: found them, we'll just have to update .eu

mvdbeek commented 4 years ago

yep, they're on the toolshed, https://toolshed.g2.bx.psu.edu/view/iuc/compose_text_param/feb3acba1e0a and should work starting from 19.05

hexylena commented 4 years ago

If you run the workflow you will now see the new parameter that needs to be set before the workflow can be run.

For me it doesn't need to be set, in fact it defaults to 0 and is collapsed by default. This is with optional: no. Also there doesn't seem to be a way to provide a default value. Is this expected?

(Please don't change the line, I have other updates for that line too.)

mvdbeek commented 4 years ago

Is this expected?

Unfortunately yes, at this point. Though we'll tackle this soon (probably at the contribution fest if there is interest).

hexylena commented 4 years ago

Got a bit closer this morning but the expression tool is failing on EU with

Traceback (most recent call last):
  File "_evaluate_expression_.py", line 1, in <module>
    from galaxy_ext.expressions.handle_job import run; run()
  File "/opt/galaxy/server/lib/galaxy_ext/expressions/handle_job.py", line 31, in run
    with open(environment_path, "r") as f:
IOError: [Errno 2] No such file or directory: ''

cat: tmp6CL_1e: No such file or directory

also .eu seems to be eating my workflow, the regex step never gets scheduled and I'm not seeing an error in the logs.

If you want to get this in, is there a known working version somewhere I can test with? Do I need to set it up locally?

hexylena commented 4 years ago

Bug

Upon clicking the workflow connection button here, no input was added.

image

Progress

Ok, I've finished most of the stylistic updates (added myself as contributor since it took a bit of time, hope that's ok for you) and I think it's fine (but again can't test easily, not working on .eu)

I've added some example datasets in places but if you have better ones, great, and if we could run the workflows on those inputs then we can have students check their answers maybe in question/solution boxes?

mvdbeek commented 4 years ago

also .eu seems to be eating my workflow, the regex step never gets scheduled and I'm not seeing an error in the logs.

That's to be expected if the expression tool doesn't run. Do you have node js installed and on galaxy's PATH ? I also added https://github.com/bardin-lab/galaxy/commit/3a6ea0cbb61f06f4495fda6570a0214c4f923b3b on our instance, but didn't quite figure out if that was really necessary.

Upon clicking the workflow connection button here, no input was added.

I can reproduce that on usegalaxy.eu, but it works fine on our local instance and on my dev instance. I see that https://github.com/usegalaxy-eu/galaxy/compare/release_19.05_europe...galaxyproject:release_19.05 is up to date, maybe a client rebuild is necessary ? Also usegalaxy.org is missing the expression tools in its entirety, so couldn't check there.

hexylena commented 4 years ago

Do you have node js installed and on galaxy's PATH ?

Oh gosh no! That's a requirement? It doesn't come from conda / normal dependencies? Our path is extremely clean. It's not in the .venv either for us, I guess it's only installed during the make client step and we do that somewhere else.

Clientbuild was latest commit on that branch, I can rebase and rebuild again later?

mvdbeek commented 4 years ago

Oh gosh no! That's a requirement? It doesn't come from conda / normal dependencies?

It's installed with common_startup.sh via nodeenv and should then be available via the virtualenv galaxy runs in in (https://github.com/galaxyproject/galaxy/blob/release_19.05/scripts/common_startup.sh#L233)

Clientbuild was latest commit on that branch, I can rebase and rebuild again later?

Hmm, the commits look fine, nothing relevant missing as far as I can tell.

hexylena commented 4 years ago

If you don't build the client server side, then common startup isn't part of the ansible playbooks, so it's never been installed for us on EU.

Is there any reason that it isn't a more normal dependency?

mvdbeek commented 4 years ago

Is there any reason that it isn't a more normal dependency?

Just that it's not a python dependency that can be installed via pip.

hexylena commented 4 years ago

Oh, I meant more like a conda dependency / run as a normal tool?

mvdbeek commented 4 years ago

That might work, we could give that a try.

mvdbeek commented 4 years ago

https://github.com/galaxyproject/galaxy/pull/8224 for making node a regular tool dependency. also it looks like the bug has disappeared after a restart ... yay!

hexylena commented 4 years ago

And after more restarts it's back!

That's to be expected if the expression tool doesn't run. Do you have node js installed and on galaxy's PATH ?

apparently we actually do, just not in venv, I'd completely forgotten about that.

galaxy@sn04:~/server$ node -v
v8.16.0
galaxy@sn04:~/server$ which node
/usr/bin/node

so I'm even more confused now. but in theory, knowing how all of these parts fit together, the training at least looks good. So, I'm happy to merge it now. EU/tool/config bugs we can figure out at some other date.

hexylena commented 4 years ago

Thanks @mvdbeek!