Use locally in an IT

[paulzierep]

I am trying to use the get / put function when developing an IT locally. I managed to install it in the docker container. The XML of the IT has this envs:

  <environment_variables>
      <environment_variable name="HISTORY_ID">$__history_id__</environment_variable>
      <environment_variable name="REMOTE_HOST">$__galaxy_url__</environment_variable>
      <environment_variable name="GALAXY_WEB_PORT">8080</environment_variable>
      <environment_variable name="GALAXY_URL">$__galaxy_url__</environment_variable>
      <environment_variable name="DEBUG">TRUE</environment_variable>
      <environment_variable name="API_KEY" inject="api_key"/>
  </environment_variables>

But when I run in the IT put -p test.txt it fails with:

DEBUG:root:Host IP determined to be b'172.17.0.1\n'
DEBUG:root:TestURL url=http://localhost:8080 obj=True
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:8080
DEBUG:root:TestURL url=http://localhost:8080 state=failure
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 obj=True
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): b'172.17.0.1':8080
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 state=failure
Traceback (most recent call last):
  File "/usr/local/bin/put", line 17, in <module>
    put( args.filepath, file_type=args.filetype, history_id=args.history_id )
  File "/usr/local/lib/python3.10/dist-packages/galaxy_ie_helpers/__init__.py", line 113, in put
    gi = get_galaxy_connection(history_id=history_id)
  File "/usr/local/lib/python3.10/dist-packages/galaxy_ie_helpers/__init__.py", line 100, in get_galaxy_connection
    raise Exception(msg)
Exception: Could not connect to a galaxy instance. Please contact your SysAdmin for help with this error

I suspect, that something is off with the configuration of my Galaxy instance for local IT development. I am using those configurations:

GALAXY_PATH=~/git/galaxy # modify
mv $GALAXY_PATH/config/galaxy.yml.interactivetools $GALAXY_PATH/config/galaxy.yml
mv $GALAXY_PATH/config/job_conf.yml.interactivetools $GALAXY_PATH/config/job_conf.yml
mv $GALAXY_PATH/config/tool_conf.xml.sample $GALAXY_PATH/config/tool_conf.xml #add interactive tools

I suspect the docker instance of the IT cannot reach the host IP when running it locally, any hints of how to set that up locally would be great.

[bgruening]

Try setting this https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/galaxy.yml.sample#L1443 to something that reaches your Galaxy server.

[bgruening]

xref: https://training.galaxyproject.org/training-material/topics/contributing/tutorials/create-new-tutorial-tours/tutorial.html

[paulzierep]

Try setting this https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/galaxy.yml.sample#L1443 to something that reaches your Galaxy server.

I tried:

  # your Galaxy, so saving notebooks doesn't fail due to invalid certificates.
  galaxy_infrastructure_url: http://127.0.0.1:8080

  # Do not set the following 2 options if you are using an upstream proxy server like nginx
  interactivetools_upstream_proxy: false
  interactivetools_proxy_host: 127.0.0.1:4002

which leads to about:blank#blocked for the IT link. and again:

DEBUG:root:Host IP determined to be b'172.17.0.1\n'
DEBUG:root:TestURL url=http://127.0.0.1:8080 obj=True
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 127.0.0.1:8080
DEBUG:root:TestURL url=http://127.0.0.1:8080 state=failure
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 obj=True
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): b'172.17.0.1':8080
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 state=failure
Traceback (most recent call last):
  File "/usr/local/bin/put", line 17, in <module>
    put( args.filepath, file_type=args.filetype, history_id=args.history_id )
  File "/usr/local/lib/python3.10/dist-packages/galaxy_ie_helpers/__init__.py", line 113, in put
    gi = get_galaxy_connection(history_id=history_id)
  File "/usr/local/lib/python3.10/dist-packages/galaxy_ie_helpers/__init__.py", line 100, in get_galaxy_connection
    raise Exception(msg)
Exception: Could not connect to a galaxy instance. Please contact your SysAdmin for help with this error

• not sure if the interactivetools_proxy_host is required.

[hexylena]

you don't want localhost/127.0.0.1, that will point inside the container at itself

you want either the local hostname of your computer, or the IP address given you to by your wifi/ethernet

[paulzierep]

OK I also tried:

  galaxy_infrastructure_url: 192.168.178.45:8080

  # Do not set the following 2 options if you are using an upstream proxy server like nginx
  interactivetools_upstream_proxy: false
  interactivetools_proxy_host: 192.168.178.45:4002

Which would be the one given by ethernet. But the server still starts with Serving on http://127.0.0.1:8080 - that does not seem right, or ? Galaxy should now be reachable via 192.168.178.45, correct ?

IT also fails now with:

DEBUG:root:Host IP determined to be b'172.17.0.1\n'
DEBUG:root:TestURL url=192.168.178.45:8080 obj=True
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 192.168.178.45:8080
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 192.168.178.45:8080
DEBUG:root:TestURL url=192.168.178.45:8080 state=failure
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 obj=True
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): b'172.17.0.1':8080
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 state=failure
Traceback (most recent call last):
  File "/usr/local/bin/put", line 17, in <module>
    put( args.filepath, file_type=args.filetype, history_id=args.history_id )
  File "/usr/local/lib/python3.10/dist-packages/galaxy_ie_helpers/__init__.py", line 113, in put
    gi = get_galaxy_connection(history_id=history_id)
  File "/usr/local/lib/python3.10/dist-packages/galaxy_ie_helpers/__init__.py", line 100, in get_galaxy_connection
    raise Exception(msg)
Exception: Could not connect to a galaxy instance. Please contact your SysAdmin for help with this error

[hexylena]

yep you need to also have your galaxy listening on 0.0.0.0

[paulzierep]

Sorry, I know litte in terms of Galaxy admin :) how do I make Galaxy listen on 0.0.0.0 ? Would:

  gunicorn:

    # Enable Galaxy gunicorn server.
    # enable: true

    # The socket to bind. A string of the form: ``HOST``, ``HOST:PORT``, ``unix:PATH``, ``fd://FD``. An IP is a valid HOST.
    # bind: 0.0.0.0:8080

work.

[paulzierep]

Thanks @hexylena with your suggestion I used this conf:

gravity:
  gx_it_proxy:
    enable: true
    port: 4002

  gunicorn:
    # Address and port to bind to. Use 0.0.0.0 to listen on all available IPs.
    bind: '0.0.0.0:8080'
    #handlers:
  #  handler:
  #    processes: 3
  #    pools:
  #      - job-handlers
  #      - workflow-schedulers

galaxy:
  interactivetools_enable: true
  interactivetools_map: database/interactivetools_map.sqlite

  # outputs_to_working_directory will provide you with a better level of isolation. It is highly recommended to set
  # this parameter with InteractiveTools.
  outputs_to_working_directory: true

  # `galaxy_infrastructure_url` needs to be reachable from IT containers.
  # For local development you can map arbitrary hostnames. See `job_conf.yml.interactivetools`
  # for an example.
  # In the local development case you should use the `http` protocol (e.g http://localhost:8080) to access
  # your Galaxy, so saving notebooks doesn't fail due to invalid certificates.
  galaxy_infrastructure_url: http://192.168.178.45:8080

  # Do not set the following 2 options if you are using an upstream proxy server like nginx
  interactivetools_upstream_proxy: false
  interactivetools_proxy_host: localhost:4002

First, I used this parameter: interactivetools_proxy_host: http://192.168.178.45:4002 which did not allow accessing the interactive tool, seem like the created url was not allowed by my brwoser or something about:blank#blocked; but now the interactive tool opens and the put function works. Thank you !!

[hexylena]

glad it was solved! if you like to write this up as an faq in the gtn I think it would be very welcome!

[paulzierep]

Definitely! can you point me where to add it in the GTN, assume in admin stuff? I will eventually use the put/get function to add Galaxy interaction to shiny (and maybe other) apps and write a GTN tutorial for it.

[hexylena]

you can fill out this form https://docs.google.com/forms/d/e/1FAIpQLScG1jTujUGpzVQllfpBkP9frVNYAqLNbZ59kjtCe9j6MYWnCQ/viewform and we'll do the rest, or, yes in topics/admin/FAQs/ would also be fine