galaxyproject / bioblend

A Python library for interacting with the Galaxy API
https://bioblend.readthedocs.io/
MIT License
85 stars 114 forks source link

Error when trying to get ids of workflow outputs #415

Open stvlar opened 2 years ago

stvlar commented 2 years ago

Hello, I'm using bioblends to get results from workflows I've developed and my code worked well until recently. I'm trying to get the ids of the outputs to then download them as files through a web platform I'm developing with django.

This is the code I'm using:

   media_path = ".."
    result_path = ".."
    max_timeout = 1*24*60*60 #days*hours*mins*secs

    if request.method == 'POST':
        form = SnippetForm(request.POST, request.FILES)
        if form.is_valid():
            form = Snippet(file=request.FILES['file'])
            form.save()

            file_name_final = form.file.name[10:]
            file_path = media_path + file_name_final

           server = 'https://usegalaxy.eu/'
           api_key = ''
           gi = bioblend.galaxy.GalaxyInstance(url=server, key=api_key)

            test = gi.tools.upload_file(file_path, 'upload history id')

            hda = test['outputs'][0].get('id')
            datamap = dict()
            datamap = {0: {'id': hda['id'], 'src': 'hda'}}

            results_run = gi.workflows.invoke_workflow('workflow id', inputs=datamap, history_name='script results')
            results_id = results_run['outputs']
            id_str = ''.join(results_id)
            #download output to user's pc
            gi.datasets.download_dataset(id_str, file_path=result_path, maxwait=max_timeout)
simonbray commented 2 years ago

Hi @stvlar,

It seems the result returned by the Galaxy API has changed, I wasn't aware of that.

Probably the best solution is to change the line after the invocation to something like the following:

results_id = gi.invocations.show_invocation(results_run['id'])['outputs']
stvlar commented 2 years ago

Thank you for your response that worked! Also I'm getting an error releated to gi.datasets.download_dataset(id_str, file_path=result_path, maxwait=max_timeout)

This is the error message

TypeError at /tool/ list indices must be integers or slices, not str

This is the traceback

Traceback (most recent call last):
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/stella/pasteur/app/views.py", line 198, in tool
    gi.datasets.download_dataset(id_str, file_path=result_path, maxwait=max_timeout)
  File "/home/stella/.local/lib/python3.8/site-packages/bioblend/galaxy/datasets/__init__.py", line 92, in download_dataset
    dataset = self.wait_for_dataset(dataset_id, maxwait=maxwait, check=False)
  File "/home/stella/.local/lib/python3.8/site-packages/bioblend/galaxy/datasets/__init__.py", line 385, in wait_for_dataset
    state = dataset['state']

Exception Type: TypeError at /tool/
Exception Value: list indices must be integers or slices, not str

`

Again this is the first time this has been an issue for me so I'm a bit lost

simonbray commented 2 years ago

That is quite strange. The error is on this line: https://github.com/galaxyproject/bioblend/blob/v0.16.0/bioblend/galaxy/datasets/__init__.py#L385

So it would suggest that the show_dataset() method (used on line 384 above) is returning a list, rather than a dict as expected.

Can you check if that is really the case?

stvlar commented 2 years ago

As far as I can see it is returning a dictionary

`


dataset | [{'create_time': ...',   
'dataset_id': '...',  
'deleted': False,   
'extension': 'auto',   
'hid': 187,   
'history_content_type': 'dataset',   
'history_id': '..',   
'id': '...',   
'name': '...',   
'purged': False,   
'state': 'queued',   
'tags': [],   
'type': 'file',   
'type_id': 'dataset',   
'update_time': '....',   
'url': '....',   
'visible': True},
-- | --

`

simonbray commented 2 years ago

[{

Looks like a list of dicts to me?

@davelopez maybe you have some idea here? :confused:

simonbray commented 2 years ago

@stvlar are you using a particular Galaxy version?

stvlar commented 2 years ago

I'm using the most recent release v21.09

stvlar commented 2 years ago

[{

Looks like a list of dicts to me?

@davelopez maybe you have some idea here? confused

yes there were more dicts there I just left them out since the only difference between them was the input file and the time

davelopez commented 2 years ago

I quickly checked and /api/datasets/{dataset_id} should return a dictionary not a list. Otherwise, https://github.com/galaxyproject/bioblend/blob/v0.16.0/bioblend/_tests/TestGalaxyDatasets.py#L84 should be failing too right? I'm confused too...

mvdbeek commented 2 years ago

It seems the result returned by the Galaxy API has changed, I wasn't aware of that.

We should not be changing the API, which endpoint do you think has changed the response ? I'd also assume both the bioblend tests and Galaxy tests would be failing if that was the case. Does the invocation in question maybe not have any outputs ? What was the original exception ?

stvlar commented 2 years ago

@mvdbeek

What was the original exception ?

The original exception was a KeyError with 'outputs' being the exception value

the original problem was this line results_id = results_run['outputs']

mvdbeek commented 2 years ago

Can you paste it please together with the response ? And ideally a reproducible example

stvlar commented 2 years ago

@mvdbeek this is the original traceback

Traceback (most recent call last):
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/stella/.local/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/stella/project/app/views.py", line 195, in tool
    results_id = results_run['outputs']

Exception Type: KeyError at /tool/
Exception Value: 'outputs'
Request Method: | POST
Request URL : http://127.0.0.1:8000/tool/
Django Version : 3.2.11
Exception Type : KeyError
Exception Value : 'outputs'
Exception Location : /home/stella/project/app/views.py, line 195, in tool
Python Executable : /usr/bin/python3
Python Version : 3.8.10
Python Path : ['/home/stella/project',  '/usr/lib/python38.zip',  '/usr/lib/python3.8',  '/usr/lib/python3.8/lib-dynload',  '/home/stella/.local/lib/python3.8/site-packages',  '/usr/local/lib/python3.8/dist-packages',  '/usr/lib/python3/dist-packages',  '/usr/lib/python3.8/dist-packages',  '/home/stella/project']
Server time : Fri, 21 Jan 2022 12:40:56 +0000

the code is in my original question

Unfortunately I can't send a reproducible example for confidentiality reasons