How to upload files to a 'list' specified in the workflow

[risharde]

Hi guys, trying to determine how to format the $inputs variable in such a way that a 'list' is created. Read below for more details:

So we have a workflow that accepts inputs (files) as a 'list'

Each file is uploaded using the code: $file = tripal_galaxy_upload_file($galaxy_conn, $species_files_fid[$i], $history['id'], $history_contents);

And saved in the $galaxy_files array variable.

This file gets put into the $inputs variable like this: array_push($inputs, array( 'id' => $galaxy_files[$i]['id'], 'src' => $galaxy_files[$i]['hda_ldda'], ) );

This causes the $inputs array to show create something like $input[0], $input[1] etc each containing the file information.

I'm wondering though if Galaxy will understand that this is a 'list'.

Full code can be seen here (please excuse the code - this is development code): https://gitlab.com/TreeGenes/orthoquery_advanced/blob/master/includes/orthoquery_advanced_galaxy.inc#L212

You can see the workflow where you can see the inputs required should be in a 'list': https://gitlab.com/TreeGenes/orthoquery_advanced/blob/master/workflows/Galaxy-Workflow-OrthoQuery_proteome.ga

Any help would be appreciated? Thanks!

[spficklin]

Hi @risharde are you using the tripal_galaxy_get_workflow_defaults function to get a pre-populated inputs array?

[risharde]

Hi Stephen, thanks for responding, yes we are

On Mon, Jan 27, 2020, 2:56 PM Stephen Ficklin notifications@github.com wrote:

Hi @risharde https://github.com/risharde are you using the tripal_galaxy_get_workflow_defaults function to get a pre-populated inputs array?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tripal/tripal_galaxy/issues/156?email_source=notifications&email_token=AICMFI7MHR676VOHFBMJSJTQ74U6TA5CNFSM4KLNVS4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKAUSTY#issuecomment-578898255, or unsubscribe https://github.com/notifications/unsubscribe-auth/AICMFI3HCFVFJGA3ZEETJTDQ74U6TANCNFSM4KLNVS4A .

[spficklin]

I may be remembering wrong, but the array returned by that function call should have it setup the way you need, you just have to "fill in the blanks". Apologies, looking at the code you linked to above I see you are calling tripal_galaxy_get_workflow_defaults, but I don't see where you're setting the files in that $parameters array.

[risharde]

Hi Stephen, thanks again for responding so quickly, here's the dpm of the $paramaters variable based on the tripal_galaxy_get_workflow_defaults function:

[0] => Array
    (
        [collection_type] => list
    )

A little confused about that 'list' part in that I'm not sure how to generate items in there, maybe you can advise me?

The $inputs code can be seen here (that's the file inputs): https://gitlab.com/TreeGenes/orthoquery_advanced/blob/master/includes/orthoquery_advanced_galaxy.inc#L198

[risharde]

@spficklin Please refer to the post above, thanks for your help

[spficklin]

Hi @risharde okay. It looks like that's not helpful and we are missing documentation on how to set this up. So, here are some instructions. There may be some mistakes here, so let me know if something doesn't quite work.

Step 1: Make sure you have your history and history contents information. Below is example code.

// Get the history that we'll use for this submission.
$error = [];
$history_name = tripal_galaxy_get_history_name($submission);
$history = tripal_galaxy_create_history($galaxy, $history_name);
if (!$history) {
    $error = $galaxy->getError();
    throw new Exception($error['message']);
}

// Get the history contents so we don't upload the same file more than once
// in the event that this invocation occurs more than once.
$ghistory_contents = new GalaxyHistoryContents($galaxy);
$history_contents = $ghistory_contents->index([
    'history_id' => $history['id'],
]);
if ($error['message']) {
    $error = $galaxy->getError();
    throw new Exception($error['message']);
}

Step 2: Upload the files that will be in your collection on Galaxy. For this example I'm just using two files, but you can expand this to be as many as you need.

// Upload files. For this example we have two, where $fid1 and $fid2 are Drupal file IDs.
$dataset1 = tripal_galaxy_upload_file($galaxy, $fid1, $history['id'], $history_contents);
$dataset2 = tripal_galaxy_upload_file($galaxy, $fid2, $history['id'], $history_contents);

// Get the history contents again to get the IDs for the uploaded files.
  $history_contents = $ghistory_contents->index([ 'history_id' => $history['id']]);

Step3 Create the collection on the Galaxy server and tell it which files to include in it. You use the element_identifiers key to specify the files that belong to the collection, and you can add as many as you want.

// Create the array containing details about the dataset 
// collection you want to create on Galaxy.
$dataset_name = 'my_collection';
$ds_details = [
     'collection_type' => 'list',
     'instance_type' => 'history',
     'name' => $collection_name,
     'history_id' => $history['id'],
     'element_identifiers' => [
           [                
                'id' => $dataset1['id'],
                'name' => $dataset1['name'],
                'src' => 'hda',
          ],
          [                
                'id' => $dataset1['id'],
                'name' => $dataset1['name'],
                'src' => 'hda',
          ],
     ],
];

// Now create the dataset collection
$gds_collection = new GalaxyDatasetCollections($galaxy);
$ds_collection = $gds_collection->create($ds_details);
if (!$ds_collection) {
   $error = $galaxy->getError();
   throw new Exception($error['message']);
}

You should be able to build the $parameters and $input_datasets as you normally would and provide those to the tripal_galaxy_invoke_workflow() function.

[risharde]

Thanks @spficklin, I've tested the code and altered a few minor details into the dev module. Running the code now shows the following error:

Error: Call to a member function getURL() on null in GalaxyDatasetCollections->create() (line 45 of /var/www/Drupal/sites/all/libraries/blend4php/src/DataSetCollections.inc).

It seems to happen here: $ds_collection = $gds_collection->create($ds_details);

https://gitlab.com/TreeGenes/orthoquery_advanced/-/blob/master/includes/orthoquery_advanced_galaxy.inc#L433

The $ds_details variable dpm looks like this

Array ( [collection_type] => list [instance_type] => history [name] => my_collection_10a478ebd43bc60f [history_id] => 10a478ebd43bc60f [element_identifiers] => Array ( [0] => Array ( [id] => fb669667d64ed853 [name] => oqa_20200203_155815_29_Mycoplasma_agalactiae.faa [src] => hda )

        [1] => Array
            (
                [id] => 5514bac2eaaa5fe6
                [name] => oqa_20200203_155815_29_Mycoplasma_gallisepticum.faa
                [src] => hda
            )

        [2] => Array
            (
                [id] => 211577fe12512055
                [name] => oqa_20200203_155815_29_Mycoplasma_genitalium.faa
                [src] => hda
            )

        [3] => Array
            (
                [id] => 82f1466cd3b4f29d
                [name] => oqa_20200203_155815_29_Mycoplasma_hyopneumoniae.faa
                [src] => hda
            )

    )

)

Any ideas on what this error means or if you see something blatantly wrong with the code or variable generated @spficklin

Thanks so much for the help and support thus far

[spficklin]

Hi @risharde I think the problem is is on line 426 you have this code:

$gds_collection = new GalaxyDatasetCollections($galaxy);
if($debug) {
   //dpm($inputs);
   dpm('GDS_COLLECTION VARIABLE:');
  dpm($gds_collection);
}                   

$ds_collection = $gds_collection->create($ds_details);

I think you should pass in $galaxy_conn instead of $galaxy on that first line.