Upgrade Resolver for gears, analyses and lookup

[ehlertjd]

Resolver now uses ContainerStorage for projections and tree knowledge. Resolver also uses DB filtering to find the next node, rather than retrieving all children then filtering.

I've added indexes for [parent, label] to projects, sessions and acquisitions. This caused intermittent breakages in a few integration tests that were relying on DB returning records in insert-order - I believe that these are all fixed. Should I include a migration closure to drop the obsoleted indexes?

This implements virtual nodes for gears, analyses, and files. By default we return all children of a node, but you can select just the files or analyses by adding files or analyses to the path. e.g. resolving scitran/Neuroscience/session-01 will return all acquisitions, analyses and files as children, but resolving scitran/Neuroscience/session-01/files will return just the list of files. This resolves the filename ambiguity that is introduced in #1089.

Gears can be retrieved by using a path of /gears/gear-name or /gears/<id:gear-id>.

In addition, this adds a new /api/lookup endpoint that takes the same input as /api/resolve, but redirects to the appropriate GET handler for the resolved node. For example, performing a lookup of scitran/Neuroscience/session-01 will redirect (transparently on the server side) to the GET handler for /api/sessions/<session-01.id>. The one exception is if you perform a lookup on a file node, it will simply return the file node (without an info object).

Finally, in order to support the resolver/lookup in SDK2, polymorphism was added to resolver.json and a node-type defined for each type. This means that we need to set a container_type attribute in any response returned from /api/resolve or /api/lookup. In the case of lookup, this is done by setting a request environment variable fw_container_type and invoking the new utility function: util.add_container_type.

Breaking Changes

• You can no longer resolve files by naming them directly. For example: /scitran/Neuroscience/ses-01/acq/scan.nii.gz MUST become /scitran/Neuroscience/ses-01/acq/files/scan.nii.gz.
• Renamed node_type to container_type to be consistent with other places where we identify the container type (e.g. search)

Review Checklist

• Tests were added to cover all code changes
• Documentation was added / updated
• Code and tests follow standards in CONTRIBUTING.md

[codecov-io]

Codecov Report

Merging #1098 into swagger-fix-undocumented-codegen will increase coverage by 0.13%. The diff coverage is 98.85%.

@@                         Coverage Diff                          @@
##           swagger-fix-undocumented-codegen    #1098      +/-   ##
====================================================================
+ Coverage                             90.81%   90.94%   +0.13%     
====================================================================
  Files                                    50       50              
  Lines                                  7031     7146     +115     
====================================================================
+ Hits                                   6385     6499     +114     
- Misses                                  646      647       +1

[ehlertjd]

@kofalt I believe I've addressed most of your comments. resolver.py did get a little simpler:

Before

github.com/AlDanial/cloc v 1.76  T=1.03 s (1.0 files/s, 387.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                           1             86             85            226
-------------------------------------------------------------------------------

After

github.com/AlDanial/cloc v 1.76  T=1.05 s (1.0 files/s, 373.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                           1             90            104            198
-------------------------------------------------------------------------------