search

airr-community / adc-api

AIRR Data Commons (ADC) API reference implementation
GNU Affero General Public License v3.0
1 stars 0 forks source link

Testing adc-api: Feedback #1

Closed lfunderburk closed 4 years ago

lfunderburk commented 5 years ago

root@043fb607ab06:/work# src/ADCtest.sh https://airr-api.ireceptor.org/airr/v1 repertoire repertoire/pass*.json

Running test repertoire/pass-all.json repertoire ERROR: Expected 60 != 14 records

Running test repertoire/pass-and-op.json repertoire ERROR: Expected 10 != 0 records

Running test repertoire/pass-and-or-op.json repertoire ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire ERROR: Error code = 500, Message = b'{\n "message": "Expected $filter to have type \"array or object\" but found \"NULL\"",\n "exception": "MongoDB\\Exception\\InvalidArgumentException",\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Exception/InvalidArgumentException.php",\n "line": 32,\n "trace": [\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Operation/Find.php",\n "line": 117,\n "function": "invalidType",\n "class": "MongoDB\\Exception\\InvalidArgumentException",\n "type": "::"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Collection.php",\n "line": 523,\n "function": "__construct",\n "class": "MongoDB\\Operation\\Find",\n "type": "->"\n },\n {\n "function": "find",\n "class": "MongoDB\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/jenssegers/mongodb/src/Jenssegers/Mongodb/Collection.php",\n "line": 45,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Sample.php",\n "line": 77,\n "function": "__call",\n "class": "Jenssegers\\Mongodb\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Http/Controllers/AirrApiController.php",\n "line": 49,\n "function": "airrRepertoireRequest",\n "class": "App\\Sample",\n "type": "::"\n },\n {\n "function": "airr_repertoire",\n "class": "App\\Http\\Controllers\\AirrApiController",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Controller.php",\n "line": 54,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php",\n "line": 45,\n "function": "callAction",\n "class": "Illuminate\\Routing\\Controller",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 212,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\ControllerDispatcher",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 169,\n "function": "runController",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 625,\n "function": "run",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php",\n "line": 41,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\SubstituteBindings",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php",\n "line": 57,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\ThrottleRequests",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 627,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 586,\n "function": "runRouteWithinStack",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 564,\n "function": "dispatchToRoute",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 176,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Foundation\\Http\\{closure}",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/fideloper/proxy/src/TrustProxies.php",\n "line": 56,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Fideloper\\Proxy\\TrustProxies",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php",\n "line": 27,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php",\n "line": 46,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 151,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 116,\n "function": "sendRequestThroughRouter",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/public/index.php",\n "line": 53,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n }\n ]\n}' ERROR: Query file repertoire/pass-and-or-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-boolean-equals-op.json repertoire ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire ERROR: Error code = 400, Message = b'{"message":"Unable to parse the filter."}' ERROR: Query file repertoire/pass-boolean-equals-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-boolean-not-equals-op.json repertoire ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire ERROR: Error code = 400, Message = b'{"message":"Unable to parse the filter."}' ERROR: Query file repertoire/pass-boolean-not-equals-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-contains-op.json repertoire ERROR: Expected 10 != 0 records

Running test repertoire/pass-deep-and-op.json repertoire ERROR: Expected 10 != 0 records

Running test repertoire/pass-equals-op.json repertoire ERROR: Expected 40 != 0 records

Running test repertoire/pass-exclude-op.json repertoire ERROR: Expected 57 != 14 records

Running test repertoire/pass-facets1_repertoire.json repertoire ERROR: Expected 2 != 1 records

Running test repertoire/pass-facets2_repertoire.json repertoire ERROR: Expected 10 != 0 records

Running test repertoire/pass-field-organism-id.json repertoire WARNING: No gold expectation for pass-field-organism-id.json INFO: Received 0 Repertoires from query PASS: Query file repertoire/pass-field-organism-id.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-field-organism-value.json repertoire WARNING: No gold expectation for pass-field-organism-value.json INFO: Received 14 Repertoires from query PASS: Query file repertoire/pass-field-organism-value.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-field-pcr_target_locus.json repertoire WARNING: No gold expectation for pass-field-pcr_target_locus.json INFO: Received 0 Repertoires from query PASS: Query file repertoire/pass-field-pcr_target_locus.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-float-from.json repertoire ERROR: Expected 50 != 14 records

Running test repertoire/pass-float-size.json repertoire ERROR: Expected 10 != 14 records

Running test repertoire/pass-greater-than-equals-op.json repertoire ERROR: Expected 20 != 5 records

Running test repertoire/pass-greater-than-op.json repertoire ERROR: Expected 10 != 5 records

Running test repertoire/pass-in-op.json repertoire ERROR: Expected 3 != 0 records

Running test repertoire/pass-is-op.json repertoire ERROR: Expected 30 != 14 records

Running test repertoire/pass-less-than-equals-op.json repertoire ERROR: Expected 20 != 0 records

Running test repertoire/pass-less-than-op.json repertoire ERROR: Expected 10 != 0 records

Running test repertoire/pass-not-equals-op.json repertoire ERROR: Expected 20 != 14 records

Running test repertoire/pass-not-op.json repertoire ERROR: Expected 30 != 14 records

Running test repertoire/pass-number-contains-op.json repertoire ERROR: Expected 40 != 0 records

Running test repertoire/pass-number-equals-op.json repertoire ERROR: Expected 10 != 0 records

Running test repertoire/pass-number-exclude-op.json repertoire ERROR: Expected 40 != 14 records

Running test repertoire/pass-number-in-op.json repertoire ERROR: Expected 20 != 0 records

Running test repertoire/pass-number-not-equals-op.json repertoire ERROR: Expected 50 != 14 records

Running test repertoire/pass-or-op.json repertoire ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire ERROR: Error code = 500, Message = b'{\n "message": "Expected $filter to have type \"array or object\" but found \"NULL\"",\n "exception": "MongoDB\\Exception\\InvalidArgumentException",\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Exception/InvalidArgumentException.php",\n "line": 32,\n "trace": [\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Operation/Find.php",\n "line": 117,\n "function": "invalidType",\n "class": "MongoDB\\Exception\\InvalidArgumentException",\n "type": "::"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Collection.php",\n "line": 523,\n "function": "__construct",\n "class": "MongoDB\\Operation\\Find",\n "type": "->"\n },\n {\n "function": "find",\n "class": "MongoDB\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/jenssegers/mongodb/src/Jenssegers/Mongodb/Collection.php",\n "line": 45,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Sample.php",\n "line": 77,\n "function": "__call",\n "class": "Jenssegers\\Mongodb\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Http/Controllers/AirrApiController.php",\n "line": 49,\n "function": "airrRepertoireRequest",\n "class": "App\\Sample",\n "type": "::"\n },\n {\n "function": "airr_repertoire",\n "class": "App\\Http\\Controllers\\AirrApiController",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Controller.php",\n "line": 54,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php",\n "line": 45,\n "function": "callAction",\n "class": "Illuminate\\Routing\\Controller",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 212,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\ControllerDispatcher",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 169,\n "function": "runController",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 625,\n "function": "run",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php",\n "line": 41,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\SubstituteBindings",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php",\n "line": 57,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\ThrottleRequests",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 627,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 586,\n "function": "runRouteWithinStack",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 564,\n "function": "dispatchToRoute",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 176,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Foundation\\Http\\{closure}",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/fideloper/proxy/src/TrustProxies.php",\n "line": 56,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Fideloper\\Proxy\\TrustProxies",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php",\n "line": 27,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php",\n "line": 46,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 151,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 116,\n "function": "sendRequestThroughRouter",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/public/index.php",\n "line": 53,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n }\n ]\n}' ERROR: Query file repertoire/pass-or-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-query1-2_repertoire.json repertoire WARNING: No gold expectation for pass-query1-2_repertoire.json INFO: Received 0 Repertoires from query PASS: Query file repertoire/pass-query1-2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-query2_repertoire.json repertoire WARNING: No gold expectation for pass-query2_repertoire.json INFO: Received 0 Repertoires from query PASS: Query file repertoire/pass-query2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

lfunderburk commented 5 years ago

@bcorrie @schristley

Tried running test suite against https://airr-api.ireceptor.org/airr/v1/ and obtained odd results.

Raising a flag in case this was not seen on your end before.

schristley commented 5 years ago

@lfunderburk The tests that give an error because the number of records returned is not the expected number are likely okay because your repository doesn't hold the test data set (Brian and I talked about the test suite having a flag to not require that exact number)

ERROR: Expected 50 != 14 records

The others are presumably code bugs as all pass*.json files should be valid queries.

You should also run the fail*.json cases to verify that the server is properly returning a 400 error code (and also not causing the server to crash).

schristley commented 5 years ago

@lfunderburk fyi, here are the results from running against the reference implementation

root@b2f9a8b226fc:/work# src/ADCtest.sh http://localhost:8080/airr/v1 repertoire repertoire/*.json

Running test repertoire/fail-bogus-op.json
PASS: Query file repertoire/fail-bogus-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-and-op.json
PASS: Query file repertoire/fail-content-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-contains-op.json
PASS: Query file repertoire/fail-content-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-equals-op.json
PASS: Query file repertoire/fail-content-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-exclude-op.json
PASS: Query file repertoire/fail-content-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-greater-than-equals-op.json
PASS: Query file repertoire/fail-content-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-greater-than-op.json
PASS: Query file repertoire/fail-content-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-in-op.json
PASS: Query file repertoire/fail-content-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-is-op.json
PASS: Query file repertoire/fail-content-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-less-than-equals-op.json
PASS: Query file repertoire/fail-content-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-less-than-op.json
PASS: Query file repertoire/fail-content-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-not-equals-op.json
PASS: Query file repertoire/fail-content-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-not-op.json
PASS: Query file repertoire/fail-content-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content-or-op.json
PASS: Query file repertoire/fail-content-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-and-op.json
PASS: Query file repertoire/fail-content2-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-contains-op.json
PASS: Query file repertoire/fail-content2-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-equals-op.json
PASS: Query file repertoire/fail-content2-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-exclude-op.json
PASS: Query file repertoire/fail-content2-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-greater-than-equals-op.json
PASS: Query file repertoire/fail-content2-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-greater-than-op.json
PASS: Query file repertoire/fail-content2-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-in-op.json
PASS: Query file repertoire/fail-content2-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-is-op.json
PASS: Query file repertoire/fail-content2-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-less-than-equals-op.json
PASS: Query file repertoire/fail-content2-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-less-than-op.json
PASS: Query file repertoire/fail-content2-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-not-equals-op.json
PASS: Query file repertoire/fail-content2-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-not-op.json
PASS: Query file repertoire/fail-content2-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content2-or-op.json
PASS: Query file repertoire/fail-content2-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content3-and-op.json
PASS: Query file repertoire/fail-content3-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content3-or-op.json
PASS: Query file repertoire/fail-content3-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content4-and-op.json
PASS: Query file repertoire/fail-content4-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-content4-or-op.json
PASS: Query file repertoire/fail-content4-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-contains-op.json
PASS: Query file repertoire/fail-field-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-equals-op.json
PASS: Query file repertoire/fail-field-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-exclude-op.json
PASS: Query file repertoire/fail-field-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-greater-than-equals-op.json
PASS: Query file repertoire/fail-field-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-greater-than-op.json
PASS: Query file repertoire/fail-field-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-in-op.json
PASS: Query file repertoire/fail-field-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-is-op.json
PASS: Query file repertoire/fail-field-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-less-than-equals-op.json
PASS: Query file repertoire/fail-field-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-less-than-op.json
PASS: Query file repertoire/fail-field-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-not-equals-op.json
PASS: Query file repertoire/fail-field-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field-not-op.json
PASS: Query file repertoire/fail-field-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-contains-op.json
PASS: Query file repertoire/fail-field2-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-equals-op.json
PASS: Query file repertoire/fail-field2-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-exclude-op.json
PASS: Query file repertoire/fail-field2-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-greater-than-equals-op.json
PASS: Query file repertoire/fail-field2-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-greater-than-op.json
PASS: Query file repertoire/fail-field2-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-in-op.json
PASS: Query file repertoire/fail-field2-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-is-op.json
PASS: Query file repertoire/fail-field2-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-less-than-equals-op.json
PASS: Query file repertoire/fail-field2-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-less-than-op.json
PASS: Query file repertoire/fail-field2-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-not-equals-op.json
PASS: Query file repertoire/fail-field2-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-field2-not-op.json
PASS: Query file repertoire/fail-field2-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-format.json
PASS: Query file repertoire/fail-format.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-missing-op.json
PASS: Query file repertoire/fail-missing-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-negative-from.json
PASS: Query file repertoire/fail-negative-from.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-negative-size.json
PASS: Query file repertoire/fail-negative-size.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-number-equals-op.json
PASS: Query file repertoire/fail-number-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-number-not-equals-op.json
PASS: Query file repertoire/fail-number-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-query-json.json
WARNING: JSON Decode error detected in repertoire/fail-query-json.json: Expecting ',' delimiter: line 8 column 18 (char 120)
PASS: Query file repertoire/fail-query-json.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-query-structure.json
PASS: Query file repertoire/fail-query-structure.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-string-equals-op.json
PASS: Query file repertoire/fail-string-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-string-from.json
PASS: Query file repertoire/fail-string-from.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-string-greater-than-equals-op.json
PASS: Query file repertoire/fail-string-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-string-greater-than-op.json
PASS: Query file repertoire/fail-string-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-string-less-than-equals-op.json
PASS: Query file repertoire/fail-string-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-string-less-than-op.json
PASS: Query file repertoire/fail-string-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-string-size.json
PASS: Query file repertoire/fail-string-size.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-contains-op.json
PASS: Query file repertoire/fail-value-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-equals-op.json
PASS: Query file repertoire/fail-value-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-exclude-op.json
PASS: Query file repertoire/fail-value-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-greater-than-equals-op.json
PASS: Query file repertoire/fail-value-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-greater-than-op.json
PASS: Query file repertoire/fail-value-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-in-op.json
PASS: Query file repertoire/fail-value-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-less-than-equals-op.json
PASS: Query file repertoire/fail-value-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-less-than-op.json
PASS: Query file repertoire/fail-value-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value-not-equals-op.json
PASS: Query file repertoire/fail-value-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value2-equals-op.json
PASS: Query file repertoire/fail-value2-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value2-exclude-op.json
PASS: Query file repertoire/fail-value2-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/fail-value2-in-op.json
PASS: Query file repertoire/fail-value2-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-all.json
INFO: Received 60 Repertoires from query
PASS: Query file repertoire/pass-all.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-and-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-and-or-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-and-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-boolean-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-boolean-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-boolean-not-equals-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-boolean-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-contains-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-deep-and-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-deep-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-equals-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-exclude-op.json
INFO: Received 57 Repertoires from query
PASS: Query file repertoire/pass-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-facets1_repertoire.json
INFO: Received 2 Facets from query
PASS: Query file repertoire/pass-facets1_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-facets2_repertoire.json
INFO: Received 10 Facets from query
PASS: Query file repertoire/pass-facets2_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-field-organism-id.json
WARNING: No gold expectation for pass-field-organism-id.json
INFO: Received 60 Repertoires from query
PASS: Query file repertoire/pass-field-organism-id.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-field-organism-value.json
WARNING: No gold expectation for pass-field-organism-value.json
INFO: Received 60 Repertoires from query
PASS: Query file repertoire/pass-field-organism-value.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-field-pcr_target_locus.json
WARNING: No gold expectation for pass-field-pcr_target_locus.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-field-pcr_target_locus.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-float-from.json
INFO: Received 50 Repertoires from query
PASS: Query file repertoire/pass-float-from.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-float-size.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-float-size.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-greater-than-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-greater-than-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-in-op.json
INFO: Received 3 Repertoires from query
PASS: Query file repertoire/pass-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-is-op.json
INFO: Received 30 Repertoires from query
PASS: Query file repertoire/pass-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-less-than-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-less-than-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-not-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-not-op.json
INFO: Received 30 Repertoires from query
PASS: Query file repertoire/pass-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-number-contains-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-number-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-number-equals-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-number-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-number-exclude-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-number-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-number-in-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-number-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-number-not-equals-op.json
INFO: Received 50 Repertoires from query
PASS: Query file repertoire/pass-number-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-or-op.json
INFO: Received 30 Repertoires from query
PASS: Query file repertoire/pass-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-query1-2_repertoire.json
WARNING: No gold expectation for pass-query1-2_repertoire.json
INFO: Received 5 Repertoires from query
PASS: Query file repertoire/pass-query1-2_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

Running test repertoire/pass-query2_repertoire.json
WARNING: No gold expectation for pass-query2_repertoire.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-query2_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK

SUMMARY: All tests passed!!!
bcorrie commented 5 years ago

@lfunderburk @schristley I added a -g flag to disable the gold testing (https://github.com/airr-community/adc-api-tests/commit/7c22d340804841dd1451192d84dade15c3c2b3df)... It will now run all the tests and ignore the gold standard results test if you give it the -g flag. Necessary for testing against a repository that doesn't have just the gold standard data set loaded...

bcorrie commented 5 years ago

Run against the iReceptor ADC implementation we now get most queries passing (still 4 failing).

Running test repertoire/pass-query1-2_repertoire.json INFO: Received 0 Repertoires from query PASS: Query file repertoire/pass-query1-2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-query2_repertoire.json INFO: Received 0 Repertoires from query PASS: Query file repertoire/pass-query2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

SUMMARY: 4 of 32 tests failed

schristley commented 4 years ago

Sounds like the test suite is fine, and any iReceptor ADC bugs you will handle in your own issue tracker, so closing this.