Open bisgaard-itis opened 3 months ago
bisgaard-itis
commented
3 months ago I also see client e2e tests failing for this reason: https://git.speag.com/oSparc/e2e-portal-testing/-/pipelines/782003/test_report?job_name=python_client_e2e_test%3A+%5Blatest_master%5D
bisgaard-itis
commented
3 months ago this is related/equal to https://github.com/ITISFoundation/osparc-simcore/issues/5696
bisgaard-itis
commented
3 months ago When profiling the start endpoint in director v2 while running 100 jobs as part of the metamodeling toy project this is what I see:
Return status: 201
{'id': '25ac7f2a-2a4b-11ef-a5ad-02420a00f1ba', 'state': 'PUBLISHED', 'result': None, 'pipeline_details': {'adjacency_list': {'42092309-01a2-5db7-bd53-ca4dae295012': []}, 'progress': 0.0, 'node_states': {'42092309-01a2-5db7-bd53-ca4dae295012': {'modified': True, 'dependencies': [], 'currentStatus': 'PUBLISHED', 'progress': None}}}, 'iteration': 1, 'cluster_id': 0, 'started': None, 'stopped': None, 'submitted': '2024-06-14T12:42:07.707421+00:00', 'url': 'http://master_director-v2:8000/v2/computations/25ac7f2a-2a4b-11ef-a5ad-02420a00f1ba?user_id=61349', 'stop_url': 'http://master_director-v2:8000/v2/computations/25ac7f2a-2a4b-11ef-a5ad-02420a00f1ba:stop?user_id=61349'}
_ ._ __/__ _ _ _ _ _/_ Recorded: 12:42:05 Samples: 8454
/_//_/// /_\ / //_// / //_'/ // Duration: 8.884 CPU time: 8.828
/ _/ v4.6.2
Program: /home/scu/.venv/bin/uvicorn simcore_service_director_v2.main:the_app --host 0.0.0.0 --log-level debug
8.881 run asyncio/runners.py:8
├─ 4.808 worker servicelib/async_utils.py:150
│ ├─ 4.805 create_computation simcore_service_director_v2/api/routes/computations.py:265
│ │ ├─ 2.702 _try_start_pipeline simcore_service_director_v2/api/routes/computations.py:210
│ │ │ ├─ 1.825 DaskScheduler.run_new_pipeline simcore_service_director_v2/modules/comp_scheduler/base_scheduler.py:149
│ │ │ │ ├─ 1.246 CompRunsRepository.create simcore_service_director_v2/modules/db/repositories/comp_runs.py:76
│ │ │ │ │ ├─ 1.227 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ │ │ └─ 1.227 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ │ │ └─ 1.227 Pool._acquire aiopg/pool.py:304
│ │ │ │ │ │ └─ 1.227 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ │ │ └─ 1.227 Lock.acquire asyncio/locks.py:93
│ │ │ │ │ │ └─ 1.227 [await] asyncio/locks.py
│ │ │ │ │ ├─ 0.011 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ ├─ 0.010 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ │ │ └─ 0.010 Connection._poll aiopg/connection.py:874
│ │ │ │ │ │ │ └─ 0.010 wait_for asyncio/tasks.py:392
│ │ │ │ │ │ │ └─ 0.010 [await] asyncio/tasks.py
│ │ │ │ │ │ └─ 0.001 Insert.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ └─ 0.001 Insert._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ └─ 0.001 Insert._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_insert sqlalchemy/sql/compiler.py:3991
│ │ │ │ │ │ └─ 0.001 _get_crud_params sqlalchemy/sql/crud.py:38
│ │ │ │ │ │ └─ 0.001 <setcomp> sqlalchemy/sql/crud.py:88
│ │ │ │ │ │ └─ 0.001 expect_as_key sqlalchemy/sql/coercions.py:224
│ │ │ │ │ │ └─ 0.001 expect sqlalchemy/sql/coercions.py:112
│ │ │ │ │ │ └─ 0.001 DMLColumnImpl._implicit_coercions sqlalchemy/sql/coercions.py:311
│ │ │ │ │ ├─ 0.005 SAConnection.scalar aiopg/sa/connection.py:178
│ │ │ │ │ │ └─ 0.005 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ └─ 0.005 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ │ └─ 0.005 Connection._poll aiopg/connection.py:874
│ │ │ │ │ │ └─ 0.005 wait_for asyncio/tasks.py:392
│ │ │ │ │ │ └─ 0.005 [await] asyncio/tasks.py
│ │ │ │ │ ├─ 0.001 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ │ │ │ └─ 0.001 _close_connection aiopg/sa/engine.py:115
│ │ │ │ │ │ └─ 0.001 SAConnection.close aiopg/sa/connection.py:384
│ │ │ │ │ │ └─ 0.001 [await] aiopg/sa/connection.py
│ │ │ │ │ ├─ 0.001 BinaryExpression.__and__ sqlalchemy/sql/operators.py:59
│ │ │ │ │ │ └─ 0.001 BinaryExpression.operate sqlalchemy/sql/elements.py:872
│ │ │ │ │ │ └─ 0.001 Comparator.__and__ sqlalchemy/sql/operators.py:59
│ │ │ │ │ │ └─ 0.001 Comparator.operate sqlalchemy/sql/type_api.py:73
│ │ │ │ │ │ └─ 0.001 _conjunction_operate sqlalchemy/sql/default_comparator.py:145
│ │ │ │ │ │ └─ 0.001 BooleanClauseList.and_ sqlalchemy/sql/elements.py:2579
│ │ │ │ │ │ └─ 0.001 BooleanClauseList._construct sqlalchemy/sql/elements.py:2504
│ │ │ │ │ │ └─ 0.001 BooleanClauseList._process_clauses_for_boolean sqlalchemy/sql/elements.py:2465
│ │ │ │ │ │ └─ 0.001 <listcomp> sqlalchemy/sql/elements.py:2502
│ │ │ │ │ │ └─ 0.001 BinaryExpression.self_group sqlalchemy/sql/elements.py:3946
│ │ │ │ │ └─ 0.001 [self] simcore_service_director_v2/modules/db/repositories/comp_runs.py
│ │ │ │ ├─ 0.547 DaskScheduler._get_pipeline_dag simcore_service_director_v2/modules/comp_scheduler/base_scheduler.py:241
│ │ │ │ │ └─ 0.547 CompPipelinesRepository.get_pipeline simcore_service_director_v2/modules/db/repositories/comp_pipelines.py:19
│ │ │ │ │ ├─ 0.534 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ │ │ └─ 0.534 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ │ │ └─ 0.534 Pool._acquire aiopg/pool.py:304
│ │ │ │ │ │ └─ 0.534 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ │ │ └─ 0.534 Lock.acquire asyncio/locks.py:93
│ │ │ │ │ │ └─ 0.534 [await] asyncio/locks.py
│ │ │ │ │ ├─ 0.009 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ ├─ 0.008 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ │ │ └─ 0.008 Connection._poll aiopg/connection.py:874
│ │ │ │ │ │ │ └─ 0.008 wait_for asyncio/tasks.py:392
│ │ │ │ │ │ │ └─ 0.008 [await] asyncio/tasks.py
│ │ │ │ │ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ │ └─ 0.001 NoneType._compose_select_body sqlalchemy/sql/compiler.py:3650
│ │ │ │ │ ├─ 0.003 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ │ │ │ └─ 0.003 _close_connection aiopg/sa/engine.py:115
│ │ │ │ │ │ └─ 0.003 SAConnection.close aiopg/sa/connection.py:384
│ │ │ │ │ │ └─ 0.003 [await] aiopg/sa/connection.py
│ │ │ │ │ └─ 0.001 ResultProxy.fetchone aiopg/sa/result.py:388
│ │ │ │ │ └─ 0.001 ResultProxy._process_rows aiopg/sa/result.py:370
│ │ │ │ └─ 0.032 publish_project_log simcore_service_director_v2/utils/rabbitmq.py:184
│ │ │ │ └─ 0.032 RabbitMQClient.publish servicelib/rabbitmq/_client.py:304
│ │ │ │ ├─ 0.029 RobustExchange.publish aio_pika/exchange.py:163
│ │ │ │ │ ├─ 0.028 Channel.basic_publish aiormq/channel.py:618
│ │ │ │ │ │ └─ 0.028 Countdown.__call__ aiormq/tools.py:85
│ │ │ │ │ │ ├─ 0.016 [await] aiormq/tools.py
│ │ │ │ │ │ └─ 0.012 Queue.put asyncio/queues.py:111
│ │ │ │ │ │ └─ 0.012 [await] asyncio/queues.py
│ │ │ │ │ └─ 0.001 Message.properties aio_pika/message.py:257
│ │ │ │ │ └─ 0.001 Properties.__init__ pamqp/commands.py:2644
│ │ │ │ │ └─ 0.001 Properties.validate pamqp/base.py:202
│ │ │ │ ├─ 0.001 LoggerRabbitMessage.body models_library/rabbitmq_messages.py:48
│ │ │ │ │ └─ 0.001 dumps json/__init__.py:183
│ │ │ │ │ └─ 0.001 JSONEncoder.encode json/encoder.py:182
│ │ │ │ │ └─ 0.001 JSONEncoder.iterencode json/encoder.py:204
│ │ │ │ │ └─ 0.001 UUID.__str__ uuid.py:279
│ │ │ │ ├─ 0.001 PoolItemContextManager.__aenter__ aio_pika/pool.py:145
│ │ │ │ │ └─ 0.001 Pool._get aio_pika/pool.py:97
│ │ │ │ │ └─ 0.001 Pool._create_item aio_pika/pool.py:83
│ │ │ │ │ └─ 0.001 RabbitMQClient._get_channel servicelib/rabbitmq/_client.py:129
│ │ │ │ │ └─ 0.001 RobustChannel.__await__ aio_pika/channel.py:42
│ │ │ │ │ └─ 0.001 RobustChannel.initialize aio_pika/channel.py:179
│ │ │ │ │ └─ 0.001 RobustChannel._open aio_pika/channel.py:155
│ │ │ │ │ └─ 0.001 UnderlayChannel.create aio_pika/abc.py:478
│ │ │ │ │ └─ 0.001 Connection.channel aiormq/connection.py:816
│ │ │ │ │ └─ 0.001 Channel.__init__ aiormq/channel.py:81
│ │ │ │ │ └─ 0.001 Queue.__init__ asyncio/queues.py:33
│ │ │ │ └─ 0.001 RobustChannel.declare_exchange aio_pika/robust_channel.py:167
│ │ │ │ └─ 0.001 RobustChannel.declare_exchange aio_pika/channel.py:225
│ │ │ │ └─ 0.001 RobustExchange.declare aio_pika/exchange.py:52
│ │ │ │ └─ 0.001 Channel.exchange_declare aiormq/channel.py:730
│ │ │ │ └─ 0.001 Channel.wrap aiormq/base.py:162
│ │ │ │ └─ 0.001 TaskWrapper.__inner aiormq/abc.py:42
│ │ │ │ └─ 0.001 [await] aiormq/abc.py
│ │ │ ├─ 0.545 _get_project_metadata simcore_service_director_v2/api/routes/computations.py:153
│ │ │ │ └─ 0.545 ProjectsMetadataRepository.get_project_ancestors simcore_service_director_v2/modules/db/repositories/projects_metadata.py:22
│ │ │ │ ├─ 0.539 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ │ └─ 0.539 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ │ └─ 0.539 Pool._acquire aiopg/pool.py:304
│ │ │ │ │ └─ 0.539 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ │ └─ 0.539 Lock.acquire asyncio/locks.py:93
│ │ │ │ │ └─ 0.539 [await] asyncio/locks.py
│ │ │ │ ├─ 0.004 get simcore_postgres_database/utils_projects_metadata.py:67
│ │ │ │ │ └─ 0.004 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ ├─ 0.003 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ │ └─ 0.003 Connection._poll aiopg/connection.py:874
│ │ │ │ │ │ └─ 0.003 wait_for asyncio/tasks.py:392
│ │ │ │ │ │ └─ 0.003 [await] asyncio/tasks.py
│ │ │ │ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ └─ 0.001 CompileState.create_for_statement sqlalchemy/sql/base.py:486
│ │ │ │ │ └─ 0.001 SelectState.create_for_statement sqlalchemy/sql/base.py:486
│ │ │ │ │ └─ 0.001 SelectState.__init__ sqlalchemy/sql/selectable.py:4382
│ │ │ │ │ └─ 0.001 Select._generate_columns_plus_names sqlalchemy/sql/selectable.py:6195
│ │ │ │ │ └─ 0.001 memoized_attribute.__get__ sqlalchemy/util/langhelpers.py:1181
│ │ │ │ │ └─ 0.001 Select._all_selected_columns sqlalchemy/sql/selectable.py:6185
│ │ │ │ └─ 0.002 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ │ └─ 0.002 _close_connection aiopg/sa/engine.py:115
│ │ │ │ └─ 0.002 SAConnection.close aiopg/sa/connection.py:384
│ │ │ │ └─ 0.002 [await] aiopg/sa/connection.py
│ │ │ └─ 0.332 UsersRepository.get_user_email simcore_service_director_v2/modules/db/repositories/users.py:10
│ │ │ ├─ 0.325 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ └─ 0.325 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ └─ 0.325 Pool._acquire aiopg/pool.py:304
│ │ │ │ └─ 0.325 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ └─ 0.325 Lock.acquire asyncio/locks.py:93
│ │ │ │ └─ 0.325 [await] asyncio/locks.py
│ │ │ ├─ 0.005 get_email simcore_postgres_database/utils_users.py:168
│ │ │ │ └─ 0.005 SAConnection.scalar aiopg/sa/connection.py:178
│ │ │ │ └─ 0.005 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ └─ 0.005 Cursor.execute aiopg/connection.py:400
│ │ │ │ └─ 0.005 Connection._poll aiopg/connection.py:874
│ │ │ │ └─ 0.005 wait_for asyncio/tasks.py:392
│ │ │ │ └─ 0.005 [await] asyncio/tasks.py
│ │ │ └─ 0.002 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ └─ 0.002 _close_connection aiopg/sa/engine.py:115
│ │ │ └─ 0.002 SAConnection.close aiopg/sa/connection.py:384
│ │ │ └─ 0.002 [await] aiopg/sa/connection.py
│ │ ├─ 0.768 CompTasksRepository.upsert_tasks_from_project simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py:88
│ │ │ ├─ 0.497 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ └─ 0.497 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ └─ 0.497 Pool._acquire aiopg/pool.py:304
│ │ │ │ └─ 0.497 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ └─ 0.497 Lock.acquire asyncio/locks.py:93
│ │ │ │ └─ 0.497 [await] asyncio/locks.py
│ │ │ ├─ 0.189 generate_tasks_list_from_project simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py:325
│ │ │ │ ├─ 0.182 _generate_task_image simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py:154
│ │ │ │ │ ├─ 0.156 CatalogClient.get_service_resources simcore_service_director_v2/modules/catalog.py:83
│ │ │ │ │ │ ├─ 0.155 wrapper_func simcore_service_director_v2/utils/client_decorators.py:55
│ │ │ │ │ │ │ └─ 0.155 async_wrapped tenacity/_asyncio.py:138
│ │ │ │ │ │ │ ├─ 0.154 AsyncRetrying.__call__ tenacity/_asyncio.py:51
│ │ │ │ │ │ │ │ └─ 0.154 CatalogClient.request simcore_service_director_v2/modules/catalog.py:59
│ │ │ │ │ │ │ │ └─ 0.154 AsyncClient.request httpx/_client.py:1519
│ │ │ │ │ │ │ │ ├─ 0.146 AsyncClient.send httpx/_client.py:1628
│ │ │ │ │ │ │ │ │ ├─ 0.131 AsyncClient._send_handling_auth httpx/_client.py:1677
│ │ │ │ │ │ │ │ │ │ └─ 0.131 AsyncClient._send_handling_redirects httpx/_client.py:1711
│ │ │ │ │ │ │ │ │ │ └─ 0.131 AsyncClient._send_single_request httpx/_client.py:1749
│ │ │ │ │ │ │ │ │ │ ├─ 0.130 AsyncHTTPTransport.handle_async_request httpx/_transports/default.py:354
│ │ │ │ │ │ │ │ │ │ │ └─ 0.130 AsyncConnectionPool.handle_async_request httpcore/_async/connection_pool.py:159
│ │ │ │ │ │ │ │ │ │ │ ├─ 0.125 AsyncHTTPConnection.handle_async_request httpcore/_async/connection.py:67
│ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.112 AsyncHTTP11Connection.handle_async_request httpcore/_async/http11.py:72
│ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.083 AsyncHTTP11Connection._receive_response_headers httpcore/_async/http11.py:179
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.083 AsyncHTTP11Connection._receive_event httpcore/_async/http11.py:216
│ │ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.081 AnyIOStream.read httpcore/_backends/anyio.py:23
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.081 SocketStream.receive anyio/_backends/_asyncio.py:1124
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.070 Event.wait asyncio/locks.py:201
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.070 [await] asyncio/locks.py
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 [await] asyncio/tasks.py
│ │ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.001 Connection.next_event h11/_connection.py:426
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 Connection._process_event h11/_connection.py:260
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 Connection._respond_to_state_changes h11/_connection.py:330
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 Connection.receive_data h11/_connection.py:352
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 ReceiveBuffer.__iadd__ h11/_receivebuffer.py:53
│ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.017 AsyncHTTP11Connection._send_request_headers httpcore/_async/http11.py:147
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.017 AsyncHTTP11Connection._send_event httpcore/_async/http11.py:170
│ │ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.016 AnyIOStream.write httpcore/_backends/anyio.py:39
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.016 SocketStream.send anyio/_backends/_asyncio.py:1158
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.015 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.015 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.015 [await] asyncio/tasks.py
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 TCPTransport.write <built-in>
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 Connection.send h11/_connection.py:491
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 Connection.send_with_data_passthrough h11/_connection.py:518
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 Connection._process_event h11/_connection.py:260
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 Connection._respond_to_state_changes h11/_connection.py:330
│ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.011 AsyncLock.__aenter__ httpcore/_synchronization.py:69
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 Lock.acquire anyio/_core/_synchronization.py:156
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 cancel_shielded_checkpoint anyio/lowlevel.py:42
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 AsyncIOBackend.cancel_shielded_checkpoint anyio/_backends/_asyncio.py:2038
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.011 [await] asyncio/tasks.py
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 [self] httpcore/_async/http11.py
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.013 AsyncLock.__aenter__ httpcore/_synchronization.py:69
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.013 Lock.acquire anyio/_core/_synchronization.py:156
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.013 cancel_shielded_checkpoint anyio/lowlevel.py:42
│ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.012 AsyncIOBackend.cancel_shielded_checkpoint anyio/_backends/_asyncio.py:2038
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.012 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.012 [await] asyncio/tasks.py
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 [self] anyio/lowlevel.py
│ │ │ │ │ │ │ │ │ │ │ ├─ 0.003 AsyncConnectionPool._close_connections httpcore/_async/connection_pool.py:301
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.003 AsyncShieldCancellation.__enter__ httpcore/_synchronization.py:209
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.003 CancelScope.__enter__ anyio/_backends/_asyncio.py:366
│ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.001 WeakKeyDictionary.__getitem__ weakref.py:415
│ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.001 cast typing.py:1737
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 current_task asyncio/tasks.py:35
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 dict.get <built-in>
│ │ │ │ │ │ │ │ │ │ │ └─ 0.002 AsyncConnectionPool._assign_requests_to_connections httpcore/_async/connection_pool.py:230
│ │ │ │ │ │ │ │ │ │ │ ├─ 0.001 <listcomp> httpcore/_async/connection_pool.py:254
│ │ │ │ │ │ │ │ │ │ │ └─ 0.001 AsyncHTTPConnection.has_expired httpcore/_async/connection.py:185
│ │ │ │ │ │ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection.has_expired httpcore/_async/http11.py:281
│ │ │ │ │ │ │ │ │ │ └─ 0.001 Logger.info logging/__init__.py:1467
│ │ │ │ │ │ │ │ │ │ └─ 0.001 Logger._log logging/__init__.py:1600
│ │ │ │ │ │ │ │ │ │ └─ 0.001 Logger.makeRecord logging/__init__.py:1585
│ │ │ │ │ │ │ │ │ └─ 0.015 Response.aread httpx/_models.py:906
│ │ │ │ │ │ │ │ │ └─ 0.015 <listcomp> httpx/_models.py:911
│ │ │ │ │ │ │ │ │ ├─ 0.009 Response.aiter_bytes httpx/_models.py:914
│ │ │ │ │ │ │ │ │ │ └─ 0.009 Response.aiter_raw httpx/_models.py:969
│ │ │ │ │ │ │ │ │ │ ├─ 0.005 Response.aclose httpx/_models.py:997
│ │ │ │ │ │ │ │ │ │ │ └─ 0.005 BoundAsyncStream.aclose httpx/_client.py:152
│ │ │ │ │ │ │ │ │ │ │ └─ 0.005 AsyncResponseStream.aclose httpx/_transports/default.py:257
│ │ │ │ │ │ │ │ │ │ │ └─ 0.005 PoolByteStream.aclose httpcore/_async/connection_pool.py:369
│ │ │ │ │ │ │ │ │ │ │ ├─ 0.004 AsyncConnectionPool._assign_requests_to_connections httpcore/_async/connection_pool.py:230
│ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.002 AsyncHTTPConnection.has_expired httpcore/_async/connection.py:185
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.002 AsyncHTTP11Connection.has_expired httpcore/_async/http11.py:281
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.002 AnyIOStream.get_extra_info httpcore/_backends/anyio.py:83
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.002 SocketStream.extra anyio/_core/_typedattr.py:60
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.002 SocketStream.extra_attributes anyio/abc/_sockets.py:58
│ │ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.001 [self] anyio/abc/_sockets.py
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 SocketStream._raw_socket anyio/_backends/_asyncio.py:1120
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 TCPTransport.get_extra_info <built-in>
│ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.001 <listcomp> httpcore/_async/connection_pool.py:254
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 AsyncHTTPConnection.is_idle httpcore/_async/connection.py:190
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection.is_idle httpcore/_async/http11.py:295
│ │ │ │ │ │ │ │ │ │ │ └─ 0.001 HTTP11ConnectionByteStream.aclose httpcore/_async/http11.py:351
│ │ │ │ │ │ │ │ │ │ ├─ 0.003 BoundAsyncStream.__aiter__ httpx/_client.py:148
│ │ │ │ │ │ │ │ │ │ │ └─ 0.003 AsyncResponseStream.__aiter__ httpx/_transports/default.py:252
│ │ │ │ │ │ │ │ │ │ │ └─ 0.003 PoolByteStream.__aiter__ httpcore/_async/connection_pool.py:361
│ │ │ │ │ │ │ │ │ │ │ ├─ 0.002 HTTP11ConnectionByteStream.__aiter__ httpcore/_async/http11.py:337
│ │ │ │ │ │ │ │ │ │ │ │ ├─ 0.001 AsyncHTTP11Connection._receive_response_body httpcore/_async/http11.py:205
│ │ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._receive_event httpcore/_async/http11.py:216
│ │ │ │ │ │ │ │ │ │ │ │ └─ 0.001 WeakSet.add _weakrefset.py:86
│ │ │ │ │ │ │ │ │ │ │ └─ 0.001 [self] httpcore/_async/connection_pool.py
│ │ │ │ │ │ │ │ │ │ └─ 0.001 [self] httpx/_models.py
│ │ │ │ │ │ │ │ │ └─ 0.006 [await] httpx/_models.py
│ │ │ │ │ │ │ │ └─ 0.008 AsyncClient.build_request httpx/_client.py:319
│ │ │ │ │ │ │ │ ├─ 0.004 AsyncClient._merge_url httpx/_client.py:370
│ │ │ │ │ │ │ │ │ └─ 0.004 URL.__init__ httpx/_urls.py:75
│ │ │ │ │ │ │ │ │ └─ 0.004 urlparse httpx/_urlparse.py:151
│ │ │ │ │ │ │ │ │ ├─ 0.002 quote httpx/_urlparse.py:449
│ │ │ │ │ │ │ │ │ │ └─ 0.002 finditer re.py:242
│ │ │ │ │ │ │ │ │ │ └─ 0.002 _compile re.py:288
│ │ │ │ │ │ │ │ │ │ ├─ 0.001 [self] re.py
│ │ │ │ │ │ │ │ │ │ └─ 0.001 isinstance <built-in>
│ │ │ │ │ │ │ │ │ └─ 0.002 <genexpr> httpx/_urlparse.py:161
│ │ │ │ │ │ │ │ ├─ 0.003 Request.__init__ httpx/_models.py:309
│ │ │ │ │ │ │ │ │ ├─ 0.002 URL.copy_merge_params httpx/_urls.py:365
│ │ │ │ │ │ │ │ │ │ └─ 0.002 URL.copy_with httpx/_urls.py:341
│ │ │ │ │ │ │ │ │ │ └─ 0.002 URL.__init__ httpx/_urls.py:75
│ │ │ │ │ │ │ │ │ │ └─ 0.002 ParseResult.copy_with httpx/_urlparse.py:124
│ │ │ │ │ │ │ │ │ │ └─ 0.002 urlparse httpx/_urlparse.py:151
│ │ │ │ │ │ │ │ │ │ ├─ 0.001 <genexpr> httpx/_urlparse.py:206
│ │ │ │ │ │ │ │ │ │ └─ 0.001 quote httpx/_urlparse.py:449
│ │ │ │ │ │ │ │ │ └─ 0.001 Request._prepare httpx/_models.py:373
│ │ │ │ │ │ │ │ │ └─ 0.001 Headers.__contains__ httpx/_models.py:272
│ │ │ │ │ │ │ │ └─ 0.001 [self] httpx/_client.py
│ │ │ │ │ │ │ └─ 0.001 [self] tenacity/_asyncio.py
│ │ │ │ │ │ └─ 0.001 quote urllib/parse.py:817
│ │ │ │ │ │ └─ 0.001 quote_from_bytes urllib/parse.py:888
│ │ │ │ │ │ └─ 0.001 str.join <built-in>
│ │ │ │ │ ├─ 0.025 ProjectNodesRepo.get simcore_postgres_database/utils_projects_nodes.py:135
│ │ │ │ │ │ ├─ 0.022 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ │ ├─ 0.020 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ │ │ │ └─ 0.020 Connection._poll aiopg/connection.py:874
│ │ │ │ │ │ │ │ └─ 0.020 wait_for asyncio/tasks.py:392
│ │ │ │ │ │ │ │ ├─ 0.019 [await] asyncio/tasks.py
│ │ │ │ │ │ │ │ └─ 0.001 Loop.call_later <built-in>
│ │ │ │ │ │ │ ├─ 0.001 SAConnection._open_cursor aiopg/sa/connection.py:95
│ │ │ │ │ │ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ │ │ └─ 0.001 <listcomp> sqlalchemy/sql/compiler.py:3485
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._label_select_column sqlalchemy/sql/compiler.py:3157
│ │ │ │ │ │ │ └─ 0.001 isinstance <built-in>
│ │ │ │ │ │ ├─ 0.001 Select.where <string>:1
│ │ │ │ │ │ │ └─ 0.001 _generative sqlalchemy/sql/base.py:105
│ │ │ │ │ │ ├─ 0.001 BinaryExpression.__and__ sqlalchemy/sql/operators.py:59
│ │ │ │ │ │ │ └─ 0.001 BinaryExpression.operate sqlalchemy/sql/elements.py:872
│ │ │ │ │ │ │ └─ 0.001 Comparator.__and__ sqlalchemy/sql/operators.py:59
│ │ │ │ │ │ │ └─ 0.001 Comparator.operate sqlalchemy/sql/type_api.py:73
│ │ │ │ │ │ │ └─ 0.001 _conjunction_operate sqlalchemy/sql/default_comparator.py:145
│ │ │ │ │ │ │ └─ 0.001 BooleanClauseList.and_ sqlalchemy/sql/elements.py:2579
│ │ │ │ │ │ │ └─ 0.001 BooleanClauseList._construct sqlalchemy/sql/elements.py:2504
│ │ │ │ │ │ │ └─ 0.001 BooleanClauseList._process_clauses_for_boolean sqlalchemy/sql/elements.py:2465
│ │ │ │ │ │ └─ 0.001 select <string>:1
│ │ │ │ │ │ └─ 0.001 Select._create sqlalchemy/sql/selectable.py:5256
│ │ │ │ │ │ └─ 0.001 Column.__getattr__ sqlalchemy/sql/elements.py:856
│ │ │ │ │ │ └─ 0.001 getattr <built-in>
│ │ │ │ │ └─ 0.001 [self] simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py
│ │ │ │ ├─ 0.004 [self] simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py
│ │ │ │ ├─ 0.002 <dictcomp> simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py:347
│ │ │ │ │ └─ 0.002 _get_node_infos simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py:130
│ │ │ │ │ ├─ 0.001 to_node_class simcore_service_director_v2/utils/computations.py:89
│ │ │ │ │ │ └─ 0.001 Pattern.match <built-in>
│ │ │ │ │ └─ 0.001 [await] simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py
│ │ │ │ └─ 0.001 _SpecialGenericAlias.__instancecheck__ typing.py:993
│ │ │ ├─ 0.064 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ ├─ 0.058 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ ├─ 0.055 Connection._poll aiopg/connection.py:874
│ │ │ │ │ │ └─ 0.055 wait_for asyncio/tasks.py:392
│ │ │ │ │ │ └─ 0.055 [await] asyncio/tasks.py
│ │ │ │ │ ├─ 0.002 [self] aiopg/connection.py
│ │ │ │ │ └─ 0.001 cursor.execute <built-in>
│ │ │ │ ├─ 0.005 Insert.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ └─ 0.005 Insert._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ └─ 0.005 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ └─ 0.005 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ └─ 0.005 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ └─ 0.005 Insert._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ ├─ 0.004 APGCompiler_psycopg2.visit_insert sqlalchemy/sql/compiler.py:3991
│ │ │ │ │ │ ├─ 0.003 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ │ └─ 0.003 OnConflictDoUpdate._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ └─ 0.003 APGCompiler_psycopg2.visit_on_conflict_do_update sqlalchemy/dialects/postgresql/base.py:2597
│ │ │ │ │ │ │ ├─ 0.001 BindParameter.__init__ sqlalchemy/sql/elements.py:1263
│ │ │ │ │ │ │ ├─ 0.001 [self] sqlalchemy/dialects/postgresql/base.py
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._truncate_bindparam sqlalchemy/sql/compiler.py:2682
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._truncated_identifier sqlalchemy/sql/compiler.py:2695
│ │ │ │ │ │ └─ 0.001 _get_crud_params sqlalchemy/sql/crud.py:38
│ │ │ │ │ │ └─ 0.001 _scan_cols sqlalchemy/sql/crud.py:363
│ │ │ │ │ │ └─ 0.001 _append_param_parameter sqlalchemy/sql/crud.py:483
│ │ │ │ │ │ └─ 0.001 _handle_values_anonymous_param sqlalchemy/sql/crud.py:220
│ │ │ │ │ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ │ │ │ │ └─ 0.001 DateTime.dialect_impl sqlalchemy/sql/type_api.py:649
│ │ │ │ │ │ └─ 0.001 WeakKeyDictionary.__getitem__ weakref.py:415
│ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ └─ 0.001 NoneType._compose_select_body sqlalchemy/sql/compiler.py:3650
│ │ │ │ │ └─ 0.001 dict._generate_delimited_and_list sqlalchemy/sql/compiler.py:1737
│ │ │ │ │ └─ 0.001 BinaryExpression._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_binary sqlalchemy/sql/compiler.py:2232
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._generate_generic_binary sqlalchemy/sql/compiler.py:2321
│ │ │ │ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.bindparam_string sqlalchemy/sql/compiler.py:2717
│ │ │ │ └─ 0.001 memoized_property.__get__ sqlalchemy/util/langhelpers.py:1110
│ │ │ │ └─ 0.001 APGCompiler_psycopg2._bind_processors sqlalchemy/sql/compiler.py:934
│ │ │ │ └─ 0.001 <genexpr> sqlalchemy/sql/compiler.py:937
│ │ │ │ └─ 0.001 <genexpr> sqlalchemy/sql/compiler.py:942
│ │ │ ├─ 0.005 [self] simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py
│ │ │ ├─ 0.005 CompTaskAtDB.to_db_model simcore_service_director_v2/models/comp_tasks.py:172
│ │ │ ├─ 0.003 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ │ └─ 0.003 _close_connection aiopg/sa/engine.py:115
│ │ │ │ └─ 0.003 SAConnection.close aiopg/sa/connection.py:384
│ │ │ │ └─ 0.003 [await] aiopg/sa/connection.py
│ │ │ ├─ 0.002 Insert.returning <string>:1
│ │ │ │ ├─ 0.001 [self] <string>
│ │ │ │ └─ 0.001 _generative sqlalchemy/sql/base.py:105
│ │ │ │ └─ 0.001 Insert.returning sqlalchemy/sql/dml.py:380
│ │ │ ├─ 0.001 SAConnection.execute aiopg/sa/connection.py:55
│ │ │ │ └─ 0.001 _IterableContextManager.__init__ aiopg/utils.py:107
│ │ │ └─ 0.001 ResultProxy.fetchone aiopg/sa/result.py:388
│ │ │ └─ 0.001 Cursor.fetchone aiopg/connection.py:505
│ │ ├─ 0.475 CompRunsRepository.get simcore_service_director_v2/modules/db/repositories/comp_runs.py:29
│ │ │ ├─ 0.460 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ └─ 0.460 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ └─ 0.460 Pool._acquire aiopg/pool.py:304
│ │ │ │ └─ 0.460 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ └─ 0.460 Lock.acquire asyncio/locks.py:93
│ │ │ │ └─ 0.460 [await] asyncio/locks.py
│ │ │ ├─ 0.011 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ ├─ 0.010 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ └─ 0.010 Connection._poll aiopg/connection.py:874
│ │ │ │ │ └─ 0.010 wait_for asyncio/tasks.py:392
│ │ │ │ │ └─ 0.010 [await] asyncio/tasks.py
│ │ │ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ └─ 0.001 NoneType._compose_select_body sqlalchemy/sql/compiler.py:3650
│ │ │ │ └─ 0.001 dict._generate_delimited_and_list sqlalchemy/sql/compiler.py:1737
│ │ │ │ └─ 0.001 BooleanClauseList._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_clauselist sqlalchemy/sql/compiler.py:1758
│ │ │ │ └─ 0.001 dict._generate_delimited_list sqlalchemy/sql/compiler.py:1730
│ │ │ │ └─ 0.001 <genexpr> sqlalchemy/sql/compiler.py:1731
│ │ │ │ └─ 0.001 <genexpr> sqlalchemy/sql/compiler.py:1733
│ │ │ │ └─ 0.001 BinaryExpression._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_binary sqlalchemy/sql/compiler.py:2232
│ │ │ │ └─ 0.001 APGCompiler_psycopg2._generate_generic_binary sqlalchemy/sql/compiler.py:2321
│ │ │ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ │ ├─ 0.002 [self] simcore_service_director_v2/modules/db/repositories/comp_runs.py
│ │ │ └─ 0.002 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ └─ 0.002 _close_connection aiopg/sa/engine.py:115
│ │ │ └─ 0.002 SAConnection.close aiopg/sa/connection.py:384
│ │ │ └─ 0.002 [await] aiopg/sa/connection.py
│ │ ├─ 0.306 _check_pipeline_not_running simcore_service_director_v2/api/routes/computations.py:101
│ │ │ └─ 0.306 CompTasksRepository.list_computational_tasks simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py:62
│ │ │ ├─ 0.280 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ └─ 0.280 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ └─ 0.280 Pool._acquire aiopg/pool.py:304
│ │ │ │ └─ 0.280 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ └─ 0.280 Lock.acquire asyncio/locks.py:93
│ │ │ │ └─ 0.280 [await] asyncio/locks.py
│ │ │ ├─ 0.017 _IterableContextManager.__anext__ aiopg/utils.py:113
│ │ │ │ └─ 0.017 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ ├─ 0.015 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ ├─ 0.013 Connection._poll aiopg/connection.py:874
│ │ │ │ │ │ └─ 0.013 wait_for asyncio/tasks.py:392
│ │ │ │ │ │ └─ 0.013 [await] asyncio/tasks.py
│ │ │ │ │ ├─ 0.001 cursor.execute <built-in>
│ │ │ │ │ └─ 0.001 Connection._create_waiter aiopg/connection.py:865
│ │ │ │ │ └─ 0.001 Loop.create_future <built-in>
│ │ │ │ ├─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ └─ 0.001 CompileState.create_for_statement sqlalchemy/sql/base.py:486
│ │ │ │ │ └─ 0.001 SelectState.create_for_statement sqlalchemy/sql/base.py:486
│ │ │ │ │ └─ 0.001 SelectState.__init__ sqlalchemy/sql/selectable.py:4382
│ │ │ │ │ └─ 0.001 SelectState._get_froms sqlalchemy/sql/selectable.py:4471
│ │ │ │ └─ 0.001 SAConnection._open_cursor aiopg/sa/connection.py:95
│ │ │ │ └─ 0.001 Connection._cursor aiopg/connection.py:931
│ │ │ │ └─ 0.001 Connection._cursor_impl aiopg/connection.py:952
│ │ │ ├─ 0.006 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ │ └─ 0.006 _close_connection aiopg/sa/engine.py:115
│ │ │ │ └─ 0.006 SAConnection.close aiopg/sa/connection.py:384
│ │ │ │ └─ 0.006 [await] aiopg/sa/connection.py
│ │ │ └─ 0.002 [self] simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py
│ │ ├─ 0.282 ProjectsRepository.get_project simcore_service_director_v2/modules/db/repositories/projects.py:17
│ │ │ ├─ 0.277 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ └─ 0.277 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ └─ 0.277 Pool._acquire aiopg/pool.py:304
│ │ │ │ └─ 0.277 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ └─ 0.277 Lock.acquire asyncio/locks.py:93
│ │ │ │ └─ 0.277 [await] asyncio/locks.py
│ │ │ ├─ 0.004 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ └─ 0.004 Cursor.execute aiopg/connection.py:400
│ │ │ │ └─ 0.004 Connection._poll aiopg/connection.py:874
│ │ │ │ ├─ 0.003 wait_for asyncio/tasks.py:392
│ │ │ │ │ └─ 0.003 [await] asyncio/tasks.py
│ │ │ │ └─ 0.001 _ready aiopg/connection.py:779
│ │ │ │ └─ 0.001 connection.poll <built-in>
│ │ │ └─ 0.001 [self] simcore_service_director_v2/modules/db/repositories/projects.py
│ │ ├─ 0.266 CompPipelinesRepository.upsert_pipeline simcore_service_director_v2/modules/db/repositories/comp_pipelines.py:31
│ │ │ ├─ 0.233 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ └─ 0.233 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ └─ 0.233 Pool._acquire aiopg/pool.py:304
│ │ │ │ └─ 0.233 Condition.__aenter__ asyncio/locks.py:13
│ │ │ │ └─ 0.233 Lock.acquire asyncio/locks.py:93
│ │ │ │ └─ 0.233 [await] asyncio/locks.py
│ │ │ ├─ 0.031 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ ├─ 0.030 Cursor.execute aiopg/connection.py:400
│ │ │ │ │ └─ 0.030 Connection._poll aiopg/connection.py:874
│ │ │ │ │ └─ 0.030 wait_for asyncio/tasks.py:392
│ │ │ │ │ └─ 0.030 [await] asyncio/tasks.py
│ │ │ │ └─ 0.001 Insert.compile sqlalchemy/sql/elements.py:439
│ │ │ │ └─ 0.001 Insert._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ └─ 0.001 Insert._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_insert sqlalchemy/sql/compiler.py:3991
│ │ │ │ └─ 0.001 _get_crud_params sqlalchemy/sql/crud.py:38
│ │ │ │ └─ 0.001 _scan_cols sqlalchemy/sql/crud.py:363
│ │ │ │ └─ 0.001 _append_param_parameter sqlalchemy/sql/crud.py:483
│ │ │ │ └─ 0.001 _handle_values_anonymous_param sqlalchemy/sql/crud.py:220
│ │ │ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ │ ├─ 0.001 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ │ └─ 0.001 _close_connection aiopg/sa/engine.py:115
│ │ │ │ └─ 0.001 SAConnection.close aiopg/sa/connection.py:384
│ │ │ │ └─ 0.001 [await] aiopg/sa/connection.py
│ │ │ └─ 0.001 Insert.on_conflict_do_update <string>:1
│ │ │ └─ 0.001 _generative sqlalchemy/sql/base.py:105
│ │ │ └─ 0.001 Insert.on_conflict_do_update <string>:1
│ │ │ └─ 0.001 check sqlalchemy/sql/base.py:129
│ │ │ └─ 0.001 Insert.on_conflict_do_update sqlalchemy/dialects/postgresql/dml.py:75
│ │ │ └─ 0.001 OnConflictDoUpdate.__init__ sqlalchemy/dialects/postgresql/dml.py:236
│ │ │ └─ 0.001 <listcomp> sqlalchemy/dialects/postgresql/dml.py:270
│ │ │ └─ 0.001 expect sqlalchemy/sql/coercions.py:112
│ │ │ └─ 0.001 DMLColumnImpl._implicit_coercions sqlalchemy/sql/coercions.py:311
│ │ ├─ 0.004 _check_pipeline_startable simcore_service_director_v2/api/routes/computations.py:114
│ │ │ └─ 0.004 find_deprecated_tasks simcore_service_director_v2/utils/computations.py:107
│ │ │ └─ 0.004 logged_gather servicelib/utils.py:94
│ │ │ └─ 0.004 [await] servicelib/utils.py
│ │ ├─ 0.001 create_minimal_computational_graph_based_on_selection simcore_service_director_v2/utils/dags.py:142
│ │ └─ 0.001 compute_pipeline_submitted_timestamp simcore_service_director_v2/utils/dags.py:220
│ └─ 0.003 [self] servicelib/async_utils.py
├─ 3.617 Pool._wakeup aiopg/pool.py:375
│ ├─ 3.585 Condition.__aenter__ asyncio/locks.py:13
│ │ └─ 3.585 Lock.acquire asyncio/locks.py:93
│ │ ├─ 3.584 [await] asyncio/locks.py
│ │ └─ 0.001 [self] asyncio/locks.py
│ └─ 0.032 Condition.__aexit__ asyncio/locks.py:19
│ └─ 0.032 [await] asyncio/locks.py
├─ 0.128 RequestResponseCycle.run_asgi uvicorn/protocols/http/httptools_impl.py:409
│ └─ 0.128 ProxyHeadersMiddleware.__call__ uvicorn/middleware/proxy_headers.py:41
│ └─ 0.128 FastAPI.__call__ fastapi/applications.py:287
│ └─ 0.128 FastAPI.__call__ starlette/applications.py:118
│ ├─ 0.127 ServerErrorMiddleware.__call__ starlette/middleware/errors.py:147
│ │ └─ 0.127 ProfilerMiddleware.__call__ servicelib/fastapi/profiler_middleware.py:38
│ │ ├─ 0.124 PrometheusInstrumentatorMiddleware.__call__ prometheus_fastapi_instrumentator/middleware.py:120
│ │ │ ├─ 0.123 ExceptionMiddleware.__call__ starlette/middleware/exceptions.py:53
│ │ │ │ └─ 0.123 AsyncExitStackMiddleware.__call__ fastapi/middleware/asyncexitstack.py:12
│ │ │ │ ├─ 0.122 APIRouter.__call__ starlette/routing.py:697
│ │ │ │ │ └─ 0.122 APIRoute.handle starlette/routing.py:265
│ │ │ │ │ └─ 0.122 app starlette/routing.py:63
│ │ │ │ │ └─ 0.122 app fastapi/routing.py:194
│ │ │ │ │ ├─ 0.098 solve_dependencies fastapi/dependencies/utils.py:546
│ │ │ │ │ │ ├─ 0.081 run_in_threadpool starlette/concurrency.py:35
│ │ │ │ │ │ │ └─ 0.081 run_sync anyio/to_thread.py:20
│ │ │ │ │ │ │ └─ 0.081 AsyncIOBackend.run_sync_in_worker_thread anyio/_backends/_asyncio.py:2087
│ │ │ │ │ │ │ ├─ 0.046 [await] anyio/_backends/_asyncio.py
│ │ │ │ │ │ │ ├─ 0.020 CapacityLimiter.__aenter__ anyio/_backends/_asyncio.py:1668
│ │ │ │ │ │ │ │ └─ 0.020 CapacityLimiter.acquire anyio/_backends/_asyncio.py:1722
│ │ │ │ │ │ │ │ └─ 0.020 CapacityLimiter.acquire_on_behalf_of anyio/_backends/_asyncio.py:1725
│ │ │ │ │ │ │ │ └─ 0.020 AsyncIOBackend.cancel_shielded_checkpoint anyio/_backends/_asyncio.py:2038
│ │ │ │ │ │ │ │ ├─ 0.019 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ │ └─ 0.019 [await] asyncio/tasks.py
│ │ │ │ │ │ │ │ └─ 0.001 [self] anyio/_backends/_asyncio.py
│ │ │ │ │ │ │ └─ 0.015 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ │ └─ 0.015 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ └─ 0.015 [await] asyncio/tasks.py
│ │ │ │ │ │ ├─ 0.014 solve_dependencies fastapi/dependencies/utils.py:546
│ │ │ │ │ │ │ ├─ 0.013 run_in_threadpool starlette/concurrency.py:35
│ │ │ │ │ │ │ │ └─ 0.013 run_sync anyio/to_thread.py:20
│ │ │ │ │ │ │ │ └─ 0.013 AsyncIOBackend.run_sync_in_worker_thread anyio/_backends/_asyncio.py:2087
│ │ │ │ │ │ │ │ ├─ 0.007 [await] anyio/_backends/_asyncio.py
│ │ │ │ │ │ │ │ ├─ 0.005 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ │ │ │ └─ 0.005 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ │ └─ 0.005 [await] asyncio/tasks.py
│ │ │ │ │ │ │ │ └─ 0.001 RunVar.get anyio/lowlevel.py:124
│ │ │ │ │ │ │ │ └─ 0.001 RunVar._current_vars anyio/lowlevel.py:107
│ │ │ │ │ │ │ │ └─ 0.001 current_token anyio/lowlevel.py:58
│ │ │ │ │ │ │ │ └─ 0.001 get_async_backend anyio/_core/_eventloop.py:153
│ │ │ │ │ │ │ │ └─ 0.001 current_async_library sniffio/_impl.py:25
│ │ │ │ │ │ │ └─ 0.001 [self] fastapi/dependencies/utils.py
│ │ │ │ │ │ ├─ 0.001 inner typing.py:306
│ │ │ │ │ │ │ └─ 0.001 _CallableGenericAlias.__hash__ typing.py:1037
│ │ │ │ │ │ └─ 0.001 is_gen_callable fastapi/dependencies/utils.py:529
│ │ │ │ │ │ └─ 0.001 isgeneratorfunction inspect.py:301
│ │ │ │ │ │ └─ 0.001 _has_code_flag inspect.py:290
│ │ │ │ │ │ └─ 0.001 _unwrap_partial functools.py:421
│ │ │ │ │ ├─ 0.021 run_endpoint_function fastapi/routing.py:159
│ │ │ │ │ │ ├─ 0.020 get_computation simcore_service_director_v2/api/routes/computations.py:430
│ │ │ │ │ │ │ ├─ 0.005 ProjectsRepository.get_project simcore_service_director_v2/modules/db/repositories/projects.py:17
│ │ │ │ │ │ │ │ ├─ 0.002 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ │ │ │ └─ 0.002 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ │ │ │ └─ 0.002 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ │ │ │ └─ 0.002 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ │ │ │ └─ 0.002 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ │ │ │ └─ 0.002 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ │ │ │ └─ 0.002 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ └─ 0.002 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ │ │ │ │ └─ 0.002 NoneType._compose_select_body sqlalchemy/sql/compiler.py:3650
│ │ │ │ │ │ │ │ │ ├─ 0.001 [self] sqlalchemy/sql/compiler.py
│ │ │ │ │ │ │ │ │ └─ 0.001 dict._generate_delimited_and_list sqlalchemy/sql/compiler.py:1737
│ │ │ │ │ │ │ │ │ └─ 0.001 BinaryExpression._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_binary sqlalchemy/sql/compiler.py:2232
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._generate_generic_binary sqlalchemy/sql/compiler.py:2321
│ │ │ │ │ │ │ │ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ │ │ │ │ │ │ ├─ 0.001 [self] simcore_service_director_v2/modules/db/repositories/projects.py
│ │ │ │ │ │ │ │ ├─ 0.001 Column.__eq__ sqlalchemy/sql/operators.py:380
│ │ │ │ │ │ │ │ │ └─ 0.001 Column.operate sqlalchemy/sql/elements.py:872
│ │ │ │ │ │ │ │ │ └─ 0.001 Comparator.__eq__ sqlalchemy/sql/operators.py:380
│ │ │ │ │ │ │ │ │ └─ 0.001 Comparator.operate sqlalchemy/sql/type_api.py:73
│ │ │ │ │ │ │ │ │ └─ 0.001 _boolean_compare sqlalchemy/sql/default_comparator.py:30
│ │ │ │ │ │ │ │ │ └─ 0.001 expect sqlalchemy/sql/coercions.py:112
│ │ │ │ │ │ │ │ │ └─ 0.001 BinaryElementImpl._literal_coercion sqlalchemy/sql/coercions.py:526
│ │ │ │ │ │ │ │ │ └─ 0.001 Column._bind_param sqlalchemy/sql/elements.py:4733
│ │ │ │ │ │ │ │ │ └─ 0.001 BindParameter.__init__ sqlalchemy/sql/elements.py:1263
│ │ │ │ │ │ │ │ │ └─ 0.001 _anonymous_label.safe_construct sqlalchemy/sql/elements.py:5395
│ │ │ │ │ │ │ │ │ └─ 0.001 sub re.py:202
│ │ │ │ │ │ │ │ │ └─ 0.001 _compile re.py:288
│ │ │ │ │ │ │ │ └─ 0.001 ProjectAtDB.convert_sql_alchemy_enum models_library/projects.py:110
│ │ │ │ │ │ │ ├─ 0.005 CompRunsRepository.get simcore_service_director_v2/modules/db/repositories/comp_runs.py:29
│ │ │ │ │ │ │ │ ├─ 0.002 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ │ │ │ ├─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ │ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ │ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ │ │ │ │ │ └─ 0.001 NoneType._compose_select_body sqlalchemy/sql/compiler.py:3650
│ │ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._row_limit_clause sqlalchemy/sql/compiler.py:1988
│ │ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.limit_clause sqlalchemy/dialects/postgresql/base.py:2451
│ │ │ │ │ │ │ │ │ └─ 0.001 ResultProxy.__init__ aiopg/sa/result.py:244
│ │ │ │ │ │ │ │ │ └─ 0.001 ResultProxy._init_metadata aiopg/sa/result.py:301
│ │ │ │ │ │ │ │ │ └─ 0.001 ResultMetaData.__init__ aiopg/sa/result.py:94
│ │ │ │ │ │ │ │ │ └─ 0.001 dict.setdefault <built-in>
│ │ │ │ │ │ │ │ ├─ 0.002 Select.limit <string>:1
│ │ │ │ │ │ │ │ │ └─ 0.002 _generative sqlalchemy/sql/base.py:105
│ │ │ │ │ │ │ │ │ └─ 0.002 Select.limit sqlalchemy/sql/selectable.py:3713
│ │ │ │ │ │ │ │ │ └─ 0.002 Select._offset_or_limit_clause sqlalchemy/sql/selectable.py:3643
│ │ │ │ │ │ │ │ │ └─ 0.002 expect sqlalchemy/sql/coercions.py:112
│ │ │ │ │ │ │ │ │ └─ 0.002 LimitOffsetImpl._literal_coercion sqlalchemy/sql/coercions.py:807
│ │ │ │ │ │ │ │ │ ├─ 0.001 [self] sqlalchemy/sql/coercions.py
│ │ │ │ │ │ │ │ │ └─ 0.001 _OffsetLimitParam.__init__ sqlalchemy/sql/elements.py:1263
│ │ │ │ │ │ │ │ │ └─ 0.001 _anonymous_label.safe_construct sqlalchemy/sql/elements.py:5395
│ │ │ │ │ │ │ │ └─ 0.001 desc <string>:1
│ │ │ │ │ │ │ │ └─ 0.001 UnaryExpression._create_desc sqlalchemy/sql/elements.py:3522
│ │ │ │ │ │ │ │ └─ 0.001 expect sqlalchemy/sql/coercions.py:112
│ │ │ │ │ │ │ ├─ 0.005 analyze_pipeline simcore_service_director_v2/api/routes/computations_tasks.py:49
│ │ │ │ │ │ │ │ ├─ 0.003 CompTasksRepository.list_tasks simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py:48
│ │ │ │ │ │ │ │ │ ├─ 0.002 _IterableContextManager.__anext__ aiopg/utils.py:113
│ │ │ │ │ │ │ │ │ │ └─ 0.002 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ │ │ │ │ └─ 0.002 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ │ │ │ │ └─ 0.002 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ │ │ │ │ └─ 0.002 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ │ │ │ │ └─ 0.002 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ │ │ │ │ └─ 0.002 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ │ │ │ │ └─ 0.002 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ │ └─ 0.002 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ │ │ │ │ │ └─ 0.002 <listcomp> sqlalchemy/sql/compiler.py:3485
│ │ │ │ │ │ │ │ │ │ └─ 0.002 APGCompiler_psycopg2._label_select_column sqlalchemy/sql/compiler.py:3157
│ │ │ │ │ │ │ │ │ │ ├─ 0.001 isinstance <built-in>
│ │ │ │ │ │ │ │ │ │ └─ 0.001 Column._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ └─ 0.001 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ │ │ │ │ │ └─ 0.001 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ │ │ │ │ │ └─ 0.001 Pool._acquire aiopg/pool.py:304
│ │ │ │ │ │ │ │ │ └─ 0.001 Condition.__aexit__ asyncio/locks.py:19
│ │ │ │ │ │ │ │ └─ 0.002 CompPipelinesRepository.get_pipeline simcore_service_director_v2/modules/db/repositories/comp_pipelines.py:19
│ │ │ │ │ │ │ │ ├─ 0.001 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ │ │ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ │ │ │ │ └─ 0.001 NoneType._compose_select_body sqlalchemy/sql/compiler.py:3650
│ │ │ │ │ │ │ │ │ └─ 0.001 dict._generate_delimited_and_list sqlalchemy/sql/compiler.py:1737
│ │ │ │ │ │ │ │ │ └─ 0.001 BinaryExpression._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_binary sqlalchemy/sql/compiler.py:2232
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._generate_generic_binary sqlalchemy/sql/compiler.py:2321
│ │ │ │ │ │ │ │ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ │ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.bindparam_string sqlalchemy/sql/compiler.py:2717
│ │ │ │ │ │ │ │ └─ 0.001 Column.__eq__ sqlalchemy/sql/operators.py:380
│ │ │ │ │ │ │ │ └─ 0.001 Column.operate sqlalchemy/sql/elements.py:872
│ │ │ │ │ │ │ │ └─ 0.001 Comparator.__eq__ sqlalchemy/sql/operators.py:380
│ │ │ │ │ │ │ │ └─ 0.001 Comparator.operate sqlalchemy/sql/type_api.py:73
│ │ │ │ │ │ │ │ └─ 0.001 _boolean_compare sqlalchemy/sql/default_comparator.py:30
│ │ │ │ │ │ │ │ └─ 0.001 isinstance <built-in>
│ │ │ │ │ │ │ ├─ 0.002 [self] simcore_service_director_v2/api/routes/computations.py
│ │ │ │ │ │ │ ├─ 0.002 Logger.debug logging/__init__.py:1455
│ │ │ │ │ │ │ │ └─ 0.002 Logger._log logging/__init__.py:1600
│ │ │ │ │ │ │ │ ├─ 0.001 Logger.findCaller logging/__init__.py:1549
│ │ │ │ │ │ │ │ │ └─ 0.001 normcase posixpath.py:52
│ │ │ │ │ │ │ │ └─ 0.001 Logger.handle logging/__init__.py:1626
│ │ │ │ │ │ │ │ └─ 0.001 Logger.callHandlers logging/__init__.py:1680
│ │ │ │ │ │ │ │ └─ 0.001 StreamHandler.handle logging/__init__.py:955
│ │ │ │ │ │ │ └─ 0.001 create_complete_dag_from_tasks simcore_service_director_v2/utils/dags.py:52
│ │ │ │ │ │ └─ 0.001 wrapper servicelib/async_utils.py:143
│ │ │ │ │ │ └─ 0.001 Queue.put asyncio/queues.py:111
│ │ │ │ │ │ └─ 0.001 Queue.put_nowait asyncio/queues.py:138
│ │ │ │ │ │ └─ 0.001 Queue._wakeup_next asyncio/queues.py:59
│ │ │ │ │ │ └─ 0.001 Future.set_result <built-in>
│ │ │ │ │ ├─ 0.002 serialize_response fastapi/routing.py:114
│ │ │ │ │ │ ├─ 0.001 [self] fastapi/routing.py
│ │ │ │ │ │ └─ 0.001 jsonable_encoder fastapi/encoders.py:29
│ │ │ │ │ │ └─ 0.001 jsonable_encoder fastapi/encoders.py:29
│ │ │ │ │ │ └─ 0.001 jsonable_encoder fastapi/encoders.py:29
│ │ │ │ │ │ └─ 0.001 is_dataclass dataclasses.py:1210
│ │ │ │ │ └─ 0.001 [self] fastapi/routing.py
│ │ │ │ └─ 0.001 AsyncExitStack.__aexit__ contextlib.py:667
│ │ │ └─ 0.001 PrometheusInstrumentatorMiddleware._get_handler prometheus_fastapi_instrumentator/middleware.py:212
│ │ │ └─ 0.001 get_route_name prometheus_fastapi_instrumentator/routing.py:69
│ │ │ └─ 0.001 _get_route_name prometheus_fastapi_instrumentator/routing.py:47
│ │ │ └─ 0.001 APIRoute.matches fastapi/routing.py:479
│ │ ├─ 0.002 [self] servicelib/fastapi/profiler_middleware.py
│ │ └─ 0.001 Request.__init__ starlette/requests.py:193
│ │ └─ 0.001 Request.__init__ starlette/requests.py:69
│ └─ 0.001 [self] starlette/applications.py
├─ 0.066 SAConnection._close aiopg/sa/connection.py:404
│ ├─ 0.064 [await] aiopg/sa/connection.py
│ └─ 0.002 Engine.release aiopg/sa/engine.py:209
│ ├─ 0.001 Pool.release aiopg/pool.py:379
│ └─ 0.001 [self] aiopg/sa/engine.py
├─ 0.047 Channel.rpc aiormq/channel.py:157
│ ├─ 0.025 CountdownContext.__aexit__ aiormq/tools.py:112
│ │ └─ 0.025 Countdown.__call__ aiormq/tools.py:85
│ │ ├─ 0.020 wait_for asyncio/tasks.py:392
│ │ │ └─ 0.020 [await] asyncio/tasks.py
│ │ └─ 0.005 [await] aiormq/tools.py
│ ├─ 0.021 Countdown.__call__ aiormq/tools.py:85
│ │ ├─ 0.019 wait_for asyncio/tasks.py:392
│ │ │ └─ 0.019 [await] asyncio/tasks.py
│ │ ├─ 0.001 [self] aiormq/tools.py
│ │ └─ 0.001 Queue.get asyncio/queues.py:150
│ │ └─ 0.001 Queue._get_loop asyncio/mixins.py:22
│ └─ 0.001 CountdownContext.__aenter__ aiormq/tools.py:109
│ └─ 0.001 Countdown.__call__ aiormq/tools.py:85
│ └─ 0.001 wait_for asyncio/tasks.py:392
│ └─ 0.001 ensure_future asyncio/tasks.py:610
│ └─ 0.001 _ensure_future asyncio/tasks.py:618
│ └─ 0.001 isfuture asyncio/base_futures.py:14
├─ 0.044 DaskClient.send_computation_tasks simcore_service_director_v2/modules/dask_client.py:282
│ ├─ 0.022 create_node_ports simcore_service_director_v2/utils/dask.py:114
│ │ └─ 0.022 ports simcore_sdk/node_ports_v2/__init__.py:19
│ │ └─ 0.022 load simcore_sdk/node_ports_v2/serialization_v2.py:32
│ │ ├─ 0.011 Nodeports.__init__ simcore_sdk/node_ports_v2/nodeports_v2.py:49
│ │ │ ├─ 0.007 Port.__init__ simcore_sdk/node_ports_v2/port.py:138
│ │ │ │ ├─ 0.004 [self] simcore_sdk/node_ports_v2/port.py
│ │ │ │ └─ 0.003 deepcopy copy.py:128
│ │ │ │ ├─ 0.002 [self] copy.py
│ │ │ │ └─ 0.001 id <built-in>
│ │ │ └─ 0.004 [self] simcore_sdk/node_ports_v2/nodeports_v2.py
│ │ ├─ 0.006 Logger.debug logging/__init__.py:1455
│ │ │ └─ 0.006 Logger._log logging/__init__.py:1600
│ │ │ ├─ 0.005 Logger.handle logging/__init__.py:1626
│ │ │ │ └─ 0.005 Logger.callHandlers logging/__init__.py:1680
│ │ │ │ └─ 0.005 StreamHandler.handle logging/__init__.py:955
│ │ │ │ ├─ 0.004 StreamHandler.emit logging/__init__.py:1088
│ │ │ │ │ ├─ 0.003 StreamHandler.format logging/__init__.py:932
│ │ │ │ │ │ └─ 0.003 CustomFormatter.format servicelib/logging_utils.py:63
│ │ │ │ │ │ └─ 0.003 CustomFormatter.format logging/__init__.py:665
│ │ │ │ │ │ ├─ 0.002 CustomFormatter.formatTime logging/__init__.py:597
│ │ │ │ │ │ │ ├─ 0.001 strftime <built-in>
│ │ │ │ │ │ │ └─ 0.001 [self] logging/__init__.py
│ │ │ │ │ │ └─ 0.001 CustomFormatter.usesTime logging/__init__.py:643
│ │ │ │ │ │ └─ 0.001 PercentStyle.usesTime logging/__init__.py:423
│ │ │ │ │ │ └─ 0.001 str.find <built-in>
│ │ │ │ │ └─ 0.001 TextIOWrapper.write <built-in>
│ │ │ │ └─ 0.001 StreamHandler.acquire logging/__init__.py:912
│ │ │ │ └─ 0.001 RLock.acquire <built-in>
│ │ │ └─ 0.001 Logger.makeRecord logging/__init__.py:1585
│ │ │ └─ 0.001 LogRecord.__init__ logging/__init__.py:283
│ │ │ └─ 0.001 basename posixpath.py:140
│ │ │ └─ 0.001 _get_sep posixpath.py:41
│ │ ├─ 0.003 DBManager.get_ports_configuration_from_node_uuid simcore_sdk/node_ports_common/dbmanager.py:137
│ │ │ ├─ 0.002 Logger.debug logging/__init__.py:1455
│ │ │ │ └─ 0.002 Logger._log logging/__init__.py:1600
│ │ │ │ └─ 0.002 Logger.handle logging/__init__.py:1626
│ │ │ │ └─ 0.002 Logger.callHandlers logging/__init__.py:1680
│ │ │ │ └─ 0.002 StreamHandler.handle logging/__init__.py:955
│ │ │ │ └─ 0.002 StreamHandler.emit logging/__init__.py:1088
│ │ │ │ └─ 0.002 StreamHandler.format logging/__init__.py:932
│ │ │ │ └─ 0.002 CustomFormatter.format servicelib/logging_utils.py:63
│ │ │ │ └─ 0.002 CustomFormatter.format logging/__init__.py:665
│ │ │ │ ├─ 0.001 CustomFormatter.formatTime logging/__init__.py:597
│ │ │ │ │ └─ 0.001 localtime <built-in>
│ │ │ │ └─ 0.001 [self] logging/__init__.py
│ │ │ └─ 0.001 _get_node_from_db simcore_sdk/node_ports_common/dbmanager.py:30
│ │ │ └─ 0.001 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ └─ 0.001 <listcomp> sqlalchemy/sql/compiler.py:3485
│ │ ├─ 0.001 loads json/__init__.py:299
│ │ │ └─ 0.001 JSONDecoder.decode json/decoder.py:332
│ │ │ └─ 0.001 JSONDecoder.raw_decode json/decoder.py:343
│ │ └─ 0.001 [self] simcore_sdk/node_ports_v2/serialization_v2.py
│ ├─ 0.015 compute_input_data simcore_service_director_v2/utils/dask.py:195
│ │ ├─ 0.012 Nodeports.inputs simcore_sdk/node_ports_v2/nodeports_v2.py:59
│ │ │ └─ 0.012 Nodeports._auto_update_from_db simcore_sdk/node_ports_v2/nodeports_v2.py:124
│ │ │ └─ 0.012 Nodeports._node_ports_creator_cb simcore_sdk/node_ports_v2/nodeports_v2.py:119
│ │ │ └─ 0.012 load simcore_sdk/node_ports_v2/serialization_v2.py:32
│ │ │ ├─ 0.006 DBManager.get_ports_configuration_from_node_uuid simcore_sdk/node_ports_common/dbmanager.py:137
│ │ │ │ ├─ 0.004 _get_node_from_db simcore_sdk/node_ports_common/dbmanager.py:30
│ │ │ │ │ ├─ 0.002 SAConnection._execute aiopg/sa/connection.py:106
│ │ │ │ │ │ ├─ 0.001 APGCompiler_psycopg2.construct_params aiopg/sa/engine.py:20
│ │ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.construct_params sqlalchemy/sql/compiler.py:964
│ │ │ │ │ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ │ │ │ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ │ │ │ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ │ │ │ │ └─ 0.001 <listcomp> sqlalchemy/sql/compiler.py:3485
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2._label_select_column sqlalchemy/sql/compiler.py:3157
│ │ │ │ │ │ └─ 0.001 Column._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ │ │ │ │ └─ 0.001 APGCompiler_psycopg2.visit_column sqlalchemy/sql/compiler.py:1559
│ │ │ │ │ │ └─ 0.001 PGIdentifierPreparer_psycopg2.quote sqlalchemy/sql/compiler.py:5410
│ │ │ │ │ ├─ 0.001 and_ <string>:1
│ │ │ │ │ │ └─ 0.001 BooleanClauseList.and_ sqlalchemy/sql/elements.py:2579
│ │ │ │ │ │ └─ 0.001 BooleanClauseList._construct sqlalchemy/sql/elements.py:2504
│ │ │ │ │ │ └─ 0.001 BooleanClauseList._process_clauses_for_boolean sqlalchemy/sql/elements.py:2465
│ │ │ │ │ │ └─ 0.001 <listcomp> sqlalchemy/sql/elements.py:2502
│ │ │ │ │ │ └─ 0.001 BinaryExpression.self_group sqlalchemy/sql/elements.py:3946
│ │ │ │ │ └─ 0.001 Column.__eq__ sqlalchemy/sql/operators.py:380
│ │ │ │ │ └─ 0.001 Column.operate sqlalchemy/sql/elements.py:872
│ │ │ │ │ └─ 0.001 Comparator.__eq__ sqlalchemy/sql/operators.py:380
│ │ │ │ │ └─ 0.001 Comparator.operate sqlalchemy/sql/type_api.py:73
│ │ │ │ │ └─ 0.001 _boolean_compare sqlalchemy/sql/default_comparator.py:30
│ │ │ │ │ └─ 0.001 expect sqlalchemy/sql/coercions.py:112
│ │ │ │ │ └─ 0.001 BinaryElementImpl._literal_coercion sqlalchemy/sql/coercions.py:526
│ │ │ │ │ └─ 0.001 Column._bind_param sqlalchemy/sql/elements.py:4733
│ │ │ │ │ └─ 0.001 BindParameter.__init__ sqlalchemy/sql/elements.py:1263
│ │ │ │ ├─ 0.001 Logger.debug logging/__init__.py:1455
│ │ │ │ │ └─ 0.001 Logger._log logging/__init__.py:1600
│ │ │ │ │ └─ 0.001 Logger.handle logging/__init__.py:1626
│ │ │ │ │ └─ 0.001 Logger.callHandlers logging/__init__.py:1680
│ │ │ │ │ └─ 0.001 StreamHandler.handle logging/__init__.py:955
│ │ │ │ │ └─ 0.001 StreamHandler.emit logging/__init__.py:1088
│ │ │ │ │ └─ 0.001 TextIOWrapper.write <built-in>
│ │ │ │ └─ 0.001 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ │ └─ 0.001 Engine._acquire aiopg/sa/engine.py:205
│ │ │ │ └─ 0.001 Pool._acquire aiopg/pool.py:304
│ │ │ ├─ 0.004 Logger.debug logging/__init__.py:1455
│ │ │ │ └─ 0.004 Logger._log logging/__init__.py:1600
│ │ │ │ ├─ 0.003 Logger.handle logging/__init__.py:1626
│ │ │ │ │ └─ 0.003 Logger.callHandlers logging/__init__.py:1680
│ │ │ │ │ ├─ 0.002 StreamHandler.handle logging/__init__.py:955
│ │ │ │ │ │ └─ 0.002 StreamHandler.emit logging/__init__.py:1088
│ │ │ │ │ │ └─ 0.002 StreamHandler.format logging/__init__.py:932
│ │ │ │ │ │ ├─ 0.001 CustomFormatter.format servicelib/logging_utils.py:63
│ │ │ │ │ │ │ └─ 0.001 CustomFormatter.format logging/__init__.py:665
│ │ │ │ │ │ │ └─ 0.001 CustomFormatter.formatMessage logging/__init__.py:649
│ │ │ │ │ │ │ └─ 0.001 PercentStyle.format logging/__init__.py:438
│ │ │ │ │ │ │ └─ 0.001 PercentStyle._format logging/__init__.py:431
│ │ │ │ │ │ └─ 0.001 [self] logging/__init__.py
│ │ │ │ │ └─ 0.001 [self] logging/__init__.py
│ │ │ │ └─ 0.001 Logger.findCaller logging/__init__.py:1549
│ │ │ ├─ 0.001 pformat pprint.py:57
│ │ │ │ └─ 0.001 PrettyPrinter.pformat pprint.py:155
│ │ │ │ └─ 0.001 PrettyPrinter._format pprint.py:167
│ │ │ │ └─ 0.001 PrettyPrinter._repr pprint.py:453
│ │ │ │ └─ 0.001 PrettyPrinter.format pprint.py:462
│ │ │ │ └─ 0.001 PrettyPrinter._safe_repr pprint.py:550
│ │ │ │ └─ 0.001 repr <built-in>
│ │ │ └─ 0.001 Nodeports.__init__ simcore_sdk/node_ports_v2/nodeports_v2.py:49
│ │ │ └─ 0.001 Port.__init__ simcore_sdk/node_ports_v2/port.py:138
│ │ │ └─ 0.001 match re.py:187
│ │ │ └─ 0.001 Pattern.match <built-in>
│ │ └─ 0.003 LinkType.get_value simcore_sdk/node_ports_v2/port.py:161
│ │ └─ 0.003 _evaluate simcore_sdk/node_ports_v2/port.py:181
│ │ └─ 0.003 get_value_link_from_port_link simcore_sdk/node_ports_v2/port_utils.py:26
│ │ └─ 0.003 Nodeports._node_ports_creator_cb simcore_sdk/node_ports_v2/nodeports_v2.py:119
│ │ └─ 0.003 load simcore_sdk/node_ports_v2/serialization_v2.py:32
│ │ └─ 0.003 DBManager.get_ports_configuration_from_node_uuid simcore_sdk/node_ports_common/dbmanager.py:137
│ │ ├─ 0.002 _ContextManager.__aenter__ aiopg/utils.py:81
│ │ │ └─ 0.002 Engine._acquire aiopg/sa/engine.py:205
│ │ │ └─ 0.002 Pool._acquire aiopg/pool.py:304
│ │ │ ├─ 0.001 Timeout.__aexit__ async_timeout/__init__.py:135
│ │ │ └─ 0.001 timeout async_timeout/__init__.py:32
│ │ │ └─ 0.001 Timeout.__init__ async_timeout/__init__.py:100
│ │ └─ 0.001 _get_node_from_db simcore_sdk/node_ports_common/dbmanager.py:30
│ │ └─ 0.001 SAConnection._execute aiopg/sa/connection.py:106
│ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ └─ 0.001 <listcomp> sqlalchemy/sql/compiler.py:3485
│ ├─ 0.005 compute_output_data_schema simcore_service_director_v2/utils/dask.py:238
│ │ ├─ 0.003 get_upload_links_from_storage simcore_sdk/node_ports_v2/port_utils.py:128
│ │ │ ├─ 0.002 get_upload_links_from_s3 simcore_sdk/node_ports_common/filemanager.py:90
│ │ │ │ └─ 0.002 wrapped simcore_sdk/node_ports_common/storage_client.py:47
│ │ │ │ └─ 0.002 get_upload_file_links simcore_sdk/node_ports_common/storage_client.py:179
│ │ │ │ └─ 0.002 _AsyncGeneratorContextManager.__aenter__ contextlib.py:194
│ │ │ │ └─ 0.002 retry_request simcore_sdk/node_ports_common/storage_client.py:95
│ │ │ │ └─ 0.002 _RequestContextManager.__aenter__ aiohttp/client.py:1196
│ │ │ │ └─ 0.002 ClientSession._request aiohttp/client.py:386
│ │ │ │ ├─ 0.001 TCPConnector.connect aiohttp/connector.py:494
│ │ │ │ │ └─ 0.001 TCPConnector._create_connection aiohttp/connector.py:934
│ │ │ │ │ └─ 0.001 TCPConnector._create_direct_connection aiohttp/connector.py:1179
│ │ │ │ │ └─ 0.001 TCPConnector._wrap_create_connection aiohttp/connector.py:1013
│ │ │ │ └─ 0.001 Timeout.__aenter__ async_timeout/__init__.py:131
│ │ │ │ └─ 0.001 Timeout._do_enter async_timeout/__init__.py:216
│ │ │ └─ 0.001 PosixPath.__new__ pathlib.py:957
│ │ │ └─ 0.001 PosixPath._from_parts pathlib.py:589
│ │ │ └─ 0.001 PosixPath._parse_args pathlib.py:569
│ │ │ └─ 0.001 _PosixFlavour.parse_parts pathlib.py:56
│ │ └─ 0.002 Nodeports.outputs simcore_sdk/node_ports_v2/nodeports_v2.py:66
│ │ └─ 0.002 Nodeports._auto_update_from_db simcore_sdk/node_ports_v2/nodeports_v2.py:124
│ │ └─ 0.002 Nodeports._node_ports_creator_cb simcore_sdk/node_ports_v2/nodeports_v2.py:119
│ │ └─ 0.002 load simcore_sdk/node_ports_v2/serialization_v2.py:32
│ │ ├─ 0.001 DBManager.get_ports_configuration_from_node_uuid simcore_sdk/node_ports_common/dbmanager.py:137
│ │ │ └─ 0.001 _ContextManager.__aexit__ aiopg/utils.py:86
│ │ │ └─ 0.001 _close_connection aiopg/sa/engine.py:115
│ │ └─ 0.001 Nodeports.__init__ simcore_sdk/node_ports_v2/nodeports_v2.py:49
│ │ └─ 0.001 Port.__init__ simcore_sdk/node_ports_v2/port.py:138
│ │ └─ 0.001 match re.py:187
│ │ └─ 0.001 _compile re.py:288
│ └─ 0.001 log_decorator_wrapper servicelib/logging_utils.py:179
│ └─ 0.001 StorageClient.get_s3_access simcore_service_director_v2/modules/storage.py:68
│ └─ 0.001 wrapper_func simcore_service_director_v2/utils/client_decorators.py:55
│ └─ 0.001 async_wrapped tenacity/_asyncio.py:138
│ └─ 0.001 AsyncRetrying.__call__ tenacity/_asyncio.py:51
│ └─ 0.001 StorageClient.request simcore_service_director_v2/modules/storage.py:63
│ └─ 0.001 AsyncClient.request httpx/_client.py:1519
│ └─ 0.001 AsyncClient.send httpx/_client.py:1628
│ └─ 0.001 AsyncClient._send_handling_auth httpx/_client.py:1677
│ └─ 0.001 AsyncClient._send_handling_redirects httpx/_client.py:1711
│ └─ 0.001 AsyncClient._send_single_request httpx/_client.py:1749
│ └─ 0.001 AsyncHTTPTransport.handle_async_request httpx/_transports/default.py:354
│ └─ 0.001 AsyncConnectionPool.handle_async_request httpcore/_async/connection_pool.py:159
│ └─ 0.001 AsyncHTTPConnection.handle_async_request httpcore/_async/connection.py:67
│ └─ 0.001 AsyncHTTPConnection._connect httpcore/_async/connection.py:103
│ └─ 0.001 AutoBackend.connect_tcp httpcore/_backends/auto.py:21
│ └─ 0.001 AnyIOBackend.connect_tcp httpcore/_backends/anyio.py:99
│ └─ 0.001 connect_tcp anyio/_core/_sockets.py:119
│ └─ 0.001 getaddrinfo anyio/_core/_sockets.py:533
├─ 0.043 log_decorator_wrapper servicelib/logging_utils.py:179
│ ├─ 0.020 DirectorV0Client.get_service_extras simcore_service_director_v2/modules/director_v0.py:76
│ │ └─ 0.020 wrapper_func simcore_service_director_v2/utils/client_decorators.py:55
│ │ └─ 0.020 async_wrapped tenacity/_asyncio.py:138
│ │ └─ 0.020 AsyncRetrying.__call__ tenacity/_asyncio.py:51
│ │ └─ 0.020 DirectorV0Client._request simcore_service_director_v2/modules/director_v0.py:71
│ │ └─ 0.020 AsyncClient.request httpx/_client.py:1519
│ │ ├─ 0.019 AsyncClient.send httpx/_client.py:1628
│ │ │ ├─ 0.015 AsyncClient._send_handling_auth httpx/_client.py:1677
│ │ │ │ └─ 0.015 AsyncClient._send_handling_redirects httpx/_client.py:1711
│ │ │ │ └─ 0.015 AsyncClient._send_single_request httpx/_client.py:1749
│ │ │ │ └─ 0.015 AsyncHTTPTransport.handle_async_request httpx/_transports/default.py:354
│ │ │ │ └─ 0.015 AsyncConnectionPool.handle_async_request httpcore/_async/connection_pool.py:159
│ │ │ │ ├─ 0.014 AsyncHTTPConnection.handle_async_request httpcore/_async/connection.py:67
│ │ │ │ │ └─ 0.014 AsyncHTTP11Connection.handle_async_request httpcore/_async/http11.py:72
│ │ │ │ │ ├─ 0.008 AsyncHTTP11Connection._receive_response_headers httpcore/_async/http11.py:179
│ │ │ │ │ │ └─ 0.008 AsyncHTTP11Connection._receive_event httpcore/_async/http11.py:216
│ │ │ │ │ │ ├─ 0.007 AnyIOStream.read httpcore/_backends/anyio.py:23
│ │ │ │ │ │ │ └─ 0.007 SocketStream.receive anyio/_backends/_asyncio.py:1124
│ │ │ │ │ │ │ ├─ 0.005 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ │ │ └─ 0.005 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ └─ 0.005 [await] asyncio/tasks.py
│ │ │ │ │ │ │ └─ 0.002 Event.wait asyncio/locks.py:201
│ │ │ │ │ │ │ └─ 0.002 [await] asyncio/locks.py
│ │ │ │ │ │ └─ 0.001 Connection.next_event h11/_connection.py:426
│ │ │ │ │ │ └─ 0.001 Connection._process_event h11/_connection.py:260
│ │ │ │ │ │ └─ 0.001 _keep_alive h11/_connection.py:72
│ │ │ │ │ ├─ 0.004 AsyncLock.__aenter__ httpcore/_synchronization.py:69
│ │ │ │ │ │ └─ 0.004 Lock.acquire anyio/_core/_synchronization.py:156
│ │ │ │ │ │ └─ 0.004 cancel_shielded_checkpoint anyio/lowlevel.py:42
│ │ │ │ │ │ └─ 0.004 AsyncIOBackend.cancel_shielded_checkpoint anyio/_backends/_asyncio.py:2038
│ │ │ │ │ │ └─ 0.004 sleep asyncio/tasks.py:593
│ │ │ │ │ │ └─ 0.004 [await] asyncio/tasks.py
│ │ │ │ │ └─ 0.002 AsyncHTTP11Connection._send_request_headers httpcore/_async/http11.py:147
│ │ │ │ │ └─ 0.002 Request.__init__ h11/_events.py:82
│ │ │ │ │ ├─ 0.001 normalize_and_validate h11/_headers.py:150
│ │ │ │ │ └─ 0.001 Headers.__iter__ _collections_abc.py:1039
│ │ │ │ └─ 0.001 [self] httpcore/_async/connection_pool.py
│ │ │ ├─ 0.003 Response.aread httpx/_models.py:906
│ │ │ │ └─ 0.003 <listcomp> httpx/_models.py:911
│ │ │ │ ├─ 0.002 [await] httpx/_models.py
│ │ │ │ └─ 0.001 Response.aiter_bytes httpx/_models.py:914
│ │ │ │ └─ 0.001 Response.aiter_raw httpx/_models.py:969
│ │ │ │ └─ 0.001 Response.aclose httpx/_models.py:997
│ │ │ │ └─ 0.001 BoundAsyncStream.aclose httpx/_client.py:152
│ │ │ │ └─ 0.001 AsyncResponseStream.aclose httpx/_transports/default.py:257
│ │ │ │ └─ 0.001 PoolByteStream.aclose httpcore/_async/connection_pool.py:369
│ │ │ │ └─ 0.001 HTTP11ConnectionByteStream.aclose httpcore/_async/http11.py:351
│ │ │ │ └─ 0.001 AsyncHTTP11Connection._response_closed httpcore/_async/http11.py:245
│ │ │ │ └─ 0.001 AsyncLock.__aenter__ httpcore/_synchronization.py:69
│ │ │ │ └─ 0.001 Lock.acquire anyio/_core/_synchronization.py:156
│ │ │ │ └─ 0.001 checkpoint_if_cancelled anyio/lowlevel.py:30
│ │ │ └─ 0.001 [self] httpx/_client.py
│ │ └─ 0.001 AsyncClient.build_request httpx/_client.py:319
│ │ └─ 0.001 AsyncClient._merge_url httpx/_client.py:370
│ │ └─ 0.001 URL.copy_with httpx/_urls.py:341
│ │ └─ 0.001 URL.__init__ httpx/_urls.py:75
│ │ └─ 0.001 ParseResult.copy_with httpx/_urlparse.py:124
│ │ └─ 0.001 urlparse httpx/_urlparse.py:151
│ │ └─ 0.001 quote httpx/_urlparse.py:449
│ ├─ 0.013 DirectorV0Client.get_service_labels simcore_service_director_v2/modules/director_v0.py:101
│ │ └─ 0.013 wrapper_func simcore_service_director_v2/utils/client_decorators.py:55
│ │ └─ 0.013 async_wrapped tenacity/_asyncio.py:138
│ │ └─ 0.013 AsyncRetrying.__call__ tenacity/_asyncio.py:51
│ │ └─ 0.013 DirectorV0Client._request simcore_service_director_v2/modules/director_v0.py:71
│ │ └─ 0.013 AsyncClient.request httpx/_client.py:1519
│ │ ├─ 0.012 AsyncClient.send httpx/_client.py:1628
│ │ │ ├─ 0.007 AsyncClient._send_handling_auth httpx/_client.py:1677
│ │ │ │ └─ 0.007 AsyncClient._send_handling_redirects httpx/_client.py:1711
│ │ │ │ └─ 0.007 AsyncClient._send_single_request httpx/_client.py:1749
│ │ │ │ ├─ 0.006 AsyncHTTPTransport.handle_async_request httpx/_transports/default.py:354
│ │ │ │ │ └─ 0.006 AsyncConnectionPool.handle_async_request httpcore/_async/connection_pool.py:159
│ │ │ │ │ ├─ 0.004 AsyncHTTPConnection.handle_async_request httpcore/_async/connection.py:67
│ │ │ │ │ │ ├─ 0.002 AsyncHTTP11Connection.handle_async_request httpcore/_async/http11.py:72
│ │ │ │ │ │ │ ├─ 0.001 AsyncHTTP11Connection._receive_response_headers httpcore/_async/http11.py:179
│ │ │ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._receive_event httpcore/_async/http11.py:216
│ │ │ │ │ │ │ │ └─ 0.001 AnyIOStream.read httpcore/_backends/anyio.py:23
│ │ │ │ │ │ │ │ └─ 0.001 _GeneratorContextManager.__enter__ contextlib.py:130
│ │ │ │ │ │ │ │ └─ 0.001 fail_after anyio/_core/_tasks.py:95
│ │ │ │ │ │ │ │ └─ 0.001 AsyncIOBackend.create_cancel_scope anyio/_backends/_asyncio.py:2047
│ │ │ │ │ │ │ │ └─ 0.001 CancelScope.__init__ anyio/_backends/_asyncio.py:351
│ │ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._send_request_headers httpcore/_async/http11.py:147
│ │ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._send_event httpcore/_async/http11.py:170
│ │ │ │ │ │ │ └─ 0.001 AnyIOStream.write httpcore/_backends/anyio.py:39
│ │ │ │ │ │ │ └─ 0.001 SocketStream.send anyio/_backends/_asyncio.py:1158
│ │ │ │ │ │ │ └─ 0.001 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ │ └─ 0.001 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ └─ 0.001 [await] asyncio/tasks.py
│ │ │ │ │ │ ├─ 0.001 AsyncLock.__aenter__ httpcore/_synchronization.py:69
│ │ │ │ │ │ │ └─ 0.001 Lock.acquire anyio/_core/_synchronization.py:156
│ │ │ │ │ │ │ └─ 0.001 Event.wait anyio/_backends/_asyncio.py:1647
│ │ │ │ │ │ │ └─ 0.001 Event.wait asyncio/locks.py:201
│ │ │ │ │ │ │ └─ 0.001 [await] asyncio/locks.py
│ │ │ │ │ │ └─ 0.001 AsyncLock.__aexit__ httpcore/_synchronization.py:80
│ │ │ │ │ │ └─ 0.001 Lock.release anyio/_core/_synchronization.py:200
│ │ │ │ │ │ └─ 0.001 get_current_task anyio/_core/_testing.py:52
│ │ │ │ │ │ └─ 0.001 get_async_backend anyio/_core/_eventloop.py:153
│ │ │ │ │ │ └─ 0.001 current_async_library sniffio/_impl.py:25
│ │ │ │ │ └─ 0.002 AsyncConnectionPool._assign_requests_to_connections httpcore/_async/connection_pool.py:230
│ │ │ │ │ ├─ 0.001 AsyncPoolRequest.assign_to_connection httpcore/_async/connection_pool.py:21
│ │ │ │ │ │ └─ 0.001 AsyncEvent.set httpcore/_synchronization.py:127
│ │ │ │ │ │ └─ 0.001 AsyncEvent.setup httpcore/_synchronization.py:116
│ │ │ │ │ └─ 0.001 <listcomp> httpcore/_async/connection_pool.py:265
│ │ │ │ │ └─ 0.001 AsyncHTTPConnection.is_available httpcore/_async/connection.py:173
│ │ │ │ └─ 0.001 Cookies.extract_cookies httpx/_models.py:1033
│ │ │ │ └─ 0.001 _CookieCompatRequest.__init__ httpx/_models.py:1181
│ │ │ │ └─ 0.001 _CookieCompatRequest.__init__ urllib/request.py:319
│ │ │ │ └─ 0.001 request_host urllib/request.py:301
│ │ │ │ └─ 0.001 urlparse urllib/parse.py:371
│ │ │ │ └─ 0.001 urlsplit urllib/parse.py:436
│ │ │ │ └─ 0.001 _splitnetloc urllib/parse.py:410
│ │ │ └─ 0.005 Response.aread httpx/_models.py:906
│ │ │ └─ 0.005 <listcomp> httpx/_models.py:911
│ │ │ ├─ 0.004 [await] httpx/_models.py
│ │ │ └─ 0.001 Response.aiter_bytes httpx/_models.py:914
│ │ │ └─ 0.001 Response.aiter_raw httpx/_models.py:969
│ │ │ └─ 0.001 Response.aclose httpx/_models.py:997
│ │ │ └─ 0.001 BoundAsyncStream.aclose httpx/_client.py:152
│ │ │ └─ 0.001 AsyncResponseStream.aclose httpx/_transports/default.py:257
│ │ │ └─ 0.001 PoolByteStream.aclose httpcore/_async/connection_pool.py:369
│ │ │ └─ 0.001 AsyncConnectionPool._assign_requests_to_connections httpcore/_async/connection_pool.py:230
│ │ │ └─ 0.001 AsyncHTTPConnection.has_expired httpcore/_async/connection.py:185
│ │ │ └─ 0.001 AsyncHTTP11Connection.has_expired httpcore/_async/http11.py:281
│ │ │ └─ 0.001 AnyIOStream.get_extra_info httpcore/_backends/anyio.py:83
│ │ │ └─ 0.001 SocketStream.extra anyio/_core/_typedattr.py:60
│ │ │ └─ 0.001 SocketStream.extra_attributes anyio/abc/_sockets.py:58
│ │ └─ 0.001 AsyncClient.build_request httpx/_client.py:319
│ │ └─ 0.001 AsyncClient._merge_url httpx/_client.py:370
│ │ └─ 0.001 URL.__init__ httpx/_urls.py:75
│ │ └─ 0.001 urlparse httpx/_urlparse.py:151
│ │ └─ 0.001 quote httpx/_urlparse.py:449
│ ├─ 0.005 _log_arguments servicelib/logging_utils.py:137
│ │ ├─ 0.003 stack inspect.py:1671
│ │ │ └─ 0.003 getouterframes inspect.py:1643
│ │ │ └─ 0.003 getframeinfo inspect.py:1604
│ │ │ └─ 0.003 findsource inspect.py:932
│ │ │ ├─ 0.001 getmodule inspect.py:850
│ │ │ │ └─ 0.001 dict.get <built-in>
│ │ │ ├─ 0.001 istraceback inspect.py:355
│ │ │ └─ 0.001 getsourcefile inspect.py:813
│ │ │ └─ 0.001 exists genericpath.py:16
│ │ ├─ 0.001 getframeinfo inspect.py:1604
│ │ │ └─ 0.001 findsource inspect.py:932
│ │ │ └─ 0.001 getsourcefile inspect.py:813
│ │ └─ 0.001 Logger.log logging/__init__.py:1532
│ │ └─ 0.001 Logger._log logging/__init__.py:1600
│ │ └─ 0.001 Logger.handle logging/__init__.py:1626
│ │ └─ 0.001 Logger.callHandlers logging/__init__.py:1680
│ │ └─ 0.001 StreamHandler.handle logging/__init__.py:955
│ │ └─ 0.001 StreamHandler.emit logging/__init__.py:1088
│ │ └─ 0.001 StreamHandler.format logging/__init__.py:932
│ │ └─ 0.001 CustomFormatter.format servicelib/logging_utils.py:63
│ │ └─ 0.001 CustomFormatter.format logging/__init__.py:665
│ │ └─ 0.001 LogRecord.getMessage logging/__init__.py:359
│ ├─ 0.003 [await] servicelib/logging_utils.py
│ ├─ 0.001 Logger.log logging/__init__.py:1532
│ │ └─ 0.001 Logger._log logging/__init__.py:1600
│ │ └─ 0.001 Logger.handle logging/__init__.py:1626
│ │ └─ 0.001 Logger.callHandlers logging/__init__.py:1680
│ │ └─ 0.001 StreamHandler.handle logging/__init__.py:955
│ │ └─ 0.001 StreamHandler.emit logging/__init__.py:1088
│ │ └─ 0.001 StreamHandler.format logging/__init__.py:932
│ │ └─ 0.001 CustomFormatter.format servicelib/logging_utils.py:63
│ │ └─ 0.001 CustomFormatter.format logging/__init__.py:665
│ │ └─ 0.001 LogRecord.getMessage logging/__init__.py:359
│ └─ 0.001 [self] servicelib/logging_utils.py
├─ 0.042 CatalogClient.get_service simcore_service_director_v2/modules/catalog.py:64
│ └─ 0.042 wrapper_func simcore_service_director_v2/utils/client_decorators.py:55
│ └─ 0.042 async_wrapped tenacity/_asyncio.py:138
│ └─ 0.042 AsyncRetrying.__call__ tenacity/_asyncio.py:51
│ ├─ 0.041 CatalogClient.request simcore_service_director_v2/modules/catalog.py:59
│ │ └─ 0.041 AsyncClient.request httpx/_client.py:1519
│ │ ├─ 0.039 AsyncClient.send httpx/_client.py:1628
│ │ │ ├─ 0.036 AsyncClient._send_handling_auth httpx/_client.py:1677
│ │ │ │ └─ 0.036 AsyncClient._send_handling_redirects httpx/_client.py:1711
│ │ │ │ └─ 0.036 AsyncClient._send_single_request httpx/_client.py:1749
│ │ │ │ └─ 0.036 AsyncHTTPTransport.handle_async_request httpx/_transports/default.py:354
│ │ │ │ └─ 0.036 AsyncConnectionPool.handle_async_request httpcore/_async/connection_pool.py:159
│ │ │ │ ├─ 0.033 AsyncHTTPConnection.handle_async_request httpcore/_async/connection.py:67
│ │ │ │ │ ├─ 0.032 AsyncHTTP11Connection.handle_async_request httpcore/_async/http11.py:72
│ │ │ │ │ │ ├─ 0.029 AsyncHTTP11Connection._receive_response_headers httpcore/_async/http11.py:179
│ │ │ │ │ │ │ └─ 0.029 AsyncHTTP11Connection._receive_event httpcore/_async/http11.py:216
│ │ │ │ │ │ │ ├─ 0.028 AnyIOStream.read httpcore/_backends/anyio.py:23
│ │ │ │ │ │ │ │ └─ 0.028 SocketStream.receive anyio/_backends/_asyncio.py:1124
│ │ │ │ │ │ │ │ ├─ 0.023 Event.wait asyncio/locks.py:201
│ │ │ │ │ │ │ │ │ └─ 0.023 [await] asyncio/locks.py
│ │ │ │ │ │ │ │ └─ 0.005 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ │ │ └─ 0.005 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ └─ 0.005 [await] asyncio/tasks.py
│ │ │ │ │ │ │ └─ 0.001 Connection.next_event h11/_connection.py:426
│ │ │ │ │ │ │ └─ 0.001 Connection._extract_next_receive_event h11/_connection.py:398
│ │ │ │ │ │ │ └─ 0.001 maybe_read_from_SEND_RESPONSE_server h11/_readers.py:94
│ │ │ │ │ │ ├─ 0.002 AsyncLock.__aenter__ httpcore/_synchronization.py:69
│ │ │ │ │ │ │ └─ 0.002 Lock.acquire anyio/_core/_synchronization.py:156
│ │ │ │ │ │ │ └─ 0.002 cancel_shielded_checkpoint anyio/lowlevel.py:42
│ │ │ │ │ │ │ └─ 0.002 AsyncIOBackend.cancel_shielded_checkpoint anyio/_backends/_asyncio.py:2038
│ │ │ │ │ │ │ ├─ 0.001 sleep asyncio/tasks.py:593
│ │ │ │ │ │ │ │ └─ 0.001 [await] asyncio/tasks.py
│ │ │ │ │ │ │ └─ 0.001 CancelScope.__exit__ anyio/_backends/_asyncio.py:397
│ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._send_request_headers httpcore/_async/http11.py:147
│ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._send_event httpcore/_async/http11.py:170
│ │ │ │ │ │ └─ 0.001 AnyIOStream.write httpcore/_backends/anyio.py:39
│ │ │ │ │ │ └─ 0.001 SocketStream.send anyio/_backends/_asyncio.py:1158
│ │ │ │ │ │ └─ 0.001 AsyncIOBackend.checkpoint anyio/_backends/_asyncio.py:2015
│ │ │ │ │ │ └─ 0.001 sleep asyncio/tasks.py:593
│ │ │ │ │ │ └─ 0.001 [await] asyncio/tasks.py
│ │ │ │ │ └─ 0.001 AsyncLock.__aenter__ httpcore/_synchronization.py:69
│ │ │ │ │ └─ 0.001 Lock.acquire anyio/_core/_synchronization.py:156
│ │ │ │ │ └─ 0.001 cancel_shielded_checkpoint anyio/lowlevel.py:42
│ │ │ │ │ └─ 0.001 AsyncIOBackend.cancel_shielded_checkpoint anyio/_backends/_asyncio.py:2038
│ │ │ │ │ └─ 0.001 sleep asyncio/tasks.py:593
│ │ │ │ │ └─ 0.001 [await] asyncio/tasks.py
│ │ │ │ └─ 0.003 AsyncConnectionPool._assign_requests_to_connections httpcore/_async/connection_pool.py:230
│ │ │ │ └─ 0.003 AsyncHTTPConnection.has_expired httpcore/_async/connection.py:185
│ │ │ │ └─ 0.003 AsyncHTTP11Connection.has_expired httpcore/_async/http11.py:281
│ │ │ │ └─ 0.003 AnyIOStream.get_extra_info httpcore/_backends/anyio.py:83
│ │ │ │ └─ 0.003 SocketStream.extra anyio/_core/_typedattr.py:60
│ │ │ │ └─ 0.003 SocketStream.extra_attributes anyio/abc/_sockets.py:58
│ │ │ │ ├─ 0.002 [self] anyio/abc/_sockets.py
│ │ │ │ └─ 0.001 SocketStream._raw_socket anyio/_backends/_asyncio.py:1120
│ │ │ └─ 0.003 Response.aread httpx/_models.py:906
│ │ │ └─ 0.003 <listcomp> httpx/_models.py:911
│ │ │ ├─ 0.002 Response.aiter_bytes httpx/_models.py:914
│ │ │ │ ├─ 0.001 Response.aiter_raw httpx/_models.py:969
│ │ │ │ │ └─ 0.001 Response.aclose httpx/_models.py:997
│ │ │ │ │ └─ 0.001 BoundAsyncStream.aclose httpx/_client.py:152
│ │ │ │ │ └─ 0.001 AsyncResponseStream.aclose httpx/_transports/default.py:257
│ │ │ │ │ └─ 0.001 PoolByteStream.aclose httpcore/_async/connection_pool.py:369
│ │ │ │ └─ 0.001 [self] httpx/_models.py
│ │ │ └─ 0.001 [await] httpx/_models.py
│ │ └─ 0.002 AsyncClient.build_request httpx/_client.py:319
│ │ ├─ 0.001 AsyncClient._merge_url httpx/_client.py:370
│ │ │ └─ 0.001 URL.copy_with httpx/_urls.py:341
│ │ │ └─ 0.001 URL.__init__ httpx/_urls.py:75
│ │ │ └─ 0.001 ParseResult.copy_with httpx/_urlparse.py:124
│ │ │ └─ 0.001 urlparse httpx/_urlparse.py:151
│ │ │ └─ 0.001 quote httpx/_urlparse.py:449
│ │ └─ 0.001 Request.__init__ httpx/_models.py:309
│ │ └─ 0.001 URL.copy_merge_params httpx/_urls.py:365
│ │ └─ 0.001 QueryParams.merge httpx/_urls.py:580
│ └─ 0.001 AsyncRetrying.iter tenacity/_asyncio.py:104
│ └─ 0.001 AsyncRetrying._run_retry tenacity/_asyncio.py:85
├─ 0.024 Queue.get asyncio/queues.py:150
│ └─ 0.024 [await] asyncio/queues.py
├─ 0.013 _get_service_details simcore_service_director_v2/modules/db/repositories/comp_tasks/_utils.py:80
│ └─ 0.013 CatalogClient.get_service simcore_service_director_v2/modules/catalog.py:64
│ ├─ 0.011 wrapper_func simcore_service_director_v2/utils/client_decorators.py:55
│ │ └─ 0.011 async_wrapped tenacity/_asyncio.py:138
│ │ └─ 0.011 AsyncRetrying.__call__ tenacity/_asyncio.py:51
│ │ ├─ 0.010 CatalogClient.request simcore_service_director_v2/modules/catalog.py:59
│ │ │ └─ 0.010 AsyncClient.request httpx/_client.py:1519
│ │ │ ├─ 0.009 AsyncClient.send httpx/_client.py:1628
│ │ │ │ ├─ 0.005 Response.aread httpx/_models.py:906
│ │ │ │ │ └─ 0.005 <listcomp> httpx/_models.py:911
│ │ │ │ │ ├─ 0.003 [await] httpx/_models.py
│ │ │ │ │ └─ 0.002 Response.aiter_bytes httpx/_models.py:914
│ │ │ │ │ └─ 0.002 Response.aiter_raw httpx/_models.py:969
│ │ │ │ │ ├─ 0.001 Response.aclose httpx/_models.py:997
│ │ │ │ │ │ └─ 0.001 BoundAsyncStream.aclose httpx/_client.py:152
│ │ │ │ │ │ └─ 0.001 AsyncResponseStream.aclose httpx/_transports/default.py:257
│ │ │ │ │ │ └─ 0.001 PoolByteStream.aclose httpcore/_async/connection_pool.py:369
│ │ │ │ │ │ └─ 0.001 AsyncConnectionPool._assign_requests_to_connections httpcore/_async/connection_pool.py:230
│ │ │ │ │ │ └─ 0.001 AsyncHTTPConnection.has_expired httpcore/_async/connection.py:185
│ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection.has_expired httpcore/_async/http11.py:281
│ │ │ │ │ │ └─ 0.001 AnyIOStream.get_extra_info httpcore/_backends/anyio.py:83
│ │ │ │ │ │ └─ 0.001 SocketStream.extra anyio/_core/_typedattr.py:60
│ │ │ │ │ │ └─ 0.001 SocketStream.extra_attributes anyio/abc/_sockets.py:58
│ │ │ │ │ │ └─ 0.001 SocketStream._raw_socket anyio/_backends/_asyncio.py:1120
│ │ │ │ │ │ └─ 0.001 TCPTransport.get_extra_info <built-in>
│ │ │ │ │ └─ 0.001 BoundAsyncStream.__aiter__ httpx/_client.py:148
│ │ │ │ │ └─ 0.001 AsyncResponseStream.__aiter__ httpx/_transports/default.py:252
│ │ │ │ │ └─ 0.001 PoolByteStream.__aiter__ httpcore/_async/connection_pool.py:361
│ │ │ │ │ └─ 0.001 HTTP11ConnectionByteStream.__aiter__ httpcore/_async/http11.py:337
│ │ │ │ │ └─ 0.001 Trace.__init__ httpcore/_trace.py:10
│ │ │ │ └─ 0.004 AsyncClient._send_handling_auth httpx/_client.py:1677
│ │ │ │ └─ 0.004 AsyncClient._send_handling_redirects httpx/_client.py:1711
│ │ │ │ └─ 0.004 AsyncClient._send_single_request httpx/_client.py:1749
│ │ │ │ └─ 0.004 AsyncHTTPTransport.handle_async_request httpx/_transports/default.py:354
│ │ │ │ ├─ 0.003 AsyncConnectionPool.handle_async_request httpcore/_async/connection_pool.py:159
│ │ │ │ │ ├─ 0.002 AsyncHTTPConnection.handle_async_request httpcore/_async/connection.py:67
│ │ │ │ │ │ └─ 0.002 AsyncHTTP11Connection.handle_async_request httpcore/_async/http11.py:72
│ │ │ │ │ │ ├─ 0.001 AsyncHTTP11Connection._receive_response_headers httpcore/_async/http11.py:179
│ │ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._receive_event httpcore/_async/http11.py:216
│ │ │ │ │ │ │ └─ 0.001 Connection.receive_data h11/_connection.py:352
│ │ │ │ │ │ │ └─ 0.001 ReceiveBuffer.__iadd__ h11/_receivebuffer.py:53
│ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._send_request_body httpcore/_async/http11.py:159
│ │ │ │ │ │ └─ 0.001 AsyncHTTP11Connection._send_event httpcore/_async/http11.py:170
│ │ │ │ │ │ └─ 0.001 Connection.send h11/_connection.py:491
│ │ │ │ │ │ └─ 0.001 Connection.send_with_data_passthrough h11/_connection.py:518
│ │ │ │ │ │ └─ 0.001 ContentLengthWriter.__call__ h11/_writers.py:63
│ │ │ │ │ └─ 0.001 AsyncConnectionPool._assign_requests_to_connections httpcore/_async/connection_pool.py:230
│ │ │ │ │ └─ 0.001 <listcomp> httpcore/_async/connection_pool.py:270
│ │ │ │ └─ 0.001 Request.__init__ httpcore/_models.py:326
│ │ │ │ └─ 0.001 enforce_headers httpcore/_models.py:61
│ │ │ │ └─ 0.001 _SpecialGenericAlias.__instancecheck__ typing.py:993
│ │ │ └─ 0.001 AsyncClient.build_request httpx/_client.py:319
│ │ │ └─ 0.001 Request.__init__ httpx/_models.py:309
│ │ │ └─ 0.001 URL.copy_merge_params httpx/_urls.py:365
│ │ │ └─ 0.001 URL.copy_with httpx/_urls.py:341
│ │ │ └─ 0.001 URL.__init__ httpx/_urls.py:75
│ │ │ └─ 0.001 ParseResult.copy_with httpx/_urlparse.py:124
│ │ │ └─ 0.001 urlparse httpx/_urlparse.py:151
│ │ │ └─ 0.001 quote httpx/_urlparse.py:449
│ │ │ └─ 0.001 finditer re.py:242
│ │ │ └─ 0.001 _compile re.py:288
│ │ │ └─ 0.001 isinstance <built-in>
│ │ └─ 0.001 AsyncRetrying.iter tenacity/_asyncio.py:104
│ │ └─ 0.001 AsyncRetrying._run_retry tenacity/_asyncio.py:85
│ │ └─ 0.001 inner tenacity/_asyncio.py:77
│ │ └─ 0.001 retry_if_exception_type.__call__ tenacity/retry.py:68
│ │ └─ 0.001 Future.failed tenacity/__init__.py:495
│ └─ 0.002 [await] simcore_service_director_v2/modules/catalog.py
├─ 0.008 dask_sub_consumer_task simcore_service_director_v2/utils/dask.py:459
│ └─ 0.008 _dask_sub_consumer simcore_service_director_v2/utils/dask.py:443
│ ├─ 0.007 DaskScheduler._task_log_change_handler simcore_service_director_v2/modules/comp_scheduler/dask_scheduler.py:354
│ │ ├─ 0.005 publish_service_log simcore_service_director_v2/utils/rabbitmq.py:147
│ │ │ └─ 0.005 RabbitMQClient.publish servicelib/rabbitmq/_client.py:304
│ │ │ ├─ 0.003 RobustExchange.publish aio_pika/exchange.py:163
│ │ │ │ └─ 0.003 Channel.basic_publish aiormq/channel.py:618
│ │ │ │ ├─ 0.002 [self] aiormq/channel.py
│ │ │ │ └─ 0.001 ChannelFrame.marshall aiormq/abc.py:209
│ │ │ │ └─ 0.001 marshal pamqp/frame.py:25
│ │ │ │ └─ 0.001 _marshal_content_header_frame pamqp/frame.py:109
│ │ │ │ └─ 0.001 ContentHeader.marshal pamqp/header.py:90
│ │ │ │ └─ 0.001 Properties.marshal pamqp/base.py:165
│ │ │ │ └─ 0.001 Properties.encode_property pamqp/base.py:154
│ │ │ │ └─ 0.001 by_type pamqp/encode.py:36
│ │ │ ├─ 0.001 [self] servicelib/rabbitmq/_client.py
│ │ │ └─ 0.001 RobustChannel.declare_exchange aio_pika/robust_channel.py:167
│ │ │ └─ 0.001 RobustChannel.declare_exchange aio_pika/channel.py:225
│ │ │ └─ 0.001 RobustExchange.__init__ aio_pika/robust_exchange.py:24
│ │ │ └─ 0.001 Lock.__init__ asyncio/locks.py:77
│ │ ├─ 0.001 [self] simcore_service_director_v2/modules/comp_scheduler/dask_scheduler.py
│ │ └─ 0.001 UUID.__init__ uuid.py:138
│ └─ 0.001 DaskScheduler._task_progress_change_handler simcore_service_director_v2/modules/comp_scheduler/dask_scheduler.py:323
│ └─ 0.001 CompTasksRepository.get_task simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py:35
│ └─ 0.001 match re.py:187
│ └─ 0.001 _compile re.py:288
│ └─ 0.001 isinstance <built-in>
├─ 0.007 Lock.__aexit__ asyncio/locks.py:19
│ └─ 0.007 [await] asyncio/locks.py
├─ 0.006 CompTasksRepository._update_task simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py:175
│ ├─ 0.002 [self] simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py
│ ├─ 0.002 SAConnection._execute aiopg/sa/connection.py:106
│ │ ├─ 0.001 [self] aiopg/sa/connection.py
│ │ └─ 0.001 Update.compile sqlalchemy/sql/elements.py:439
│ │ └─ 0.001 Update._compiler sqlalchemy/sql/elements.py:563
│ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ └─ 0.001 Update._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ └─ 0.001 APGCompiler_psycopg2.visit_update sqlalchemy/sql/compiler.py:4168
│ │ └─ 0.001 dict._generate_delimited_and_list sqlalchemy/sql/compiler.py:1737
│ │ └─ 0.001 BooleanClauseList._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ └─ 0.001 APGCompiler_psycopg2.visit_clauselist sqlalchemy/sql/compiler.py:1758
│ │ └─ 0.001 dict._generate_delimited_list sqlalchemy/sql/compiler.py:1730
│ │ └─ 0.001 <genexpr> sqlalchemy/sql/compiler.py:1731
│ │ └─ 0.001 <genexpr> sqlalchemy/sql/compiler.py:1733
│ │ └─ 0.001 BinaryExpression._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ └─ 0.001 APGCompiler_psycopg2.visit_binary sqlalchemy/sql/compiler.py:2232
│ │ └─ 0.001 APGCompiler_psycopg2._generate_generic_binary sqlalchemy/sql/compiler.py:2321
│ │ └─ 0.001 BindParameter._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ └─ 0.001 APGCompiler_psycopg2.visit_bindparam sqlalchemy/sql/compiler.py:2467
│ │ └─ 0.001 APGCompiler_psycopg2._truncate_bindparam sqlalchemy/sql/compiler.py:2682
│ │ └─ 0.001 APGCompiler_psycopg2._truncated_identifier sqlalchemy/sql/compiler.py:2695
│ │ └─ 0.001 _anonymous_label.apply_map sqlalchemy/sql/elements.py:5440
│ │ └─ 0.001 prefix_anon_map.__missing__ sqlalchemy/sql/base.py:1025
│ ├─ 0.001 RowProxy.__getattr__ aiopg/sa/result.py:61
│ │ └─ 0.001 RowProxy.__getitem__ aiopg/sa/result.py:33
│ └─ 0.001 Column.__eq__ sqlalchemy/sql/operators.py:380
│ └─ 0.001 Column.operate sqlalchemy/sql/elements.py:872
│ └─ 0.001 Comparator.__eq__ sqlalchemy/sql/operators.py:380
│ └─ 0.001 Comparator.operate sqlalchemy/sql/type_api.py:73
│ └─ 0.001 _boolean_compare sqlalchemy/sql/default_comparator.py:30
│ └─ 0.001 expect sqlalchemy/sql/coercions.py:112
│ └─ 0.001 BinaryElementImpl._literal_coercion sqlalchemy/sql/coercions.py:526
│ └─ 0.001 Column._bind_param sqlalchemy/sql/elements.py:4733
│ └─ 0.001 BindParameter.__init__ sqlalchemy/sql/elements.py:1263
│ └─ 0.001 _anonymous_label.safe_construct sqlalchemy/sql/elements.py:5395
│ └─ 0.001 sub re.py:202
│ └─ 0.001 _compile re.py:288
├─ 0.004 [self] asyncio/runners.py
├─ 0.004 Channel._reader aiormq/channel.py:417
│ ├─ 0.002 Channel._on_return_frame aiormq/channel.py:314
│ │ ├─ 0.001 RobustChannel._on_return aio_pika/channel.py:214
│ │ │ └─ 0.001 IncomingMessage.__init__ aio_pika/message.py:352
│ │ │ └─ 0.001 IncomingMessage.__setattr__ aio_pika/message.py:281
│ │ └─ 0.001 Channel._read_content aiormq/channel.py:239
│ ├─ 0.001 Channel._get_frame aiormq/channel.py:149
│ │ └─ 0.001 Queue.get asyncio/queues.py:150
│ │ └─ 0.001 Loop.create_future <built-in>
│ └─ 0.001 Channel._on_confirm_frame aiormq/channel.py:372
│ └─ 0.001 Channel._confirm_delivery aiormq/channel.py:348
├─ 0.003 sem_task servicelib/utils.py:117
│ └─ 0.003 DaskScheduler._schedule_pipeline simcore_service_director_v2/modules/comp_scheduler/base_scheduler.py:602
│ └─ 0.003 int._send_running_tasks_heartbeat simcore_service_director_v2/modules/comp_scheduler/base_scheduler.py:328
│ └─ 0.003 DaskScheduler._get_pipeline_tasks simcore_service_director_v2/modules/comp_scheduler/base_scheduler.py:250
│ └─ 0.003 CompTasksRepository.list_computational_tasks simcore_service_director_v2/modules/db/repositories/comp_tasks/_core.py:62
│ ├─ 0.002 _IterableContextManager.__anext__ aiopg/utils.py:113
│ │ └─ 0.002 SAConnection._execute aiopg/sa/connection.py:106
│ │ ├─ 0.001 Cursor.execute aiopg/connection.py:400
│ │ │ └─ 0.001 cursor.execute <built-in>
│ │ └─ 0.001 Select.compile sqlalchemy/sql/elements.py:439
│ │ └─ 0.001 Select._compiler sqlalchemy/sql/elements.py:563
│ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:722
│ │ └─ 0.001 APGCompiler_psycopg2.__init__ sqlalchemy/sql/compiler.py:421
│ │ └─ 0.001 APGCompiler_psycopg2.process sqlalchemy/sql/compiler.py:498
│ │ └─ 0.001 Select._compiler_dispatch sqlalchemy/sql/visitors.py:71
│ │ └─ 0.001 bool.visit_select sqlalchemy/sql/compiler.py:3377
│ │ └─ 0.001 CompileState.create_for_statement sqlalchemy/sql/base.py:486
│ │ └─ 0.001 SelectState.create_for_statement sqlalchemy/sql/base.py:486
│ │ └─ 0.001 SelectState.__init__ sqlalchemy/sql/selectable.py:4382
│ │ └─ 0.001 Select._generate_columns_plus_names sqlalchemy/sql/selectable.py:6195
│ │ └─ 0.001 SelectState._column_naming_convention sqlalchemy/sql/selectable.py:4430
│ └─ 0.001 UUID.__init__ uuid.py:138
├─ 0.003 Lock.__aenter__ asyncio/locks.py:13
│ └─ 0.003 Lock.acquire asyncio/locks.py:93
│ └─ 0.003 [await] asyncio/locks.py
├─ 0.003 _ready aiopg/connection.py:779
│ ├─ 0.002 [self] aiopg/connection.py
│ └─ 0.001 decode encodings/utf_8.py:15
├─ 0.003 HttpToolsProtocol.data_received uvicorn/protocols/http/httptools_impl.py:170
│ ├─ 0.001 HttpToolsProtocol.on_headers_complete uvicorn/protocols/http/httptools_impl.py:247
│ ├─ 0.001 HttpToolsProtocol._unset_keepalive_if_required uvicorn/protocols/http/httptools_impl.py:139
│ │ └─ 0.001 TimerHandle.cancel <built-in>
│ └─ 0.001 HttpToolsProtocol.on_message_begin uvicorn/protocols/http/httptools_impl.py:221
├─ 0.002 DaskScheduler._process_task_result simcore_service_director_v2/modules/comp_scheduler/dask_scheduler.py:232
│ └─ 0.002 parse_output_data simcore_service_director_v2/utils/dask.py:141
│ └─ 0.002 Port.set_value simcore_sdk/node_ports_v2/port.py:371
│ └─ 0.002 dump simcore_sdk/node_ports_v2/serialization_v2.py:109
│ └─ 0.002 compute_node_hash models_library/utils/nodes.py:32
│ └─ 0.002 get_node_io_payload_cb simcore_sdk/node_ports_v2/serialization_v2.py:120
│ └─ 0.002 load simcore_sdk/node_ports_v2/serialization_v2.py:32
│ ├─ 0.001 Nodeports.__init__ simcore_sdk/node_ports_v2/nodeports_v2.py:49
│ │ └─ 0.001 Port.__init__ simcore_sdk/node_ports_v2/port.py:138
│ └─ 0.001 DBManager.get_ports_configuration_from_node_uuid simcore_sdk/node_ports_common/dbmanager.py:137
│ └─ 0.001 _ContextManager.__aenter__ aiopg/utils.py:81
│ └─ 0.001 Engine._acquire aiopg/sa/engine.py:205
│ └─ 0.001 Pool._acquire aiopg/pool.py:304
│ └─ 0.001 Condition.__aenter__ asyncio/locks.py:13
│ └─ 0.001 Lock.acquire asyncio/locks.py:93
├─ 0.002 Queue.put asyncio/queues.py:111
│ └─ 0.002 [await] asyncio/queues.py
├─ 0.001 _outer_done_callback asyncio/tasks.py:864
├─ 0.001 AsyncIOMainLoop._handle_events tornado/platform/asyncio.py:200
│ └─ 0.001 SSLIOStream._handle_events tornado/iostream.py:677
├─ 0.001 _release_waiter asyncio/tasks.py:387
└─ 0.001 Connection.__reader aiormq/connection.py:614
└─ 0.001 FrameReceiver.__anext__ aiormq/connection.py:228
└─ 0.001 FrameReceiver.get_frame aiormq/connection.py:172
└─ 0.001 unmarshal pamqp/frame.py:44From this I draw the following conclusion: Quite a lot of time is spent aquiring asyncio locks for writing into the db, but there is not a single query/function which seems to carry a large overhead. Hence, perhaps the solution forward is to make this endpoint a long running task and simply return a 202 (accepted)
wvangeit
commented
2 months ago Just to update the status. @bisgaard-itis are you (still) working on this?
bisgaard-itis
commented
2 months ago Just to update the status. @bisgaard-itis are you (still) working on this?
Not actively, no. I will assign @pcrespov as well and set the state to Planned in the MetaModeling project instead of In Progress.
Is there an existing issue for this?
Which deploy/s?
No response
Current Behavior
When running
make parallel-masterin the project https://github.com/wvangeit/osparc-pyapi-tests I can see that we still have a performance bottleneck in the following webserver endpointAs is shown in the following plot from Grafana (obtained while running the above command), this endpoint is too slow.
This is the cause of a lot of the 502 exit statuses encountered. The reason for this is the following: By default the python client retries when a 504 (gateway timeout) is returned. However, it seems that even thought the request from api server to web server is cancelled, the
startendpoint finishes and the job is started. So next time the user'sosparcclient retries to start the job, the webserver responds that it is already running (403, which is then converted into a 502 by the api-server).Solutions to the following tasks are required:
Expected Behavior
No response
Steps To Reproduce
No response
Anything else?
No response