Read only study uploads outputs for dynamic services

GitHK commented 1 year ago

dynamic-sidecar powered devices currently upload their ports even if the study is in read only mode. NOTE: their status is not saved!

It is required to block this behaviour.

https://github.com/ITISFoundation/osparc-issues/issues/638
requires https://github.com/ITISFoundation/osparc-simcore/pull/3974 (bring in can_save parameter when the dynamic service is started)
related to https://github.com/ITISFoundation/osparc-issues/issues/774

GitHK commented 1 year ago

Since #3974 was reverted. the can_save should be extracted and added again to the start of a dynamic sidecar

GitHK commented 1 year ago

Upon thinking more about this. The following can happen.

When a study is opened in read_only mode and output ports are blocked the following is true:

output data will still be pulled when the study is opened (this is good)
output data will not be uploaded when there is a change in the port (this can be either bad or good)

When data is not uploaded to the output ports in read only mode:

we make sure that no user who has read only access can ever change anything in a study
we might break existing published studies or templates which are set to read only and rely on this side effect for one of the processing steps to work

elisabettai commented 1 year ago

Hi @GitHK, what exactly do you mean "when a study is opened in read_only mode"? Are these two cases "read_only" for you?

A study is opened by a registered user who has only read access to the study
A study is opened by an anonymous user (who obviously only has read access to the study).

I have just done a test with case 1 and I see that the viewer (registered user) can write to an output port, but then this output is not saved (this bahavior looks good to me).

In the frontend logs, when the viewer writes an output I see line: uploading /dy-volumes/home/jovyan/work/outputs/output_3/test.txt, please wait... and then nothing happens.

GitHK commented 1 year ago

@elisabettai Yes, they are both read only. I don't see how you managed to do this experiment let's have a look together.

GitHK commented 1 year ago

Currently this is working as desired due to some side effect. See logs below

log_level=WARNING | log_timestamp=2023-05-08 12:58:39,031 | log_source=simcore_service_dynamic_sidecar.modules.outputs._manager:_remove_downloads(151) | log_msg=outputs_manager_port_keys-output_4 ended with exception: Insufficient access rights to write data 4a80a0aa-df7f-11ed-8dda-02420a0b76c8/8beea7be-a155-4628-a434-7fdb991977ec/outputs/output_4/untitled.txt
Traceback (most recent call last):
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_common/storage_client.py", line 30, in wrapped
    ret = await handler(*args, **kwargs)
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_common/storage_client.py", line 131, in get_upload_file_links
    response.raise_for_status()
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 403, message='Insufficient access rights to write data 4a80a0aa-df7f-11ed-8dda-02420a0b76c8/8beea7be-a155-4628-a434-7fdb991977ec/outputs/output_4/untitled.txt', url=URL('http://production_storage:8080/v0/locations/0/files/4a80a0aa-df7f-11ed-8dda-02420a0b76c8%2F8beea7be-a155-4628-a434-7fdb991977ec%2Foutputs%2Foutput_4%2Funtitled.txt?user_id=114328&link_type=PRESIGNED&file_size=0')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_dynamic_sidecar/modules/outputs/_manager.py", line 133, in _upload_ports
    await upload_outputs(
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_dynamic_sidecar/modules/nodeports.py", line 177, in upload_outputs
    await PORTS.set_multiple(ports_values, progress_bar=sub_progress)
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_v2/nodeports_v2.py", line 172, in set_multiple
    results = await logged_gather(*tasks)
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/utils.py", line 134, in logged_gather
    raise error
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_v2/port.py", line 336, in _set
    new_value = await port_utils.push_file_to_store(
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_v2/port_utils.py", line 219, in push_file_to_store
    store_id, e_tag = await filemanager.upload_file(
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_common/filemanager.py", line 325, in upload_file
    store_id, upload_links = await get_upload_links_from_s3(
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_common/filemanager.py", line 174, in get_upload_links_from_s3
    await storage_client.get_upload_file_links(
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_sdk/node_ports_common/storage_client.py", line 42, in wrapped
    raise exceptions.StorageInvalidCall(err.message) from err
simcore_sdk.node_ports_common.exceptions.StorageInvalidCall: Insufficient access rights to write data 4a80a0aa-df7f-11ed-8dda-02420a0b76c8/8beea7be-a155-4628-a434-7fdb991977ec/outputs/output_4/untitled.txt

To fix it I will disable the outputs watcher if the study is in READ ONLY mode! This will disable the watcher from detecting changes and uploading them to the output ports

ITISFoundation / osparc-simcore

Read only study uploads outputs for dynamic services #4044