I/O failures are not reported to the user

[leafty]

A user can mount a cloud storage (e.g. s3://giab) where they have no write permissions in read-write mode. When this happens, the user can do file system operations just fine on the mounted storage.

In reality the I/O operations are only performed on the sessions's PVC but must fail to be synced to the S3 bucket.

[olevski]

Logs from rlcone when this occurs:

I0531 14:53:53.645952       1 rclone.go:348] rclone log:        status code: 403, request id: E5NPCGW6GKVF5QTV, host id: p02LV6e6H1Sp7iEMK+4z7H9OAxsD3UY+0Z1I2Rcf/M0ml2byFD42O+z3RtVJd8Rf/7fhyViMPu4=
I0531 14:53:53.645972       1 rclone.go:348] rclone log: 2024/05/31 14:53:53 ERROR : ubnafsdlfjlsdjkfadf: vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file from cache to remote: AccessDenied: Access Denied
I0531 14:53:53.645980       1 rclone.go:348] rclone log:        status code: 403, request id: E5NPCGW6GKVF5QTV, host id: p02LV6e6H1Sp7iEMK+4z7H9OAxsD3UY+0Z1I2Rcf/M0ml2byFD42O+z3RtVJd8Rf/7fhyViMPu4=
I0531 14:53:53.748718       1 rclone.go:348] rclone log: 2024/05/31 14:53:53 ERROR : x: Failed to copy: AccessDenied: Access Denied
I0531 14:53:53.748794       1 rclone.go:348] rclone log:        status code: 403, request id: E5NSD2TT18SM662E, host id: sheGMgiK6iAfv//Pn479EdTUAU5K09u02uiV+ZMCWzwQrxUMOeX+KVugDgXOUsLsaTbpidk8iXA=
I0531 14:53:53.748816       1 rclone.go:348] rclone log: 2024/05/31 14:53:53 ERROR : x: vfs cache: failed to upload try #2, will retry in 20s: vfs cache: failed to transfer file from cache to remote: AccessDenied: Access Denied
I0531 14:53:53.748835       1 rclone.go:348] rclone log:        status code: 403, request id: E5NSD2TT18SM662E, host id: sheGMgiK6iAfv//Pn479EdTUAU5K09u02uiV+ZMCWzwQrxUMOeX+KVugDgXOUsLsaTbpidk8iXA=
I0531 14:54:03.971915       1 rclone.go:348] rclone log: 2024/05/31 14:54:03 ERROR : ubnafsdlfjlsdjkfadf: Failed to copy: AccessDenied: Access Denied