search

cruise-automation / webviz

web-based visualization libraries
https://webviz.io/
Apache License 2.0
2.01k stars 408 forks source link

Loading not-public files from GCS bucket #564

Open Klaudioz opened 3 years ago

Klaudioz commented 3 years ago

I've my bag files over a Google bucket and webviz can load the files perfectly using allUsers with Storage Legacy Bucket Reader and Storage Legacy Object Reader permissions (Public to internet). But using allAuthenticatedUsers it fails showing the message: Error initializing player. Error: Fetching remote file failed.

Is anyway to make it work using allAuthenticatedUsers permissions ?. To reach the hosted webviz server actually is needed to be authenticated.

vidaaudrey commented 3 years ago

Hi @Klaudioz, you could set up the webviz server which can have full access to GCS bucket via a service account, and have the webviz client request a signed URL from the server which will bypass the auth issue you just mentioned. It's also more secure.

Klaudioz commented 3 years ago

Thank you @vidaaudrey . Is any tutorial or documentation to do it ?. I've tried to deploy a webviz server in my kubernetes cluster, but it's not so easy like to do it with webviz.

vidaaudrey commented 3 years ago

Unfortunately we don't have any tutorial yet. If you have a node express app set up, we could help review the code. You could start by manually generating signed URL through gsutil and test if it works for you. Then set a simple server with the endpoint to generate signed urls that allows http://localhost:8080 to access. And finally you could modify the server code to server the static webviz files if needed.

Klaudioz commented 3 years ago

Thanks for your answer. One thing I did was mounting the GCS bucket with bag files over the webviz container using gcsfuse. Is any way to make webviz able to read the mounted files ?. I couldn't find any way to set or configure the way that webviz runs.

vidaaudrey commented 3 years ago

I'm not sure how gcsfuse would work. If you can find some js client code to read the gcsfuse bag data, you could modify the PlayerManager code to build the player by using BagDataProvider or you create create your own data providers as well should you need some special data handling. BagDataProvider is easier to get started since it accepts either a local file path or a remote URL.