eto-ai / rikai

Parquet-based ML data format optimized for working with unstructured data
https://rikai.readthedocs.io/en/latest/
Apache License 2.0
138 stars 19 forks source link

Security hardening of TorchHub Registry #484

Closed da-liii closed 2 years ago

da-liii commented 2 years ago

as discussed at #436

Options for security hardening

Options for the TorchHub Registry:

  1. Option to enable torchhub registery (Disabled by default)
  2. Option to define a white list of {org}/{proj}

These options should be immutable during runtime.

For the first Option, here is the expected error message by default:

>                   raise Py4JJavaError(
                        "An error occurred while calling {0}{1}{2}.\n".
                        format(target_id, ".", name), value)
E                   py4j.protocol.Py4JJavaError: An error occurred while calling o57.sql.
E                   : ai.eto.rikai.sql.model.ModelNotFoundException:
E                   TorchHub Registery is disabled by default for security concerns.
E                   Be cautious and set `rikai.sql.ml.registry.torchhub.enabled` to true
E                   only for personal usage or testing purpose.
E
E                       at ai.eto.rikai.sql.model.torchhub.TorchHubRegistry.resolve(TorchHubRegistry.scala:31)
E                       at ai.eto.rikai.sql.model.Registry$.resolve(Registry.scala:194)

Report to upstream

https://github.com/pytorch/pytorch/issues/71205

da-liii commented 2 years ago

We have move TorchHubRegistry to rikai.experimental by #436 .