What should I do to connect `Data Schema` file to AWS S3 or GCP Cloud Storage.

[minicats]

What should I do to connect Data Schema file to AWS S3 or GCP Cloud Storage.

[paveltiunov]

@minicats Hey Arone! You should use either Athena or BigQuery to query those: https://docs.aws.amazon.com/athena/latest/ug/getting-started.html https://cloud.google.com/bigquery/docs/loading-data-cloud-storage

[minicats]

@paveltiunov What I want to say... Data Schema file equal to

cube(`Users`, {
  sql: `SELECT * FROM users`
});

I want to make that Date Shema(Cube.js Schema) is connected to AWS S3 or GCP Cloud Storage.

[willhausman]

@minicats When creating the server you can specify a repositoryFactory option. https://cube.dev/docs/@cubejs-backend-server-core#options-reference-repository-factory

You can return an instance of your own object which implements async dataSchemaFiles() to return your files from where ever you need them stored. Know that it is usually only called once per appId, but it also will be called on every request when you're running the server in dev mode.

CubejsServerCore.create({
  repositoryFactory: (context) => ({
    async dataSchemaFiles() {
      // load your files from some other api
      // return them in the form [ { fileName, content } ]
    }
  })
});

@paveltiunov This should probably be added more explicitly to the documentation. It's obvious that repositoryFactory exists, but it's not obvious what you need to do if you want to run your own FileRepository implementation. Probably adding some more TypeScript definitions would help it, too.

[paveltiunov]

@willhausman Yeah. Agree! Would love PR for that!

[minicats]

@willhausman @paveltiunov Thanks!

[rongfengliang]

i write one simple s3-filerepository https://github.com/rongfengliang/cube.js-s3-filerepository (do some test with minio)