search

ESM-VFC / esm-vfc-api-demo

MIT License
3 stars 2 forks source link

Add some simple end-to-end workflow with NEMO / FESOM and Pangaea #5

Open willirath opened 4 years ago

willirath commented 4 years ago

We should add a more realistic example here.

benbovy commented 4 years ago

I think that the current, multi-apps architecture (xpublish's FastAPI apps + Flask app) will quickly become problematic if we want to add more realistic examples. Currently:

  1. For each dataset, we need to run a xpublish FastAPI application
  2. Additionally, we need to run a top-level FastAPI application, acting like a reverse proxy, in which we mount all single dataset applications
  3. If we want to add or remove one or more datasets, we need to manage 1 and 2 manually or automatically and restart the top-level application.
  4. The Flask application mirrors some API provided by xpublish (duplicated maintenance)
  5. Accessing the datasets in the Flask application using fsspec's HTTPFileSystem introduces overhead.

I'm not sure if there is any benefit of having all those layers here. It's probably not worth the extra complexity.

Running a single FastAPI application would be much simpler, IMO. That's not currently possible with xpublish, but it could be supported with little effort (see https://github.com/jhamman/xpublish/issues/25).