new format for databroker v2 catalogs

prjemian commented 1 year ago

With databroker 2.0 coming, there will be a new format for catalogs. We'll need some conversion tools from the old format to the new ones.

intake: databroker 1.2.5 and previous

This is the current format.

sources:
  training:
    args:
      asset_registry_db: mongodb://localhost:27017/training-bluesky
      metadatastore_db: mongodb://localhost:27017/training-bluesky
    driver: bluesky-mongo-normalized-catalog
  class_2021_03:
    args:
      asset_registry_db: mongodb://localhost:27017/class_2021_03-bluesky
      metadatastore_db: mongodb://localhost:27017/class_2021_03-bluesky
    driver: bluesky-mongo-normalized-catalog

databroker 2.0.0 uses tiled `profile.yml`

For databroker 2.0.0, it uses a profile.yml format. The long term view is that databroker will use a tiled server to provide r/w access. But it is possible to avoid the tiled server by providing a direct configuration.

training:
    direct:
        trees:
        - path: /
          tree: databroker.mongo_normalized:MongoAdapter.from_uri
          args:
            uri: mongodb://localhost:27017/training-bluesky

class_2021_03:
    direct:
        trees:
        - path: /
          tree: databroker.mongo_normalized:MongoAdapter.from_uri
          args:
            uri: mongodb://localhost:27017/class_2021_03-bluesky

tiled: server `config.yml`

This is a standard tiled config.yml for MongoDB catalogs.

trees:
    - path: training
      tree: databroker.mongo_normalized:Tree.from_uri
      args:
            uri: mongodb://localhost:27017/training-bluesky

  - path: class_2021_03
    tree: databroker.mongo_normalized:Tree.from_uri
    args:
      # for unsecured access
      uri: mongodb://localhost:27017/class_2021_03-bluesky

prjemian commented 1 year ago

Still use databroker.catalog_search_path() to discover list of directories in which to place these files. The location for databroker 2.0 will be different than before. Typically ~/.config/tiled/profiles/ is in this list.

prjemian commented 1 month ago

BCDA-APS / apstools