The current spec requires that the catalog_file point to a csv file. In some cases, it would be useful to embed the catalog "table" in the catalog itself. A so called single-file-catalog. STAC has an extension that does this (see here).
I think the cleanest way to do this would be to make catalog_file key optional and support a key catalog_dict which is a json dictionary that represents the data that would otherwise be in the csv. Exactly one of the two keys would be required but the catalog creator could choose.
As @andersy005 points out in https://github.com/NCAR/intake-esm/pull/179#issuecomment-553630201, this could lead to large catalogs when the data table is very large. The same is true when this conceptual approach is taken in STAC but there are clear use cases when you don't want to carry around multiple files.
The current spec requires that the
catalog_filepoint to a csv file. In some cases, it would be useful to embed the catalog "table" in the catalog itself. A so called single-file-catalog. STAC has an extension that does this (see here).I think the cleanest way to do this would be to make
catalog_filekey optional and support a keycatalog_dictwhich is a json dictionary that represents the data that would otherwise be in the csv. Exactly one of the two keys would be required but the catalog creator could choose.As @andersy005 points out in https://github.com/NCAR/intake-esm/pull/179#issuecomment-553630201, this could lead to large catalogs when the data table is very large. The same is true when this conceptual approach is taken in STAC but there are clear use cases when you don't want to carry around multiple files.