search

clamsproject / clams-python

CLAMS SDK for python
http://sdk.clams.ai/
Apache License 2.0
4 stars 1 forks source link

discussion on `consumer` specs #93

Open keighrim opened 2 years ago

keighrim commented 2 years ago

(related: cataloguing consumer apps https://github.com/clamsproject/clams-apps/issues/22)

keighrim commented 1 year ago

I added https://github.com/clamsproject/consumer-evaluation repository, based on assumption that evaluation code will be consumers of MMIF. However, often ML evaluation pipeline takes a collection of documents instead of a single document, but the way other consumers we have currently were developed has been to handle a single MMIF input.

That said, my questions:

  1. ~Should evaluation code (or pipeline) be "consumers"?~ the repository in question is now renamed.
  2. More generally, should we limit the scope of the consumers to handling of a single input at a time? Or putting in other words, should we introduce a new class for "collection" consumers? Besides of evaluation, I can see dashboard builders (we talked about a possibility of using elastic-kibana for this) fall into this category.
marcverhagen commented 1 year ago

We could start with limiting consumers to one MMIF file at the time, like the producers. It is possible that there are use cases for collection-level consumers (and maybe collection-level producers as well), the question there is whether it is better to add collection consumers to the code here or run a batch like GBH does.

keighrim commented 5 months ago

Regarding "interface" of the consumers;

The first consumer (prototype) was mmif-visualizer and it was developed not only as a standalone web app, but also as a "display application" integrated to the galaxy workflow engine.

The /display route in the mmif-viz was originally designed for such integration to galaxy

https://github.com/clamsproject/mmif-visualizer/blob/1ff27c6c48ff04b37de714719aaa5776cd8acfe8/app.py#L41-L44

https://github.com/clamsproject/wfe-galaxy/blob/dec9e60472c229efb7161c148e95e21012b74c22/make_appliance.py#L416

(but during recent updates, the route has been re-purposed, and no longer support galaxy integration)

However, given

  1. we don't have a clear answer to https://github.com/clamsproject/mmif-summarizer/issues/9 , and
  2. having a common user interface or "entry point" for all consumers DOES make a lot of sense

I'd like to go back to the idea of using HTTP as the common interface, and want bring a common route (that any HTTP clients can call, including galaxy engine) defined in the interface.