responsible-ai-collaborative / aiid

The AI Incident Database seeks to identify, define, and catalog artificial intelligence incidents.
https://incidentdatabase.ai
Other
167 stars 35 forks source link

Classification Interface #1334

Open smcgregor opened 1 year ago

smcgregor commented 1 year ago

This is a design issue to explore how we can better support the classification process from a user perspective (e.g., CSET and others classifying incidents). This is distinct from supporting them from a database structure perspective.

@npit has one potential design here: https://docs.google.com/document/d/1rFARyGcPJr6YhHu9MXdEkl0Yu7CMaJap417KUNWPjAc/edit?usp=sharing

@npit also mentioned: Doccano 0 and ellogon 1 as potential reference points.

lmcnulty commented 1 year ago

If we were to go in heavily on annotation, I wonder if we could integrate hypothes.is?

smcgregor commented 1 year ago

@lmcnulty any experience with it? It is open source, but it looks like it may be difficult to cram into the use case for which we need it.

lmcnulty commented 1 year ago

I only have experience with it as a user. It just came to mind because it allows annotation of web pages, associated discussion, and user-level tagging of annotations. It seems like it could give us many of the features called for in npit's design for very little effort, and looking at the API, it seems like we could plausibly integrate it with our systems in a useful way. Building our own annotation system would offer us more control – and generally be pretty cool – but it also seems like a very large amount of engineering work that we might want to put into something else instead.

smcgregor commented 1 year ago

Agreed on hypothes.is. I think we are likely to rework most of the taxonomy functionality to be more flexible, but the snippeting features would come in subsequent update -- we can evaluate the build/adopt question in a couple months.