cproctor / qualitative-coding

Qualitative coding for computer scientists
Other
12 stars 3 forks source link

Propose structure of new CLI command for AI-assisted coding #22

Open cproctor opened 10 months ago

cproctor commented 10 months ago
cproctor commented 3 months ago

As noted in the docs, you can already define a custom editor, which could delegate to an LLM or whatever you want:

You could automatically apply codes to a document by writing a script and defining it as an editor (see editors); the script would receive the path to a corpus file and the codes file, and would write codes into the codes file. This opens the possibility of methodological innovation. For example, when we were analyzing student-written computer programs, we combined manual qualitative coding with automated static analysis of the programs, which added codes marking syntactic structures and manipulation of variables. This allowed us to integrate what students were doing (via our qualitative coding) with how they were doing it (via static analysis).

However, I think there's still a use case for a custom autocode command; it would be desirable to have a cleaner interface (JSON), and to provide capacity for training data and testing metrics.