cproctor / qualitative-coding

Qualitative coding for computer scientists
Other
9 stars 3 forks source link

Propose structure of new CLI command for AI-assisted coding #22

Open cproctor opened 8 months ago

cproctor commented 8 months ago
cproctor commented 3 weeks ago

As noted in the docs, you can already define a custom editor, which could delegate to an LLM or whatever you want:

You could automatically apply codes to a document by writing a script and defining it as an editor (see editors); the script would receive the path to a corpus file and the codes file, and would write codes into the codes file. This opens the possibility of methodological innovation. For example, when we were analyzing student-written computer programs, we combined manual qualitative coding with automated static analysis of the programs, which added codes marking syntactic structures and manipulation of variables. This allowed us to integrate what students were doing (via our qualitative coding) with how they were doing it (via static analysis).

However, I think there's still a use case for a custom autocode command; it would be desirable to have a cleaner interface (JSON), and to provide capacity for training data and testing metrics.