review priorities and larger goals of code - Githubissues

cu-mkp / manuscript-object

Data extraction, transformation, manipulation, and analysis of the Making and Knowing Project's core dataset of the Digital Critical Edition, Secrets of Craft and Nature.

5 stars 3 forks source link

review priorities and larger goals of code #76

Open njr2128 opened 3 years ago

njr2128 commented 3 years ago

Go over open issues and goals in order to best set re-design/re-factoring steps

njr2128 commented 3 years ago

areas to consider:

things this repo does: generates derivatives
- refactoring has taken place to optimize this process, and we seem to be in a good place in this area
- one potential change: change the way we file write (to make testing easier)
other thing it does: make this interactable, so that visualizations can be made easily by others, for examples
- i.e., analysis, visualization, etc of the text data
- to go further down this road, we should consult users of this part of it --> sandbox?
- what are the kinds of analysis and visualizations we want to see and/or enable? (PHS should be part of this)
- TO DO: look back at user stories, example queries we created in the past (bring Greg up to speed by THC/NJR)
SHOULD WE DECOUPLE THESE TWO THINGS?

njr2128 commented 3 years ago

manuscript-object code should enable ready use/creation of datasets that can then be used for visualization or analysis

njr2128 commented 3 years ago

A "package" - Separate place for "i want this data in this form" to use for other things

gschare commented 3 years ago

update.py (the process of generating and writing derivatives) should ideally be a simple, single Python script (with some switches and configuration) that simply requests from the "manuscript object" data structures with various transformations applied, e.g. "an array of all folios rendered in plain text", i.e. allFolios/txt.

Most importantly, the core manuscript object ought not to perform any "actions" (writing files) itself. Rather its entire functionality should be to return Python data structures, which may then be used by some script to actually write that data. There are design principles guiding why I propose this separation that I won't get into right now. Most salient of these, however, is that while manuscript object itself should be highly flexible, the scripts which use manuscript object must be allowed to ossify to some degree and serve narrow, specific purposes.

njr2128 commented 3 years ago

spectrum of visualization purposes/services:

exploratory
illustrative
supportive
serving as evidence for new inferences or interpretations

njr2128 commented 3 years ago

building out jupyter notebooks

Matthew's notebook
Terry's old one based on J Reeves
Greg's version