Hook for 'catalog' query?

[mvahowe]

In Chaliki, at startup, there's a query to find out what's in Proskomma, and to display a summary for the user: https://github.com/Proskomma/chaliki/blob/bf11261b8ad738564bd165f825af2047058299ec/src/Pages/Standard/data/Data.js#L16

Would it be possible to have a hook to do this? We should probably add some more fields to make that hook more widely useful.

[klappy]

If I understand correctly, we are looking for a way to run this query prior to importing documents into Proskomma.

1. Create instance of Proskomma.
2. Run this query.
3. Import documents.
4. Run other queries.

I made the assumption that nothing runs between steps 1 and 3 and is currently not possible.

I should be able to accommodate for this this one of a dozen ways. Let me see which approach fits this design pattern...

In the mean time, you should be able to do the following:

1. Use ‘useProskomma’ with an empty array for documents to be imported.
2. Run the query on the empty Proskomma instance using ‘useQuery’
3. Update the documents array and the import will run.
4. Run other queries via ‘useQuery’.

————-

With the above, I’m starting to gather that the purpose of the query is to return what docs are already imported in Proskomma so that you can import new documents without reimporting documents that are already there.

Currently, but only temporarily, this library creates a fresh new Proskomma instance every time the documents array is changed.

I am planning to build the above use case into the design pattern of the ‘useProskomma’ hook to do this for us. When the documents array coming in changes, a query is run to evaluate if new documents are in the array. If there is new content it will import them. The state of the ‘useProskomma’ hook could likely be the result of this query as it makes sense.

Once this is implemented, there still won’t be a way to run a query prior to importing documents unless we add in callbacks to run during lifecycle events such as onImport.

————

1. Do I understand the use case correctly?
2. Does the proposed implementation handle the root need?
3. Do we have other use cases for running queries prior to importing? And if so would an onImport callback handle them?

Thanks!

[mvahowe]

@klappy The assumption in Chaliki is that Proskomma is the model, and the model tells the view what content it has. Obviously, if Proskomma is fired up by an app that feeds it content, the app 'knows' what content is in there. But, architecturally, this seems backwards to me and, also

• it can't possibly work as soon as there are multiple users, or background processes, or anything other than one particular view controlling everything, maybe including a second window in the same app.
• it's not trivial to catch potential import errors
• The various Proskomma serialized formats import multiple documents or even multiple docSets in one go. The only way for the view to know what has been imported would be to introspect those formats

So my view logic would be

1. get an instance of Proskomma, with or without content, from somewhere
2. run a query to find out what's in it
3. run other queries
4. eventually, run mutations that change the content of Proskomma, at which point the first query should run again.

Now there are clearly cases where there's no doubt what is in the Proskomma instance, and I wouldn't want to stop devs from making those kind of assumptions. But I think we need to be able to have an option for a clean MVP separation too.

[Imad-Hamzi]

Hello everyone, I am using the useProskomma and useQuery hooks. For this I have created a usfm file in which I store different data. Here is a query I would like to try: '{\n' + ' docSet(id:"%docSetId%") {\n' + ' document(bookCode:"%bookCode%") {\n' + ' title: header(id: "toc2"){\n' + ' mainSequence {\n' + ' blocks(withScriptureCV: "%chapter%") {\n' + ' bs { payload } {\n' + ' items { type subType payload }\n' + ' }\n' + ' }\n' + ' nav: cvNavigation(chapter:"%chapter%" verse: "1") {\n' + ' previousChapter\n' + ' nextChapter\n' + ' }\n' + ' }\n' + ' }\n' + '}'; The question I have is how to retrieve the docSet, the document, the title... with a usfm data and not JSON?

Thank you

[klappy]

Here is my proposal for the way useProskomma hook will work in the future...

1. useProskomma can have optional serialized data passed into it for resuming state, and allow that to be the only prop instead of documents. It's returned "state" is the Proskomma instance, stateId for changes, and onStateChange() for a trigger to later update stateId. Although Proskomma instance is not stored in state but just passed as if it was merged with the current state.
2. useImport is a new custom hook that new documents are passed into as props. The other prop would be the Proskomma instance passed from the useProskomma response. With each import it would trigger onStateChange() from useProskomma that would mutate the stateId as needed.
3. useQuery would work like it does now. It can then run queries on pk before new documents are imported.
4. useImported, is a new custom hook that wraps useQuery that runs the query to return what documents are loaded.

My goal is to decouple react state management from duplicating and storing the documents content after import as well as allow each hook to do only one thing. I now realize that by breaking the pattern of doing one thing it was creating the issue to begin with.

@mvahowe This will at least get us started in the direction and kick the tires for the above use cases. My gut tells me there will be some unforeseen react side effects that we will need to optimize for.

[mvahowe]

@klappy I think this is sounding better to me. But, on point 1, there are quite a few options, many of which are already in use, eg

• import a USFM document
• import a USX document
• import a table, a tree, key-value entries...
• import multiple documents for one docSet from a single JSON file (what several PoCs currently do)
• import multiple docSets from a single JSON file using proskomma-freeze

That last option would be the 'resume state' one.

In addition, we may want to add content to an existing document, which is the neatest way to handle trees and tables, and also the way to make verse mapping working (via vrs files). To do this, we need the Proskomma id of the document, ie having the USFM that created it doesn't help.

If we're looking for The One Way, it should really be via mutations, which can do what import does, but which can also handle all the other cases.

[klappy]

From my understanding proskomma-freeze is the only way to save and resume state. If that is the case, then separating the importing of state vs documents/docSets seems to help clarify that. Importing (resuming) state will be done in the useProskomma because it is picking up where another instance left off. Importing anything other than state should be done via the useImport as they are not any Proskomma state representation but something else to be parsed and imported to Proskomma.

I will keep reviewing the mutation examples to figure out what that custom hook/lifecycle could look like.

[mvahowe]

From my understanding proskomma-freeze is the only way to save and resume state.

It should be the easiest way to save and resume "everything". But it's also possible to save and load one docSet at a time (ie all the documents in that docSet as one file) and proskomma-freeze just adds another layer of JSON on top of that docSet-level functionality. So, depending on the granuality of saving, there may be more than one way to do it.

[mvahowe]

@klappy I think you are right that thawing something saved using proskomma-freeze should happen when Proskomma is instantiated, ie in useProskomma. For that to be useful we need a way to freeze the entire state.

[klappy]

Refactor for v1 has begun to handle the useImport implementation discussed above: https://github.com/unfoldingWord/proskomma-react-hooks/commit/ef117fd283ca92bda6ee15da54da6b4d80bc7758

https://proskomma-react-hooks.netlify.app/

The following has yet to be implemented:

• documents that aren't usfm, ie. usx
• importing multiple docsets via single json file using proskomma-freeze
• useImported to query to see what documents are imported into proskomma

[klappy]

Just realizing that useImported has the intention to address this issue. In the title, Hook for 'catalog' query, catalog is a better term to be used. I'll implement it via useCatalog to return the imported docsets.

[klappy]

Implemented useCatalog here: https://github.com/unfoldingWord/proskomma-react-hooks/pull/5

Remaining sub-issues discussed here should be discussed in other issues:

• importing resumed state via proskomma-freeze: https://github.com/unfoldingWord/proskomma-react-hooks/issues/4
• importing non-usfm files: https://github.com/unfoldingWord/proskomma-react-hooks/issues/3