search

h-REA / holo-rea-proto

REA economic network coordination software implemented on top of the Holochain prototype (Go version).
GNU General Public License v3.0
17 stars 1 forks source link

Enable usage of DHT types in UI code #14

Closed pospi closed 5 years ago

pospi commented 5 years ago

It would be very helpful to get things to a point where we can share some types and classes between the DHT and the UI, in order to more easily type the return values of schema resolve callbacks. This may also involve adding deserialisation methods to the existing DHT classes, which seems useful.

Seems I've gone about as far as I can without understanding the compilation toolchain you've setup for the DHT, so this might be the time to standardise it. I wanted to pull getFixtures out of events.ts to use in the GraphQL layer, but in doing so I encountered a whole heap of type errors and I'm unsure why. I also had to disable noImplicitAny and strictBindCallApply in my UI TS config to get things to compile, FWIW.

It's probably due to differing tsconfig.json, but there are a few of those files so I'm not sure which to attack first. If you can answer these questions, I can probably create something that integrates better with the UI code:

Hopefully going through this process means I end up being able to import the DHT files into the UI code and use your classes and interfaces without issue.

Thinking ahead, I expect the way things will play out is that a lot of your backend code will make it across to the frontend before the current backend falls off the radar and we upgrade to Rust. (Or, ideally, the AssemblyScript SDK magically gets done soon and we can have code sharing between DHT & client apps.)

pospi commented 5 years ago

BTW, if you want to see what the errors are, uncomment this line: https://github.com/holo-rea/holo-rea-proto/blob/9498853/src/graphql-adapter/src/types.ts#L35

sqykly commented 5 years ago

It would be very helpful to get things to a point where we can share some types and classes between the DHT and the UI, in order to more easily type the return values of schema resolve callbacks. This may also involve adding deserialisation methods to the existing DHT classes, which seems useful.

Type sharing is found exclusively in zomes.d.ts (https://github.com/holo-rea/holo-rea-proto/blob/ef7e8677a086514801242c45279e59cf60ed6d1c/build/test/src/zomes.d.ts). A good description of what that is doing is found in the web API doc (https://github.com/holo-rea/holo-rea-proto/blob/master/docs/web-api.md).

Deserializing and serializing are done by the TS classes on the backend. Each class inherits most of that from common.HoloObject, or rather its subclass, VfObject.

Seems I've gone about as far as I can without understanding the compilation toolchain you've setup for the DHT, so this might be the time to standardise it. I wanted to pull getFixtures out of events.ts to use in the GraphQL layer, but in doing so I encountered a whole heap of type errors and I'm unsure why. I also had to disable noImplicitAny and strictBindCallApply in my UI TS config to get things to compile, FWIW.

The compilation toolchain is a complicated beast. The getFixtures functions are not doing anything client-side; they rely on the backend TS classes to construct the objects that are stored as fixtures, which in turn rely on the serialization capabilities of common.HoloObject, which finally relies on the Holochain API. So there is no possibility of having it run on the client, unless you want to use the mock API. The mock API is to be regarded as an awful idea, though.

It's probably due to differing tsconfig.json, but there are a few of those files so I'm not sure which to attack first.

It's been a while since I messed with it, but I believe the tsconfig in build is used to separate out the declarations to evade all the import issues that go-lochain has. The individual build/staging/zomeName/tsconfig.json are used to compile the final JS. The files can't all be compiled together, or typescript will ruin them with module import schemes.

* which files are the single source of truth for the DHT code?

The backend, which is not necessarily the same as the frontend, is all the ts and json files in src.

* are the JSON schema files needed, or can they be generated by the build somehow?

The JSON schema in src are not technically JSON schema; they have an extra $extends property to let them inherit. In addition, go-lochain does not do $ref. To fix both issues, the files all undergo pre-processing in build/JSON during the build process. They are needed if you want the holochain app to run. Any modifications should be done by hand (or generated from tools, but that's not much better) to the src schema.

* what are the `staging` folders for, and can they be removed?

staging is for 2 things:

  • Holding temporary intermediate files generated during the build. Those can be removed; they are removed when build starts.

  • Holding the particular tsconfig.json files needed to make the individual ts files compile into bin. These can't be removed, and changing their location will require a change in their paths; they are meant to be sitting in the folder that contains all of the files to compile.

    • which scripts in build are part of your build pipeline and what does each script do? The only one that is important right now is bin.sh. The individual files other than build.sh are just the steps in bin.sh. It doesn't call any of them, I just used/will use them to test individual steps of the build process. They are all inlined into bin.sh to cut down on redefining variables and functions.

build.sh is planned to compile the server, the ui, and the tests, and run any server side tests. Since I haven't had a working UI to compile, and abandoned server-side tests in go-lochain, it does nothing. If you want to integrate the UI build, either do it there (remembering to source bin.sh) or make a ui.sh, compile the UI there, and call that (and bin.sh) in build.sh.

Hopefully going through this process means I end up being able to import the DHT files into the UI code and use your classes and interfaces without issue.

Sadly, I think you will run into problems there. No part of the HoloREA backend is meant to be shared directly with anything client-side. The classes are dependent on the Holochain API as described above, so sadly, they are definitely not happening without a mock Holochain API, which is regarded as a very bad idea. The interfaces you can get to client side are all in zomes.d.ts and described in detail by web-api.md.

I don't know if DHT files are a thing. All the data entry is at run time now.

sqykly commented 5 years ago

One more thing: https://github.com/holo-rea/holo-rea-proto/blob/master/docs/build.md

That's the basics of the build process.

pospi commented 5 years ago

This all great documentation to have, even if I don't end up rebuilding it for the Go-lochain version.

Re getFixtures: I was thinking of wiring up getFixtures in the zomes file with getFixtures in the DHT code so that they share the constants, but I'm not sure that's really worth the effort for this version...

sqykly commented 5 years ago

Re getFixtures: I was thinking of wiring up getFixtures in the zomes file with getFixtures in the DHT code so that they share the constants, but I'm not sure that's really worth the effort for this version...

There is no zomes version. All of the functions in zomes.d.ts are the functions in the zomes. They all have all the same signatures and fields because they are not 2 separate things, but one and the same. The reason zomes.d.ts exists at all: there is no mechanism by which one can call a holochain function from the client directly. Importing them from source into the client will invariably fail because the client is not a holochain server. The only way to interact with a holochain app from the client is through its web-accessible zome functions. The DNA declares them, the zome code defines them, zomes.d.ts tells your typescript on the client side what they do, and zomes.js implements one function that encapsulates the protocol for calling up the web API. All of those functions it exports are just names and URLs to the client side, nothing else is happening there.

Multiple representations are needed as they can only stand on one side of the client/server barrier, but there is exactly one set of functions.

pospi commented 5 years ago

I'm gonna close this, I think what we have is good enough.

We crossed wires a bit on those last two comments- I was simply considering taking the string identifiers out of getFixtures and putting them into a common file; and importing those same constants in both the DHT code and in the UI. Not importing the whole getFixtures method into the frontend.

But I don't think that's really necessary for this prototype; and also we won't be able to do it when we have different languages in the DHT & UI anyway ;)