Open jorabin opened 12 months ago
Git submodules have serious problems. We should discuss alternatives.
On Wed, Nov 22, 2023 at 6:16 AM Jo @.***> wrote:
Continuing from #2 https://github.com/derecalliance/cryptography/issues/2, and we don't have somewhere for "overall architectural issues", so having more general conversation here.
Request review from team members especially @lbaird https://github.com/lbaird and @rsinha https://github.com/rsinha . Derec Language Bindings
For each language binding (tbd as to which will be created and maintained) there are several artefacts required for the binding:
- the API interface itself
- protobufs compiled to the language specific interface
- this implementation of Shamir sharing
propose that these (and others that may arise) be treated together in a way that is appropriate to the language, so specifically: For Java
derec-java-api repo will contain a Maven multi-module Java project containing the language specific bindings for the various artefacts and will package them appropriately as JARs for eventual distribution via Maven under the groupid org.derecalliance.derec with artifact ids: derec-api, derec-protobuf and derec-shamir. Extract Interfaces
In the course of this process, the SplitterFactory and Spiltter interfaces should be moved into the api interface, as well as being enhanced with error handling etc. Git Submodule strategy
Earlier discussion relates to how resources such as protobuf and the rust source here can be shared to a repo and multimodule project as discussed here. Reservations were expressed about using submodules. Subsequent experiment shows that this appears to work well, providing changes are only made in the source repo. i.e. don't make changes in the modules into which the submodules are included. Earlier mention of Git subtrees does not seem to work well. Packaging of binaries
derec-shamir is to contain the binaries for aarch64 and x86_64 at least. (It's my understanding that the implementation dependencies are "pure rust" but if this is not the case a) can it be made so and b) if not, then os variants will be needed)
Those that do not wish to use the pre-built artefact should be free to rebuild using the (Maven) build provided in the repo.
— Reply to this email directly, view it on GitHub https://github.com/derecalliance/cryptography/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDBYW44KOGYRRDPH6F3QX3YFXUJDAVCNFSM6AAAAAA7WEJCW2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGAYDMMRRG4YTOOA . You are receiving this because you were mentioned.Message ID: @.***>
Please can we identify the nature of those problems and what is required to avoid them. I believe that by changing only the source repo we should be in good shape.
Please let's also identify alternatives. As mentioned subtrees don't seem right and surely we must be able to do better than copy and paste from source repo.
Agreed at meeting of 2023-12-05 to use Git Submodules, where necessary. Awaiting further review of this issues as to the strategy regarding collecting together language bindings for protobufs, crypto and the api itself.
Continuing from #2, and we don't have somewhere for "overall architectural issues", so having more general conversation here.
Request review from team members especially @lbaird and @rsinha .
Derec Language Bindings
For each language binding (tbd as to which will be created and maintained) there are several artefacts required for the binding:
propose that these (and others that may arise) be treated together in a way that is appropriate to the language, so specifically:
For Java
derec-java-api repo will contain a Maven multi-module Java project containing the language specific bindings for the various artefacts and will package them appropriately as JARs for eventual distribution via Maven under the groupid
org.derecalliance.derec
with artifact ids:derec-api
,derec-protobuf
andderec-shamir
.Extract Interfaces
In the course of this process, the
SplitterFactory
andSpiltter
interfaces should be moved into the api interface, as well as being enhanced with error handling etc.Git Submodule strategy
Earlier discussion relates to how resources such as protobuf and the rust source here can be shared to a repo and multimodule project as discussed here. Reservations were expressed about using submodules. Subsequent experiment shows that this appears to work well, providing changes are only made in the source repo. i.e. don't make changes in the modules into which the submodules are included. Earlier mention of Git subtrees does not seem to work well.
Packaging of binaries
derec-shamir
is to contain the binaries foraarch64
andx86_64
at least. (It's my understanding that the implementation dependencies are "pure rust" but if this is not the case a) can it be made so and b) if not, then os variants will be needed)Those that do not wish to use the pre-built artefact should be free to rebuild using the (Maven) build provided in the repo.