Constellation-Labs / euclid-development-environment

Quickly spin up basic dev environments to build local metagraph projects with our extendable base.
Apache License 2.0
4 stars 13 forks source link

Building with custom p12 files is only copying "P12_GENESIS_FILE_NAME" #7

Closed Proph151Music closed 1 year ago

Proph151Music commented 1 year ago

I was doing a build with my own custom p12 files. I used the cl-wallet.jar to create 3 different p12 files. I placed the 3 new p12 files in "source/p12-files" and removed the default p12 files.

I edited the .env file with all of the correct information.

I thought everything went fine. I did the hydra install and hydra build with no issue (that I knew). But when I did the hydra start_genesis it loaded all the images... but when it got to, "Joining currency l1 containers to build the cluster", it never was able to join them. No error or anything it just went through all of the tries (it actually tried 11/10 times). Then it moved on through the rest of the process.

I noticed 2 of the cluster node urls failed. 9300 and 9400 failed to load.

So... I decided to try building each separately.

I ran --only currency-l1 and I noticed it would copy "techware-key.p12" which in my .env file is "P12_GENESIS_FILE_NAME"

I ran --only currency-l0 and it copied the same file "techware-key.p12".

I ran --only global-l0 and it copied the same file "techware-key.p12".

It never tried to copy my other p12 files. Only the "P12_GENESIS_FILE_NAME".

Proph151Music commented 1 year ago

Here is the relevant portion of my .env file...


P12_GENESIS_FILE_KEY_ALIAS=techware-key
P12_GENESIS_FILE_PASSWORD=********

P12_NODE_2_FILE_NAME=techware-key-1.p12
P12_NODE_2_FILE_KEY_ALIAS=techware-key-1
P12_NODE_2_FILE_PASSWORD=********

P12_NODE_3_FILE_NAME=techware-key-2.p12
P12_NODE_3_FILE_KEY_ALIAS=techware-key-2
P12_NODE_3_FILE_PASSWORD=********```
Proph151Music commented 1 year ago

Here is the build output from when I performed the build. Notice it only ever copies techware-key.p12.


[+] Building 12.2s (8/8) FINISHED                                               
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 1.02kB                                     0.0s
 => [internal] load .dockerignore                                          0.1s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/library/ubuntu-with-java-and-s  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 77B                                           0.0s
 => CACHED [1/3] FROM docker.io/library/ubuntu-with-java-and-sbt           0.0s
 => [2/3] COPY p12-files/techware-key.p12 techware-key.p12                 0.1s
 => [3/3] RUN apt-get install wget -y &&     echo "Getting global-l0 jar  11.0s
 => exporting to image                                                     1.0s
 => => exporting layers                                                    0.9s
 => => writing image sha256:3f20112fbc5f17a09cbd619fca7217e59db7976f2edca  0.0s 
 => => naming to docker.io/library/global-l0                               0.0s
 Building Currency L0 image... (NO CACHE)                                        
[+] Building 47.2s (10/10) FINISHED                                             
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 686B                                       0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/library/ubuntu-with-java-and-s  0.0s
 => CACHED [1/5] FROM docker.io/library/ubuntu-with-java-and-sbt           0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 18.58kB                                       0.0s
 => [2/5] COPY project/techware-metagraph codebase/techware-metagraph      0.1s
 => [3/5] COPY p12-files/techware-key.p12 techware-key.p12                 0.1s
 => [4/5] RUN cd codebase/techware-metagraph  &&     sbt currencyL0/asse  42.0s
 => [5/5] RUN mv codebase/techware-metagraph/modules/l0/target/scala-2.13  0.8s
 => exporting to image                                                     4.0s 
 => => exporting layers                                                    4.0s 
 => => writing image sha256:3e7a18a6c5fbcc01d67a4de222c24e264be798b04e335  0.0s 
 => => naming to docker.io/library/currency-l0                             0.0s 
Global L0 image built
Currency L0 image built                                                         
Building Currency L1 image... (NO CACHE)
[+] Building 48.1s (10/10) FINISHED                                             
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load build definition from Dockerfile                       0.1s
 => => transferring dockerfile: 749B                                       0.0s
 => [internal] load metadata for docker.io/library/ubuntu-with-java-and-s  0.0s
 => CACHED [1/5] FROM docker.io/library/ubuntu-with-java-and-sbt           0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 1.91kB                                        0.0s
 => [2/5] COPY project/techware-metagraph codebase/techware-metagraph      0.1s
 => [3/5] COPY p12-files/techware-key.p12 techware-key.p12                 1.0s
 => [4/5] RUN cd codebase/techware-metagraph &&     sbt currencyL1/assem  41.7s
 => [5/5] RUN mv codebase/techware-metagraph/modules/l1/target/scala-2.13  0.8s
 => exporting to image                                                     4.3s 
 => => exporting layers                                                    4.3s 
 => => writing image sha256:5776c6707e0e92916c03fd20b7f90a1248ce76ff4893a  0.0s 
 => => naming to docker.io/library/currency-l1-initial-validator           0.0s```
Proph151Music commented 1 year ago

Using the most recent develop branch still has an issue when using custom p12 files. Building with the default P12 files doesn't seem to have any issues.



Docker Compose v2 found. Using command: docker compose

Creating docker custom-network...

Network created

Starting global-l0 image ...

[+] Running 1/1

 ✔ Container l0-global  Started                                                                                                                                                                       0.7s 

global-l0 image started

global-l0 still booting... waiting 30s (1/10)

global-l0 still booting... waiting 30s (2/10)

global-l0 still booting... waiting 30s (3/10)

global-l0 still booting... waiting 30s (4/10)

global-l0 still booting... waiting 30s (5/10)

global-l0 still booting... waiting 30s (6/10)

global-l0 still booting... waiting 30s (7/10)

global-l0 still booting... waiting 30s (8/10)

global-l0 still booting... waiting 30s (9/10)

Could not find the global-l0 instance, make sure to run the global-l0 container```
IPadawans commented 1 year ago

Hey @Proph151Music, so I've just tested with custom p12 files (different from the default ones) and it works smoothly. Can you please run again and then run this docker logs --follow l0-global in another terminal while the global-l0 is starting? Maybe we are having another issue that we need to check. Be sure to build again the global-l0 before running.

Proph151Music commented 1 year ago

What is the proper way to create custom P12 files for the SDK? I have tried 2 different ways but I have the same issue either way. This log all I did was edit the .env with the details I wanted to use for all 3 p12 files and renamed the default p12 files in the source/p12-files directory.

docker logs --follow l0-global

21:53:44.059 [io-compute-blocker-1] INFO o.t.s.a.TessellationIOApp - App environment: Dev

21:53:44.062 [io-compute-blocker-1] INFO o.t.s.a.TessellationIOApp - App version: 2.0.0-alpha.2

java.io.IOException: PKCS12 key store mac invalid - wrong password or corrupted file.

at org.bouncycastle.jcajce.provider.keystore.pkcs12.PKCS12KeyStoreSpi.engineLoad(Unknown Source)

at org.bouncycastle.jcajce.provider.keystore.util.AdaptingKeyStoreSpi.engineLoad(Unknown Source)

at java.base/java.security.KeyStore.load(KeyStore.java:1479)

at org.tessellation.keytool.KeyStoreUtils$.$anonfun$unlockKeyStore$3(KeyStoreUtils.scala:140)

at delay @ org.tessellation.keytool.KeyStoreUtils$.$anonfun$unlockKeyStore$2(KeyStoreUtils.scala:140)

at as @ com.monovore.decline.effect.CommandIOApp$.addVersionFlag(CommandIOApp.scala:67)

at delay @ org.tessellation.keytool.KeyStoreUtils$.unlockKeyStore(KeyStoreUtils.scala:139)

at flatTap @ org.tessellation.keytool.KeyStoreUtils$.unlockKeyStore(KeyStoreUtils.scala:139)

at delay @ org.tessellation.keytool.KeyStoreUtils$.reader(KeyStoreUtils.scala:115)

at main$ @ com.monovore.decline.effect.CommandIOApp.main(CommandIOApp.scala:12)

at main$ @ com.monovore.decline.effect.CommandIOApp.main(CommandIOApp.scala:12)

at main$ @ com.monovore.decline.effect.CommandIOApp.main(CommandIOApp.scala:12)

at flatMap @ org.tessellation.sdk.app.TessellationIOApp.$anonfun$main$8(TessellationIOApp.scala:84)
Proph151Music commented 1 year ago

The other method I tried was creating my own p12 files by exporting the details: export CL_KEYSTORE="techware-genesis.p12" export CL_KEYALIAS="techware-genesis" export CL_PASSWORD="MyPassword"

and then running this command: java -jar cl-keytool.jar generate

Repeating for the other 2 files.

Then I edit the .env and overwrite the files in source/p12-files.

Both methods end with the same results. So maybe I am doing something wrong?

marcinwadon commented 1 year ago

hey @Proph151Music, what's your java version?

Proph151Music commented 1 year ago

java -version

openjdk version "1.8.0_292"

OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10)

OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode)

techware@techware-box:~/euclid-development-environment$ npm -v

9.6.4

marcinwadon commented 1 year ago

thanks. this one and java 11 are supported, so that's not a case here :+1:

Proph151Music commented 1 year ago

Could it be because I use a $ in my password?

marcinwadon commented 1 year ago

@Proph151Music try without, you'll see

Proph151Music commented 1 year ago

Will make a new build now and report back...

Proph151Music commented 1 year ago

docker logs --follow l0-global

I have the same issue without the $ in the password.

Here is my .ENV file (privacy parts edited)

GITHUBTOKEN=ghp****

PROJECT_NAME=techware-metagraph

DEFAULT_TESSELLATION_PROJECT_VERSION=v2.0.0-alpha.2 DEFAULT_TESSELLATION_PROJECT_VERSION_TEMPLATE=2.0.0-alpha.2

P12_GENESIS_FILE_NAME=techware-gen.p12 P12_GENESIS_FILE_KEY_ALIAS=techware-gen P12_GENESIS_FILE_PASSWORD=****

P12_NODE_2_FILE_NAME=techware-node2.p12 P12_NODE_2_FILE_KEY_ALIAS=techware-node2 P12_NODE_2_FILE_PASSWORD=****

P12_NODE_3_FILE_NAME=techware-node3.p12 P12_NODE_3_FILE_KEY_ALIAS=techware-node3 P12_NODE_3_FILE_PASSWORD=****

Proph151Music commented 1 year ago

Oh man guys... I'm so sorry. This was user error! :face_with_spiral_eyes: My dumbass didn't realize I was copy/pasting an extra character into the export password field. So it was different then what I was putting in my .env file! LMAO

All I can do is laugh at myself. LOL