bcgov / von-network

A portable development level Indy Node network.
Apache License 2.0
163 stars 188 forks source link

Error Initializing open pool ledger #278

Closed PenguinTaro closed 1 year ago

PenguinTaro commented 1 year ago

Hi @WadeBarnes , can you suggest the solution for the same. MY application was running well for 5 months but unfortunately my docker containers stop responding last week and hence my application servers got down. When I restart the von-network , i can't establish the open pool connection and my nodes aren't communicating with each other. Attaching the error logs. error_logs.txt

WadeBarnes commented 1 year ago

Thanks @PenguinTaro. What version of von-network are you using? I would recommend using the latest code from the main branch. There have been some enhancements made to help with various startup issues.

I had a quick look through the logs but I haven't identified anything completely obvious just yet.

How do the issues you're encountering manifest themselves? What's displayed in the ledger browser? What's the output of docker ps -a.

When you say you're application has been running for 5 months ... I hope you're not using von-network as a production ledger, it was design as a provisional ledger for development and test purposes only.

PenguinTaro commented 1 year ago

@WadeBarnes , Thank you for looking into it. I don't remember which version it is but for sure it's not the latest one.
The issue happen when I restarted the von-network for the re-setup of my whole application. The display in the ledger says : Can't initialize the open pool ledger , when i do docker ps -a, I can see all the containers up and running. I have been using the von-network as production ledger only. If this isn't for the production can you suggest me the right way to use the ledger for production?

WadeBarnes commented 1 year ago

Try upgrading to the latest code to get you going.

WadeBarnes commented 1 year ago

Documents for creating a new production level indy-network can be found here; https://github.com/hyperledger/indy-node/pull/1814. Setup docs for various platforms can be found here; https://github.com/hyperledger/indy-node/pull/1820

PenguinTaro commented 1 year ago

@WadeBarnes Okay, I will look into what you have shared.

I tried to delete the image and run the ./manage build and get this the following error.

bcadmin@100:~/von-network$ ./manage build

Step 1/9 : FROM bcgovimages/von-image:node-1.12-4
 ---> a7a078ab7f6e
Step 2/9 : ENV LOG_LEVEL ${LOG_LEVEL:-info}
 ---> [Warning] IPv4 forwarding is disabled. Networking will not work.
 ---> Running in ab6e70a01c54
Removing intermediate container ab6e70a01c54
 ---> ff3651ad4658
Step 3/9 : ENV RUST_LOG ${RUST_LOG:-warning}
 ---> [Warning] IPv4 forwarding is disabled. Networking will not work.
 ---> Running in 9ab086d0f92c
Removing intermediate container 9ab086d0f92c
 ---> 6da3638da9b5
Step 4/9 : ADD config ./config
 ---> 2ef5ffe3abe2
Step 5/9 : ADD server/requirements.txt server/
 ---> d2580dc19d29
Step 6/9 : RUN pip install --no-cache-dir -r server/requirements.txt
 ---> [Warning] IPv4 forwarding is disabled. Networking will not work.
 ---> Running in 9ca86c19fb82
Collecting pyyaml~=5.1.1 (from -r server/requirements.txt (line 1))
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8c649d6e10>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pyyaml/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8c649d6fd0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pyyaml/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8c649d6198>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pyyaml/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8c649d6208>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pyyaml/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8c649d64e0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pyyaml/
  Could not find a version that satisfies the requirement pyyaml~=5.1.1 (from -r server/requirements.txt (line 1)) (from versions: )
No matching distribution found for pyyaml~=5.1.1 (from -r server/requirements.txt (line 1))
The command '/bin/sh -c pip install --no-cache-dir -r server/requirements.txt' returned a non-zero code: 1

Can you look into and help me through?

WadeBarnes commented 1 year ago

@PenguinTaro, I'm wondering if this is/was an intermittent issue fetching the package. The build is working for me on both Windows 11 (using Git Bash and WSL) and MAC M2 w/ Sonoma 14.0. Could you try again?

What OS are you running?

Pro Tip:

WadeBarnes commented 1 year ago

Ah, wondering if this is the issue with both your build and with the nodes not connecting; IPv4 forwarding is disabled. Networking will not work.. Similar issue here https://github.com/bcgov/von-network/issues/280

WadeBarnes commented 1 year ago

Also based in the build output, it appears you're not building with the latest code either.

I expect the line RUN pip install --no-cache-dir -r server/requirements.txt to look like RUN pip3 install -U pip && pip install --no-cache-dir -r server/requirements.txt && python -m pip install pip==9.0.3

PenguinTaro commented 1 year ago

I found the issue and turned out that IPv4 was set false somehow and hence it was disturbing the communication btw the nodes. So, i check my IPv4 status using : sysctl net.ipv4.ip_forward . I enabled it by changing net.ipv4.ip_forward = 0 to 1 and it it started working.

Thank you for helping @WadeBarnes .