Problem getting submodules #34

Open hamedgibago opened 7 months ago

Hello, I want to run your source code, but when in part I want to submodule your repository from GitLab with this code:

cd tool && git submodule update --init --recursive && npm install

I get this error for cloning submodules from your repositories:

Cloning into '/home/hamed/Delphi-BFT/themis-gosig'... fatal: unable to access 'https://gitlab.sec.uni-passau.de/cb/themis.git/': server certificate verification failed. CAfile: none CRLfile: none fatal: clone of 'https://gitlab.sec.uni-passau.de/cb/themis.git' into submodule path '/home/hamed/Delphi-BFT/themis-gosig' failed Failed to clone 'themis-gosig'. Retry scheduled Cloning into '/home/hamed/Delphi-BFT/themis-mir'... fatal: unable to access 'https://gitlab.sec.uni-passau.de/cb/themis.git/': server certificate verification failed. CAfile: none CRLfile: none fatal: clone of 'https://gitlab.sec.uni-passau.de/cb/themis.git' into submodule path '/home/hamed/Delphi-BFT/themis-mir' failed Failed to clone 'themis-mir'. Retry scheduled Cloning into '/home/hamed/Delphi-BFT/themis-gosig'... fatal: unable to access 'https://gitlab.sec.uni-passau.de/cb/themis.git/': server certificate verification failed. CAfile: none CRLfile: none fatal: clone of 'https://gitlab.sec.uni-passau.de/cb/themis.git' into submodule path '/home/hamed/Delphi-BFT/themis-gosig' failed Failed to clone 'themis-gosig' a second time, aborting

Is it possible to download some of them that are in github instead those which are in gitlab? For example change the submodule file and remove them in gitlab? I have a doubt that this one is required or not?

https://gitlab.sec.uni-passau.de/cb/library.git

Should be solved through commit:

https://github.com/Delphi-BFT/tool/commit/3c50d9eebfda22a22e3e235ffb05ae19a65e1baa

Thank you but after pulling new request getting new error. I don't know why it is looking for themis-gosig although url and folder is deteled?

fatal: No url found for submodule path 'themis-gosig' in .gitmodules

Can you try again?

Thank you. It worked but when I want to pull request and cd into libhotstuff directory, it is empty. So after executing this command:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED=ON -DHOTSTUFF_PROTO_LOG=ON

I received this error:

CMake Warning: No source or binary directory provided. Both will be assumed to be the same as the current working directory, but note that this warning will become a fatal error in future CMake releases.

CMake Error: The source directory "/home/hamed/Delphi-BFT/libhotstuff" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI.

Thank you. It worked but when I want to pull request and cd into libhotstuff directory, it is empty. So after executing this command:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED=ON -DHOTSTUFF_PROTO_LOG=ON

I received this error:

CMake Warning: No source or binary directory provided. Both will be assumed to be the same as the current working directory, but note that this warning will become a fatal error in future CMake releases.

CMake Error: The source directory "/home/hamed/Delphi-BFT/libhotstuff" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI.

I called git pull --force and directory was pulled. Thank you. If any problem I will ask again.

Another problem is in spite of git clean -f and git reset --hard HEAD and git pull --forceand running build command, mentioned in previous comment, folder libhotstuff/salticidae is empty. The error is:


  The source directory

    /home/hamed/Delphi-BFT/libhotstuff/salticidae

  does not contain a CMakeLists.txt file.

CMake Error at /usr/share/cmake-3.22/Modules/ExternalProject.cmake:2882 (message):
  No download info given for 'libsecp256k1' and its source directory:

   /home/hamed/Delphi-BFT/libhotstuff/secp256k1

  is not an existing non-empty directory.  Please specify one of:

   * SOURCE_DIR with an existing non-empty directory
   * DOWNLOAD_COMMAND
   * URL
   * GIT_REPOSITORY
   * SVN_REPOSITORY
   * HG_REPOSITORY
   * CVS_REPOSITORY and CVS_MODULE
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/ExternalProject.cmake:3716 (_ep_add_download_command)
  CMakeLists.txt:30 (ExternalProject_Add)```

Another problem is in spite of git clean -f and git reset --hard HEAD and git pull --forceand running build command, mentioned in previous comment, folder libhotstuff/salticidae is empty. The error is:

  The source directory

    /home/hamed/Delphi-BFT/libhotstuff/salticidae

  does not contain a CMakeLists.txt file.

CMake Error at /usr/share/cmake-3.22/Modules/ExternalProject.cmake:2882 (message):
  No download info given for 'libsecp256k1' and its source directory:

   /home/hamed/Delphi-BFT/libhotstuff/secp256k1

  is not an existing non-empty directory.  Please specify one of:

   * SOURCE_DIR with an existing non-empty directory
   * DOWNLOAD_COMMAND
   * URL
   * GIT_REPOSITORY
   * SVN_REPOSITORY
   * HG_REPOSITORY
   * CVS_REPOSITORY and CVS_MODULE
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/ExternalProject.cmake:3716 (_ep_add_download_command)
  CMakeLists.txt:30 (ExternalProject_Add)```

I found my problem. I just called this code: cd tool && git submodule update --init --recursive && npm install with --force parameter.

Thanks for testing. If there are more problems with pulling the dependencies, feel free to update this issue

Thank you. Sure 🙏

I finnaly managed to run experiments, but during running simulation several errors happened in C++ codes such as:

2024-02-20T16:52:01.817Z [error] : childprocess: In file included from /home/hamed/Delphi-BFT/libhotstuff/salticidae/include/salticidae/network.h:28, from /home/hamed/Delphi-BFT/libhotstuff/examples/hotstuff_client.cpp:26: /home/hamed/Delphi-BFT/libhotstuff/salticidae/include/salticidae/event.h: In member function ‘void salticidae::NotifyFd::notify()’: /home/hamed/Delphi-BFT/libhotstuff/salticidae/include/salticidae/event.h:560:26: warning: ignoring return value of ‘ssize_t write(int, const void*, size_t)’ declared with attribute ‘warn_unused_result’ [-Wunused-result] 560 | void notify() { write(fd, &dummy, 8); } |

0r

2024-02-20T16:51:57.169Z [error] : childprocess: In file included from /home/hamed/Delphi-BFT/libhotstuff/salticidae/include/salticidae/stream.h:31, from /home/hamed/Delphi-BFT/libhotstuff/salticidae/include/salticidae/netaddr.h:29, from /home/hamed/Delphi-BFT/libhotstuff/examples/hotstuff_client.cpp:25: /home/hamed/Delphi-BFT/libhotstuff/salticidae/include/salticidae/crypto.h: In member function ‘void salticidae::SHA256::reset()’: /home/hamed/Delphi-BFT/libhotstuff/salticidae/include/salticidae/crypto.h:45:25: warning: ‘int SHA256_Init(SHA256_CTX*)’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations] 45 | if (!SHA256_Init(&ctx)) | ~~~~~~~~~~~^~~~~~

?

Hi,

Can you provide the experimental discription file that you used?

There seems to be a problem within libhotstuff, and its use of salticidae. Maybe can you also post information on your operating system and the versions of installed dependencies such as CMake, C++, libuv and openssl for a comparison.

Aditionally, maybe @bentoumia knows more on this.

My Os is Ubuntu 22.04 LTS. CMake version 3.22.1 OpenSSL 3.0.2 15

and this is my experimental discription file:

#GLOBAL
RESOURCE_MONITOR_INTERVAL=2000
SHADOW_FILE=shadow.yaml
NETWORK_FILE=network.gml
SHADOW_PROCESS=shadow
STATS_FILE=results.csv
#SHADOW_TEMPLATE="DEFINE A TEMPLATE HERE"
LOG_LEVEL=debug
# HotStuff Stuff

HOTSTUFF_DIR=/home/hamed/Delphi-BFT/libhotstuff/
HOTSTUFF_EXECUTION_DIR=/home/hamed/Delphi-BFT/libhotstuff/
HOTSTUFF_EXPERIMENTS_OUTPUT_DIR=/home/hamed/Delphi-BFT/Experiments/myHotStuffExperiments
HOTSTUFF_REPLICA_HOST_PREFIX=hotStuffReplica
HOTSTUFF_CLIENT_HOST_PREFIX=hotStuffClient

# THE FOLLOWING PATHS MUST BE RELATIVE TO YOUR HOTSTUFF DIRECTORY

HOTSTUFF_REPLICAS_FILE=scripts/deploy/replicas.txt
HOTSTUFF_CLIENTS_FILE=scripts/deploy/clients.txt
HOTSTUFF_GENSCRIPT_WORKING_DIR=scripts/deploy
HOTSTUFF_REPLICA_BIN=examples/hotstuff-app
HOTSTUFF_CLIENT_BIN=examples/hotstuff-client
HOTSTUFF_STATS_SCRIPT=scripts/thr_hist.py

# Kauri Stuff

KAURI_DIR=/home/hamed/Delphi-BFT/Kauri-Public/
KAURI_EXECUTION_DIR=/home/hamed/Delphi-BFT/Kauri-Public/
KAURI_EXPERIMENTS_OUTPUT_DIR=/home/hamed/Delphi-BFT/Experiments/myKauriExperiments
KAURI_REPLICA_HOST_PREFIX=kauriReplica
KAURI_CLIENT_HOST_PREFIX=kauriClient

KAURI_REPLICA_BIN=examples/hotstuff-app
KAURI_CLIENT_BIN=examples/hotstuff-client

#BFT-SMaRt Stuff

BFTSMART_DIR=/home/hamed/Delphi-BFT/library/
BFTSMART_EXECUTION_DIR=/home/hamed/Delphi-BFT/library/build/install/library
BFTSMART_EXPERIMENTS_OUTPUT_DIR=/home/hamed/Delphi-BFT/Experiments/myBFT-SMaRtExperiments
BFTSMART_REPLICA_CLASS=bftsmart.demo.microbenchmarks.ThroughputLatencyServer
#BFTSMART_REPLICA_CLASS=bftsmart.benchmark.ThroughputLatencyServer
#BFTSMART_CLIENT_CLASS=bftsmart.demo.microbenchmarks.ThroughputLatencyClientICGDynamic
BFTSMART_CLIENT_CLASS=bftsmart.demo.microbenchmarks.ThroughputLatencyClient
#BFTSMART_CLIENT_CLASS=bftsmart.benchmark.ThroughputLatencyClientICG
#BFTSMART_CLIENT_CLASS=bftsmart.demo.microbenchmarks.AsyncLatencyClient
BFTSMART_REPLICA_HOST_PREFIX=bftSmartReplica
BFTSMART_CLIENT_HOST_PREFIX=bftSmartClient
BFTSMART_JAVA_ARGS=-Djava.security.properties=config/java.security -Dlogback.configurationFile=config/logback.xml -cp lib/*
BFTSMART_REPLICA_PORT=11000
BFTSMART_REPLICA_SECONDARY_PORT=11001
BFTSMART_CLIENT_PORT=11100

# THE FOLLOWING PATHS MUST BE RELATIVE TO YOUR BFTSMART DIRECTORY

BFTSMART_SYSTEM_CONFIG_FILE=build/install/library/config/system.config
BFTSMART_HOSTS_FILE=build/install/library/config/hosts.config
BFTSMART_VIEW_FILE=build/install/library/config/currentView

#Themis Stuff

THEMIS_DIR=/home/hamed/Delphi-BFT/themis
THEMIS_EXECUTION_DIR=/home/hamed/Delphi-BFT/themis
THEMIS_EXPERIMENTS_OUTPUT_DIR=/home/hamed/Delphi-BFT/Experiments/myThemisExperiments
THEMIS_REPLICA_HOST_PREFIX=themisReplica
THEMIS_CLIENT_HOST_PREFIX=themisClient
THEMIS_REPLICA_BIN=target/debug/themis-bench-app
THEMIS_CLIENT_BIN=target/debug/bench-client
THEMIS_PROMETHEUS_PORT=8080
THEMIS_REPLICA_PORT=10003
THEMIS_CLIENT_PORT=10002
THEMIS_KEYS_DIR=keys

# THE FOLLOWING PATHS MUST BE RELATIVE TO YOUR THEMIS DIRECTORY

THEMIS_CONFIG_PATH=config/default
THEMIS_CONFIG_FILE_PATH=config/default/config.toml
THEMIS_PBFT_CONFIG_FILE_PATH=config/default/pbft.toml

GOSIG_DIR=/home/hamed/Delphi-BFT/themis-gosig
GOSIG_EXECUTION_DIR=/home/hamed/Delphi-BFT/themis-gosig
GOSIG_EXPERIMENTS_OUTPUT_DIR=/home/hamed/Delphi-BFT/Experiments/myGosigExperiments
GOSIG_REPLICA_HOST_PREFIX=gosigReplica
GOSIG_CLIENT_HOST_PREFIX=gosigClient
GOSIG_REPLICA_BIN=target/release/themis-bench-app
GOSIG_CLIENT_BIN=target/release/bench-client
GOSIG_PEER_PORT=10003
GOSIG_CLIENT_PORT=10002
GOSIG_GOSSIP_PORT=10004
GOSIG_KEYS_DIR=keys

# THE FOLLOWING PATHS MUST BE RELATIVE TO YOUR GOSIG DIRECTORY

GOSIG_CONFIG_PATH=config/bls/
GOSIG_CONFIG_FILE_PATH=config/bls/gosig.toml
GOSIG_BLS_CONFIG_FILE_PATH=config/bls/bls.toml
GOSIG_GOSSIP_CONFIG_FILE_PATH=config/bls/gossip.toml
GOSIG_PBFT_CONFIG_FILE_PATH=config/bls/pbft.toml

# Narwhal-Tusk Stuff

TUSK_DIR=/home/hamed/Delphi-BFT/narwhal/benchmark
TUSK_KEYS_DIR=benchmark
TUSK_EXECUTION_DIR=/home/hamed/Delphi-BFT/narwhal/benchmark/
TUSK_EXPERIMENTS_OUTPUT_DIR=/home/hamed/Delphi-BFT/Experiments/myNarwhalTuskExperiments
TUSK_REPLICA_HOST_PREFIX=tuskReplica
TUSK_CLIENT_HOST_PREFIX=tuskClient

TUSK_REPLICA_BIN=node
TUSK_CLIENT_BIN=benchmark_client

TUSK_REPLICA_HOST_PREFIX=tuskReplica
TUSK_CLIENT_HOST_PREFIX=tuskClient

Hi,

this is not the experimental description file but your environment file (the .env).

The experimental description file is the input that you provide when starting simulations from the CLI and is specified as YAML file.

Hi,

this is not the experimental description file but your environment file (the .env).

The experimental description file is the input that you provide when starting simulations from the CLI and is specified as YAML file.

Hi, I didn't understand. You mean that I passed a file (description or env) wrong?

Which command did you use to start the simulation?

Which command did you use to start the simulation?

npm run simulation -- examples/hotstuff/hs3-aws.yaml

@bergerch Hello, Is it possible to compile and run the project in windows or some of parts of C++ projects such as libhotstuff is specially written in Linux version?

@hamedgibago unfortunately, no. Windows is not supported.

@bergerch I have troubles compiling java org.bitcoin project under secp256k1. I found that your secp256k1 is an old version an latest one does not have java files. I tried to add latest version of secp256k1 without java or your version with removing java files but I have errors. The errors are that project can not file com.google.common.base.preconditions. I am using vscode under linux and after installing Google Cloud Code also it did not solve the problem. Any idea to use newer version of secp256k1 or removing java files without errors?

Hi, sorry for the late reply.

weird, I do not remember java org.bitcoin being a dependency. Can you provide me a reference where this originates from?

I finnaly manage d to download required jar files and added them in build path of java project, but I found that this version of project using old revision of secp256k1 and in the latest version, java project is removed.

Delphi-BFT / tool

Problem getting submodules #34