search

digital-asset / daml

The Daml smart contract language
https://www.digitalasset.com/developers
797 stars 199 forks source link

daml ledger export script does not work against authenticated SSL ledger #16147

Open dasormeter opened 1 year ago

dasormeter commented 1 year ago

Affected Daml version

Verified to be problematic on DAML 2.4.x and above (may also affect earlier version -- not tested)

Bug description

The daml ledger export script command does not work when run against an authenticated ledger with SSL enabled, even when all required arguments are passed

To reproduce

  1. Create a canton ledger with SSL and ledger authentication enabled -- (this can be done pretty easily by creating a free Daml Hub ledger, which has SSL and ledger authentication enabled by default and allocates a default user aligned to your name - https://hub.daml.com)

  2. Once the ledger is available, follow these instructions to gather all the credentials and tokens needed for speaking to an authenticated ledger over SSL a. hostname for your ledger b. root cert for Let's encrypt c. JWT token for your user d. party name for your user e. output dir where you want the export scripts to be saved

  3. Try to run the daml ledger export script command against your newly created ledger, which will be in the form below (substitute the specific values here for the appropriate ones for your ledger)

    daml ledger export script --host=<your_hostname_without_http_or_https> --port=443 --tls --cacrt=<path_to_isrgrootx1.pem> --party=<your_party_name> --sdk-version=2.4.0 --output=<your_output_directory> --access-token-file=<file_holding_your_jwt_token>

  4. see error

    17:15:06.305 [script-export-akka.actor.default-dispatcher-4] INFO  akka.event.slf4j.Slf4jLogger - Slf4jLogger started
io.grpc.StatusRuntimeException: INTERNAL: http2 exception
17:15:06.647 [script-export-akka.actor.default-dispatcher-13] INFO  akka.actor.CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason]

Expected behavior

When the same command is run directly without going through the daml wrapper it succeeds -- it fails when using the daml wrapper on M1 Mac, Intel Mac and a Unix machine

Command that succeeds

bazel run -- //daml-script/export:export script --host=somehost.somedomain.app --port=443 --tls --cacrt=<path_to_isrgrootx1.pem> --party=<some_party> --sdk-version=2.4.0 --output=<some_output_dir> --access-token-file=<some_file_containing_jwt_token>

INFO: Invocation ID: 50b6db96-6f2c-4ee2-9817-7fb8360900ac
INFO: Analyzed target //daml-script/export:export (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //daml-script/export:export up-to-date:
  bazel-bin/daml-script/export/export.jar
  bazel-bin/daml-script/export/export
INFO: Elapsed time: 0.552s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/daml-script/export/export script '--host=somehost.somedomain.app '--port=443' --tls '
INFO: Build completed successfully, 1 total action
16:43:06.592 [main] INFO  c.d.ledger.api.tls.TlsConfiguration - Client TLS - enabled.
16:43:06.595 [main] INFO  c.d.ledger.api.tls.TlsConfiguration - Client TLS - enabled protocols: TLSv1.2, TLSv1.3.
16:43:06.595 [main] INFO  c.d.ledger.api.tls.TlsConfiguration - Client TLS - enabled cipher suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256.
[INFO] [01/24/2023 16:43:09.829] [script-export-akka.actor.default-dispatcher-6] [CoordinatedShutdown(akka://script-export)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
❯ ls /tmp/blah
Export.daml args.json   daml.yaml   deps
stefanobaghino-da commented 1 year ago

The --sdk-version is different across the two examples. Can you please check this doesn't make a difference? From the error I suspect it might also be beneficial to increase the logging level.

akrmn commented 1 year ago

I had a quick look into this and it looks like the problem is that the daml assistant doesn't forward the ssl configuration to the export jar. In the snippet below, you can see that ledgerFlags only has --host and --port (plus --max-inbound-message-size if present in the args to the assistant) but crucially it lacks --tls, --cacrt and --access-token-file, even though they should be available in the output of getDefaultArgs.

https://github.com/digital-asset/daml/blob/4aa5b7d7685895399aa231917aac5875fe5a46f1/daml-assistant/daml-helper/src/DA/Daml/Helper/Ledger.hs#L539-L552

The fix should be a matter of adding the missing flags in ledgerFlags depending on the result of getDefaultArgs

dasormeter commented 1 year ago

I had a quick look into this and it looks like the problem is that the daml assistant doesn't forward the ssl configuration to the export jar. In the snippet below, you can see that ledgerFlags only has --host and --port (plus --max-inbound-message-size if present in the args to the assistant) but crucially it lacks --tls, --cacrt and --access-token-file, even though they should be available in the output of getDefaultArgs.

https://github.com/digital-asset/daml/blob/4aa5b7d7685895399aa231917aac5875fe5a46f1/daml-assistant/daml-helper/src/DA/Daml/Helper/Ledger.hs#L539-L552

The fix should be a matter of adding the missing flags in ledgerFlags depending on the result of getDefaultArgs

Thanks @akrmn -- why not just pass through all the getDefaultArgs here? is it dropping the other passed through values as well (--access-token-file, --party, --start, --end, --sdk-version, etc?)

❯ daml ledger export script --help
Usage: script-export [script] [options]

  --help                   Show this help message.
  --host <value>           Daml ledger host to connect to.
  --port <value>           Daml ledger port to connect to.
  --pem <value>            TLS: The pem file to be used as the private key.
  --crt <value>            TLS: The crt file to be used as the cert chain.
                                   Required for client authentication.
  --cacrt <value>          TLS: The crt file to be used as the trusted root CA.
  --tls                    TLS: Enable tls. This is redundant if --pem, --crt or --cacrt are set
  --access-token-file <value>
                           File from which the access token will be read, required to interact with an authenticated ledger.
  --party <value>          Export ledger state as seen by these parties. Pass --party multiple times or use a comma-separated list of party names to specify multiple parties. Use either --party or --all-parties but not both.
  --all-parties            Export ledger state as seen by all known parties. Use either --party or --all-parties but not both.
  --start <value>          The transaction offset (exclusive) for the start position of the export. Optional, by default the export includes the beginning of the ledger.
  --end <value>            The transaction offset (inclusive) for the end position of the export. Optional, by default the export includes the current end of the ledger.
  --max-inbound-message-size <value>
                           Optional max inbound message size in bytes. Defaults to 4194304
Command: script [options]
Export ledger state in Daml script format
  --acs-batch-size <value>
                           Batch this many create commands into one transaction when recreating the ACS.
  --set-time <value>       Emit setTime commands to replicate transaction time stamps. Only works on ledgers in static-time mode.
  -o, --output <value>     Create the Daml script under this directory prefix.
  --sdk-version <value>    Specify this Daml version in the generated project.
dasormeter commented 1 year ago

also should we add so that export is shown as one of the available commands when you run daml ledger or do we generally not do this for alpha features?

❯ daml ledger
Missing: COMMAND

Usage: daml ledger COMMAND
  Interact with a remote Daml ledger. You can specify the ledger in daml.yaml
  with the ledger.host and ledger.port options, or you can pass the --host and
  --port flags to each command below. If the ledger is authenticated, you should
  pass the name of the file containing the token using the --access-token-file
  flag or the `daml.access-token-file` field in daml.yaml.

Available options:
  -h,--help                Show this help text

Available commands:
  list-parties             List parties known to ledger
  allocate-parties         Allocate parties on ledger if they don't exist
  upload-dar               Upload DAR file to ledger
  fetch-dar                Fetch DAR from ledger into file
  metering-report          Report on Ledger Use
dasormeter commented 1 year ago

The --sdk-version is different across the two examples. Can you please check this doesn't make a difference? From the error I suspect it might also be beneficial to increase the logging level.

Just copy/pasta mismatch in the examples -- it fails regardless of the version passed. I updated the examples to show 2.4.0 in all cases.

How can the logging level be increased on daml ledger .... commands? Just through logback-fu, you mean?