IBM / aspera-cli

IBM Aspera CLI
Apache License 2.0
77 stars 17 forks source link

ascli is adding multiple packages to source path on downloads (receive) #131

Closed dremerten closed 1 year ago

dremerten commented 1 year ago

I'm running into an issue with ascli and downloading ALL packages from a Shared_inbox Faspex5 version: 5.0.6.1GA

1. I'm sending 10 Packages to a shared_inbox (linux_single) with expiration policy anyone. 2. Next I'm wanting to download ALL package from the shared inbox via ascli with this command:

ascli faspex5 packages receive ALL --once-only=yes --box=$SHARED_INBOX  --to-folder=./automated_downloaded_packages --log-level=debug --insecure=yes > $FILE

RESULT It successfully downloads 2 packages and the remaining 8 fail with

2023-11-17 10:13:45.637 [307e-7fafea706740] ERR No such file or directory
2023-11-17 10:13:45.637 [307e-7fafea706740] ERR Source path does not exist.
2023-11-17 10:13:45.638 [307e-7fafea706740] ERR FASP Session Error id=bad893ce-0052-4a18-914f-a2e11cf1a29c (error 63, Source path stat failed)
2023-11-17 10:13:45.644 [307e-7fafe8bcd700] LOG FASP Session Start uuid=bad893ce-0052-4a18-914f-a2e11cf1a29c op=send status=failed errcode=63 errstr="Source path stat failed" source=/5.0.6.2 Package Performance Test 2023-11-17-15687 - f92d705b-bf99-43c6-9569-8eccff6bb5bd.aspera-package/PKG - 5.0.6.2 Package Performance Test 2023-11-17-15687/5.0.6.2 Package Performance Test 2023-11-17-12200 - 4120fbf1-231d-45d2-9e29-14265562de9f.aspera-package/PKG - 5.0.6.2 Package Performance Test 2023-11-17-12200 dest=automated_downloaded_packages source_prefix=- local=9.30.217.193:33001 peer=9.46.89.158:41180 tcp_port=33001 os="Linux 3.10.0-1160.95.1.el7.x86_64 #1 SMP" ver=4.4.3.891 ecd7192 lic=10:0:50236 peeros="Linux 5.15.0-88-generic #98-Ubuntu SMP M" peerver=4.4.2.550 c4b4270 peerlic=53:1:53 proto_sess=20003 proto_udp=20000 proto_bwmeas=20000 proto_data=2000f

ISSUE: Here is the comparison between ascli's download transfer_spec(FAILS) VS connect's download transfer_spec (SUCCEEDS)

ASCLI

"paths": [
    {
      "source": "/Performance Test d945f746-0652-4a67-ad8d-3ad4168af463-2023-11-17 - 4e8f6eeb-ee62-4ace-af74-25b2e1dbe8f9.aspera-package/PKG - Performance Test d945f746-0652-4a67-ad8d-3ad4168af463-2023-11-17/Performance Test 7e9823a9-c68e-44d9-bc93-06a927ecc532-2023-11-17 - 743bc2da-94bd-4878-840c-e83c2f6729f5.aspera-package/PKG - Performance Test 7e9823a9-c68e-44d9-bc93-06a927ecc532-2023-11-17"
    }
  ]

CONNECT

    "paths": [
        {
            "destination": "",
            "source": "/Performance Test d945f746-0652-4a67-ad8d-3ad4168af463-2023-11-17 - 4e8f6eeb-ee62-4ace-af74-25b2e1dbe8f9.aspera-package/PKG - Performance Test d945f746-0652-4a67-ad8d-3ad4168af463-2023-11-17/"
        },
        {
            "is_truncated_for_logging": true,
            "reason": "to avoid logging too much when a huge number of files is selected"
        }
    ]
dremerten commented 1 year ago

The Gotcha is if I repeat the ascli cmd a total of 5times ( successfully downloading 2 Packages at a time). Then I can download all 10 Packages.

laurent-martin commented 1 year ago

You win a prize for finding a bug, thank you...

beta with fix: https://ibm.biz/aspera-cli-beta

The problem was that a variable got changed in the loop, and the "source", got erroneously the path of 2 packages concatenated...

dremerten commented 1 year ago

Nice! Thank you for your speedy fix!

Tested with dev build

ascli --version
4.15.0.pre
[
  {
    "package": "6416",
    "status": "success"
  },
  {
    "package": "6415",
    "status": "success"
  },
  {
    "package": "6414",
    "status": "success"
  },
  {
    "package": "6413",
    "status": "success"
  },
  {
    "package": "6412",
    "status": "success"
  },
  {
    "package": "6411",
    "status": "success"
  },
  {
    "package": "6410",
    "status": "success"
  },
  {
    "package": "6409",
    "status": "success"
  },
  {
    "package": "6408",
    "status": "success"
  },
  {
    "package": "6407",
    "status": "success"
  }
]