search

IBM / CAST

CAST can enhance the system management of cluster-wide resources. It consists of the open source tools: cluster system management (CSM) and burst buffer.
Eclipse Public License 1.0
27 stars 34 forks source link

How do you actually copy a file with `bbcmd copy`? #999

Open tonyhutter opened 3 years ago

tonyhutter commented 3 years ago

Question:

I'm unable to find any good examples on how to use bbcmd copy to do a simple copy from the burst buffer to GPFS. Please tell me what I'm doing wrong:

# I'm on a one node allocation with a 1GB burst buffer.

$ GPFSDIR=<my GPFS home dir>
$ dd if=/dev/zero of=$BBPATH/bigfile bs=100M count=9 > /dev/null
$ echo -e "$BBPATH/bigfile\t$GPFSDIR/bigfile\n" > filelist
$ bbcmd copy --pretty --contrib=0 --tag=42 --filelist=`pwd`/filelist
...
butte19:bb_api165141.breadcrumbs.bbproxy.msgin_starttransfer.exit.count                         1
butte19:bb_api165141.breadcrumbs.bbproxy.msgin_starttransfer.exit.ts                            1615409380.902909
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.bbserver.msgin_gettransferhandle.exit.count  1
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.bbserver.msgin_gettransferhandle.exit.ts     1615409380.899144
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.getHandle.entry.count                   1
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.getHandle.entry.ts                      1615409380.891056
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.getHandle.exit.count                    1
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.getHandle.exit.ts                       1615409380.899025
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.lockLocalMetadataIfNeeded.entry.count   5
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.lockLocalMetadataIfNeeded.entry.ts      1615409380.891114
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.lockLocalMetadataIfNeeded.exit.count    5
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.lockLocalMetadataIfNeeded.exit.ts       1615409380.897499
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTagInfo.entry.count                1
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTagInfo.entry.ts                   1615409380.891742
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTagInfo.exit.count                 1
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTagInfo.exit.ts                    1615409380.898919
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTransfer.entry.count               1
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTransfer.entry.ts                  1615409380.891375
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTransfer.exit.count                1
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.queueTransfer.exit.ts                   1615409380.898974
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.transferQueueIsLocked.entry.count       13
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.transferQueueIsLocked.entry.ts          1615409380.889739
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.transferQueueIsLocked.exit.count        13
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.transferQueueIsLocked.exit.ts           1615409380.897567
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.unlockTransferQueueIfNeeded.entry.count 5
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.unlockTransferQueueIfNeeded.entry.ts    1615409380.891497
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.unlockTransferQueueIfNeeded.exit.count  5
butte2-vm1:bb_proxy_6 (192_168_128_19).breadcrumbs.xfer.unlockTransferQueueIfNeeded.exit.ts     1615409380.897364
env.contribid                                                                                   999999998
env.jobid                                                                                       134713
env.jobstepid                                                                                   1
error.func                                                                                      validateContribId
error.line                                                                                      36
error.sourcefile                                                                                /u/tgooding/workspace/castbuild/bb/src/bbconndata.cc
error.text                                                                                      Invalid contribid value.  Value is 999999998
id                                                                                              1
in.apicall                                                                                      BB_StartTransfer
out.handle                                                                                      4294969986
out.lvuuid                                                                                      53d0a6e1-dccd-490e-824b-1d1cc0940d68
out.transferHandle                                                                              4294969986
rc                                                                                              -1
# Version
$ rpm -q --whatprovides `which bbcmd`
ibm-burstbuffer-1.7.2-3094.ppc64le

Answer:

place final answer here

Approach:

here replace this with a short summary of how you addressed the problem. in the comments place step by step notes of progress as you go

What is next:

Define the next steps and follow up here