Closed jjkoehorst closed 2 months ago
/scratch/jasperk/gocmd -c irods_environment.json mkdir /unlock/home/wur.gutbrain_axis/bla
works perfectly its just uploading files?
debug
➜ snellius /scratch/jasperk/gocmd -c irods_environment.json sync ./meta i:/unlock/home/wur.gutbrain_axis/data/ --progress --clear --single_threaded --thread_num 1 --max_file_num 1 --no_hash --no_bulk_reg --clear --debug
DEBU[2024-05-22 08:01:55.620] use sessionID - 847426 function=ProcessCommonFlags package=flag
DEBU[2024-05-22 08:01:55.620] reading config file/dir - /scratch/snellius/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-05-22 08:01:55.620] reading icommands environment file - /scratch/snellius/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-05-22 08:01:55.620] reading environment file /scratch/snellius/irods_environment.json function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-05-22 08:01:55.621] reading environment session file /scratch/snellius/irods_environment.json.cwd function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-05-22 08:01:55.621] reading environment password file /home/jasperk/.irods/.irodsA function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-05-22 08:01:55.621] reading config from environment variables function=LoadAndOverwriteConfigFromEnv package=commons
DEBU[2024-05-22 08:01:55.621] run bput with args: [-c irods_environment.json --diff ./meta i:/unlock/home/wur.gutbrain_axis/data/ --progress --clear --single_threaded --thread_num 1 --max_file_num 1 --no_hash --no_bulk_reg --clear --debug] function=syncFromLocalToIRODS package=subcmd
DEBU[2024-05-22 08:01:55.621] use sessionID - 847426 function=ProcessCommonFlags package=flag
DEBU[2024-05-22 08:01:55.621] reading config file/dir - /scratch/snellius/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-05-22 08:01:55.621] reading icommands environment file - /scratch/snellius/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-05-22 08:01:55.621] reading environment file /scratch/snellius/irods_environment.json function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-05-22 08:01:55.621] reading environment session file /scratch/snellius/irods_environment.json.cwd function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-05-22 08:01:55.621] reading environment password file /home/jasperk/.irods/.irodsA function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-05-22 08:01:55.621] reading config from environment variables function=LoadAndOverwriteConfigFromEnv package=commons
DEBU[2024-05-22 08:01:55.621] clearing local bundle files in /tmp function=CleanUpOldLocalBundles package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.657] Connecting to scomp1789.wurnet.nl:1247 function=Connect package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:55.660] Start up a connection with CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:55.813] Start up CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:55.814] Client policy - CS_NEG_REQUIRE, server policy - CS_NEG_REQUIRE function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:55.814] Start up SSL function=sslStartup package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:55.850] Logging in using pam authentication method function=loginPAMWithToken package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:55.859] Created a new connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.868] failed to commit transaction error=SYS_NO_API_PRIV function=endTransaction package=session struct=IRODSSession
DEBU[2024-05-22 08:01:55.870] Returning a connection function=Return package=session struct=ConnectionPool
INFO[2024-05-22 08:01:55.870] determining staging dir... function=processBputCommand package=subcmd
DEBU[2024-05-22 08:01:55.870] get default staging dir function=processBputCommand package=subcmd
INFO[2024-05-22 08:01:55.870] use staging dir - /unlock/home/wur.gutbrain_axis/data/.gocmd_staging function=processBputCommand package=subcmd
DEBU[2024-05-22 08:01:55.870] clearing irods temp dir /unlock/home/wur.gutbrain_axis/data/.gocmd_staging function=processBputCommand package=subcmd
DEBU[2024-05-22 08:01:55.871] clearing old irods bundle files in /unlock/home/wur.gutbrain_axis/data/.gocmd_staging function=CleanUpOldIRODSBundles package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.871] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.876] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.876] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.888] Returning a connection function=Return package=session struct=ConnectionPool
deleted 0 old irods bundles in /unlock/home/wur.gutbrain_axis/data/.gocmd_staging
DEBU[2024-05-22 08:01:55.888] deleted 0 old irods bundles in /unlock/home/wur.gutbrain_axis/data/.gocmd_staging function=CleanUpOldIRODSBundles package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] bundle-uploading a local directory /scratch/snellius/meta function=bputOne package=subcmd
DEBU[2024-05-22 08:01:55.888] assigned a new bundle 0 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] checking if target /unlock/home/wur.gutbrain_axis/data/meta for source /scratch/snellius/meta exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.888] start input thread function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] start extract thread 1 function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] start extract thread 0 function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] start extract thread 2 function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] start transfer thread 0 function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] start stale file remove and dir create thread function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.888] start bundle thread function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.894] Returning a connection function=Return package=session struct=ConnectionPool
skip adding a dir /scratch/snellius/meta to the bundle. The dir already exists!
DEBU[2024-05-22 08:01:55.894] skip adding a dir /scratch/snellius/meta to the bundle. The dir already exists! function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.897] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.897] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.904] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.904] adding a file /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.904] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.904] assigned a new bundle 1 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.904] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.904] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.910] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.910] adding a file /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.910] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.910] creating a tarball for bundle 0 to /tmp/bundle_e7e6f3c3e6179c20b9bc8b15337b1558.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.910] deleting exising data objects and creating new collections in the bundle 0 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.910] assigned a new bundle 2 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.910] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.910] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.915] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.915] adding a file /scratch/snellius/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.915] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.915] assigned a new bundle 3 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.915] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.915] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.921] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.921] adding a file /scratch/snellius/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.921] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-14697_94133-G07_TTGTATCAGGTGGCCTCTGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.921] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:55.922] assigned a new bundle 4 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.922] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:55.922] Connecting to scomp1789.wurnet.nl:1247 function=Connect package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:55.925] Start up a connection with CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
bundle 0 - Bundling ... 0.0% [..................] [0 in 200ms]
bundle 0 - Cleaning & making dirs ... 0.0% [..................] [0 in 200ms]
DEBU[2024-05-22 08:01:56.108] Start up CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.108] Client policy - CS_NEG_REQUIRE, server policy - CS_NEG_REQUIRE function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.109] Start up SSL function=sslStartup package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.118] Logging in using pam authentication method function=loginPAMWithToken package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.128] Created a new connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.132] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.132] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.142] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.142] adding a file /scratch/snellius/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.142] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.142] assigned a new bundle 5 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.142] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.146] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.146] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.152] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.152] adding a file /scratch/snellius/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.153] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-14708_94133-G08_TCCGGACTAGATTGTTCGTC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.153] deleted exising data objects in the bundle 0 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.153] deleting exising data objects and creating new collections in the bundle 1 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.153] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.153] assigned a new bundle 6 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.153] skip - creating a tarball for bundle 0 to /tmp/bundle_e7e6f3c3e6179c20b9bc8b15337b1558.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.153] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.153] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.159] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.159] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.163] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.163] adding a file /scratch/snellius/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.163] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.163] uploading bundle 0 to /unlock/home/wur.gutbrain_axis/data/.gocmd_staging/bundle_e7e6f3c3e6179c20b9bc8b15337b1558.tar function=processBundleUpload package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.163] creating a tarball for bundle 1 to /tmp/bundle_fe8c3d6a85dd8c6091e1d7679699e285.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.163] assigned a new bundle 7 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.163] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.165] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.165] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.172] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.172] adding a file /scratch/snellius/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.172] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-15071_94133-G09_AGCCTATGATGTAGGTGGTG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.172] assigned a new bundle 8 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.172] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.172] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.178] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.178] adding a file /scratch/snellius/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] deleted exising data objects in the bundle 1 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] deleting exising data objects and creating new collections in the bundle 2 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.178] skip - creating a tarball for bundle 1 to /tmp/bundle_fe8c3d6a85dd8c6091e1d7679699e285.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] creating a tarball for bundle 2 to /tmp/bundle_d5d638eec06dea875e713dfe72ba0944.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] assigned a new bundle 9 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] upload data object /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=UploadDataObject package=fs
DEBU[2024-05-22 08:01:56.178] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.178] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.178] Connecting to scomp1789.wurnet.nl:1247 function=Connect package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.182] Start up a connection with CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
bundle 0 - Bundling ... done! [1 in 230ms]
bundle 0 - Cleaning & making dirs ... done! [1 in 230ms]
bundle 1 - Cleaning & making dirs ... done! [1 in 30ms]
bundle 1 - Bundling ... done! [1 in 0s]
bundle 0 - Uploading ... 0.0% [..................] [0B in 0s]
bundle 2 - Cleaning & making dirs ... 0.0% [..................] [0 in 0s]
DEBU[2024-05-22 08:01:56.190] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.190] adding a file /scratch/snellius/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.190] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-15099_94133-G10_GGACCAGTGGTAGGCTCGCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.190] assigned a new bundle 10 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.191] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.193] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.193] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.200] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.200] adding a file /scratch/snellius/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.200] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.200] deleted exising data objects in the bundle 2 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.200] deleting exising data objects and creating new collections in the bundle 3 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.200] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.200] assigned a new bundle 11 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.200] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.200] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.207] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.207] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.211] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.211] adding a file /scratch/snellius/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.211] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-15783_94134-B12_AGAATTCGCCGAGCTTGCCG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.211] assigned a new bundle 12 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.211] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.213] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.214] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.219] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.219] adding a file /scratch/snellius/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.219] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.219] deleted exising data objects in the bundle 3 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.219] assigned a new bundle 13 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.219] skip - creating a tarball for bundle 2 to /tmp/bundle_d5d638eec06dea875e713dfe72ba0944.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.219] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.219] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.225] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.225] adding a file /scratch/snellius/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.225] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16025_94133-G11_TTCCTACAGCAGAATTCGCC_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.225] deleting exising data objects and creating new collections in the bundle 4 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.225] creating a tarball for bundle 3 to /tmp/bundle_5f7e41cb519bde14e7c18aca9e6eb8e0.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.225] assigned a new bundle 14 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.225] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.225] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.231] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.231] adding a file /scratch/snellius/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.231] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.231] assigned a new bundle 15 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.231] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.231] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.238] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.238] adding a file /scratch/snellius/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16027_94134-A01_TCTCATGATATCCACGTGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] skip - creating a tarball for bundle 3 to /tmp/bundle_5f7e41cb519bde14e7c18aca9e6eb8e0.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] creating a tarball for bundle 4 to /tmp/bundle_688c0385d97dcdd589771394876fb86b.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] skip - creating a tarball for bundle 4 to /tmp/bundle_688c0385d97dcdd589771394876fb86b.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] creating a tarball for bundle 5 to /tmp/bundle_0946bb3b94dcc0c47822d8cf2b0fb4ca.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] skip - creating a tarball for bundle 5 to /tmp/bundle_0946bb3b94dcc0c47822d8cf2b0fb4ca.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] assigned a new bundle 16 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.238] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.238] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.244] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.244] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.247] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.247] adding a file /scratch/snellius/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.247] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.247] creating a tarball for bundle 6 to /tmp/bundle_3c4358053f13e4268cd370379c2ac8c5.tar function=processBundleTar package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.247] assigned a new bundle 17 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.247] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.250] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.250] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.258] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.258] adding a file /scratch/snellius/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.258] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16028_94133-H01_GGTCTATTAACGTGTCGTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.258] assigned a new bundle 18 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.258] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.258] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.264] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.264] adding a file /scratch/snellius/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.264] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.264] deleted exising data objects in the bundle 4 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.264] deleting exising data objects and creating new collections in the bundle 5 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.264] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.264] assigned a new bundle 19 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.264] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.264] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.271] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.271] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.275] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.275] adding a file /scratch/snellius/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.275] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16029_94133-H02_TGATGGCTACATGTTGTTGG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.275] assigned a new bundle 20 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.276] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.278] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.278] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.284] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.284] adding a file /scratch/snellius/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.284] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.284] deleted exising data objects in the bundle 5 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.284] deleting exising data objects and creating new collections in the bundle 6 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.284] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.284] assigned a new bundle 21 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.284] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.284] skip - creating a tarball for bundle 6 to /tmp/bundle_3c4358053f13e4268cd370379c2ac8c5.tar function=processBundleTar package=commonbundle 2 - Cleaning & making dirs ... done! [1 in 20ms]
bundle 2 - Bundling ... done! [1 in 20ms]
bundle 3 - Cleaning & making dirs ... done! [1 in 20ms]
bundle 4 - Cleaning & making dirs ... done! [1 in 30ms]
bundle 3 - Bundling ... done! [1 in 0s]
bundle 4 - Bundling ... done! [1 in 0s]
bundle 5 - Bundling ... done! [1 in 0s]
bundle 6 - Bundling ... done! [1 in 20ms]
bundle 5 - Cleaning & making dirs ... done! [1 in 20ms]
bundle 0 - Uploading ... 0.0% [..................] [0B in 100ms]
bundle 6 - Cleaning & making dirs ... 0.0% [..................] [0 in 0s]
DEBU[2024-05-22 08:01:56.289] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.289] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.296] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.296] adding a file /scratch/snellius/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.296] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16030_94133-H03_AGGACAGGCCAGCTGGAATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.296] assigned a new bundle 22 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.296] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.298] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.298] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.305] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.305] adding a file /scratch/snellius/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.305] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.305] assigned a new bundle 23 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.305] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.305] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.312] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.312] adding a file /scratch/snellius/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.312] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16648_94133-H04_CTAATAACCGCGATGCGGTT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.312] deleted exising data objects in the bundle 6 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.312] deleting exising data objects and creating new collections in the bundle 7 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.312] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.312] assigned a new bundle 24 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.312] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.312] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.317] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.317] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.321] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.321] adding a file /scratch/snellius/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.321] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.321] assigned a new bundle 25 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.321] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.324] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.324] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.331] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.331] adding a file /scratch/snellius/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.331] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16735_94133-H05_CCAGGCACCATAATTAGCGT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.331] assigned a new bundle 26 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.331] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.331] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.337] Start up CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.337] Client policy - CS_NEG_REQUIRE, server policy - CS_NEG_REQUIRE function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.338] Start up SSL function=sslStartup package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.338] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.338] adding a file /scratch/snellius/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.338] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.338] deleted exising data objects in the bundle 7 function=processBundleRemoveFilesAndMakeDirs package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.338] assigned a new bundle 27 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.338] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.338] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.344] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.344] adding a file /scratch/snellius/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.344] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-16986_94133-H06_AATCCAATTGCCTCTACATG_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.345] assigned a new bundle 28 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.345] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.345] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.347] Logging in using pam authentication method function=loginPAMWithToken package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.350] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.350] adding a file /scratch/snellius/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.350] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.350] assigned a new bundle 29 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.350] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.350] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.354] Created a new connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.356] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.356] adding a file /scratch/snellius/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.356] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-17474_94133-H07_CACAGCGGTCCAATAGGAAT_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.356] assigned a new bundle 30 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.356] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.356] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.362] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.363] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.363] adding a file /scratch/snellius/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.363] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.363] assigned a new bundle 31 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.363] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.363] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.368] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.369] adding a file /scratch/snellius/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.369] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-17584_94133-H08_GTGACGGAGCTGGACCGCCA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.369] assigned a new bundle 32 function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.369] checking if target /unlock/home/wur.gutbrain_axis/data/meta/DNA16-18030_94133-H09_TATGACAATCATACAAGCAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz for source /scratch/snellius/meta/DNA16-18030_94133-H09_TATGACAATCATACAAGCAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz exists function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.369] Reuse an idle connection function=Get package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.376] Returning a connection function=Return package=session struct=ConnectionPool
DEBU[2024-05-22 08:01:56.376] adding a file /scratch/snellius/meta/DNA16-18030_94133-H09_TATGACAATCATACAAGCAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz to the bundle as it doesn't exist function=Schedule package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.376] > scheduled a local file bundle-upload /scratch/snellius/meta/DNA16-18030_94133-H09_TATGACAATCATACAAGCAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz function=Schedule package=commons struct=BundleTransferManager
ERRO[2024-05-22 08:01:56.376] failed to upload file /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz in bundle 0 to /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz: failed to open data object /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz: failed to open data object: function=Start package=commons struct=BundleTransferManager
DEBU[2024-05-22 08:01:56.377] Disconnecting the connection function=Disconnect package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.377] Disconnecting the connection function=Disconnect package=connection struct=IRODSConnection
DEBU[2024-05-22 08:01:56.377] Disconnecting the connection function=Disconnect package=connection struct=IRODSConnection
ERRO[2024-05-22 08:01:56.377] failed to perform sync (from local to iRODS):
github.com/cyverse/gocommands/cmd/subcmd.processSyncCommand
/github/workspace/cmd/subcmd/sync.go:92
- failed to perform bput ./meta to /unlock/home/wur.gutbrain_axis/data:
github.com/cyverse/gocommands/cmd/subcmd.processBputCommand
/github/workspace/cmd/subcmd/bput.go:175
- failed to walk for /scratch/snellius/meta:
github.com/cyverse/gocommands/cmd/subcmd.bputOne
/github/workspace/cmd/subcmd/bput.go:241
- failed to schedule /scratch/snellius/meta/DNA16-18030_94133-H09_TATGACAATCATACAAGCAA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz:
github.com/cyverse/gocommands/cmd/subcmd.bputOne.func1
/github/workspace/cmd/subcmd/bput.go:234
- failed to upload file /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz in bundle 0 to /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz:
github.com/cyverse/gocommands/commons.(*BundleTransferManager).processBundleUpload
/github/workspace/commons/bundle_transfer.go:1156
- failed to open data object /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz:
github.com/cyverse/go-irodsclient/irods/fs.UploadDataObject
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.6/irods/fs/data_object_bulk.go:143
- failed to open data object:
github.com/cyverse/go-irodsclient/irods/fs.OpenDataObjectWithOperation
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.6/irods/fs/data_object.go:1540
- function=main package=main
iRODS Error (code: '-1', message: '')
Error Trace:
- failed to perform sync (from local to iRODS):
github.com/cyverse/gocommands/cmd/subcmd.processSyncCommand
/github/workspace/cmd/subcmd/sync.go:92
- failed to perform bput ./meta to /unlock/home/wur.gutbrain_axis/data:
github.com/cyverse/gocommands/cmd/subcmd.processBputCommand
/github/workspace/cmd/subcmd/bput.go:175
- failed to walk for /scratch/snellius/meta:
github.com/cyverse/gocommands/cmd/subcmd.bputOne
/github/workspace/cmd/subcmd/bput.go:241
- failed to schedule /scratch/snellius/meta/DNA16-18030_94133-H09_TATGACAATCATACAAGCAA_L001_R2_001_BH3MLWDSX5.filt.fastq.gz:
github.com/cyverse/gocommands/cmd/subcmd.bputOne.func1
/github/workspace/cmd/subcmd/bput.go:234
- failed to upload file /scratch/snellius/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz in bundle 0 to /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz:
github.com/cyverse/gocommands/commons.(*BundleTransferManager).processBundleUpload
/github/workspace/commons/bundle_transfer.go:1156
- failed to open data object /unlock/home/wur.gutbrain_axis/data/meta/DNA16-14185_94133-G06_GTTGGATGAATTACCTGGAA_L001_R1_001_BH3MLWDSX5.filt.fastq.gz:
github.com/cyverse/go-irodsclient/irods/fs.UploadDataObject
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.6/irods/fs/data_object_bulk.go:143
- failed to open data object:
github.com/cyverse/go-irodsclient/irods/fs.OpenDataObjectWithOperation
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.6/irods/fs/data_object.go:1540
-
➜ snellius
Ah, I think I got it...
Level 0: #### !!RDM WARNING!! pep_api_data_obj_put_pre: You cannot upload the file without calculating and verifying the checksum. You need to upload with -K (not -k) to guarantee integrity on your filetransfer to irods
This is from iCommands, should have checked that first... Is this option available in gocmd?
There's no such flag supported in GoCommands yet. I'll figure out what that is and add it in GoCommands soon.
FYI, working on it now. But it will take few weeks to finish it due to other tasks.
@iychoi, If in one way or another I can help please let me know :)
@jjkoehorst Thanks. Actually the latest commit includes -k
and -K
flags. I just need more time to test before releasing. If you can build from code, can you try it?
Looks to be working!
bundle 0 - Bundling ... done! [458.00K in 420ms]
bundle 0 - Cleaning & making dirs ... done! [50 in 480ms]
bundle 1 - Cleaning & making dirs ... done! [50 in 490ms]
bundle 2 - Cleaning & making dirs ... done! [50 in 230ms]
bundle 1 - Bundling ... done! [66.87M in 560ms]
bundle 0 - Uploading ... done! [1.75GB in 15.14s]
bundle 2 - Bundling ... done! [76.96M in 150ms]
bundle 3 - Cleaning & making dirs ... done! [50 in 230ms]
bundle 0 - Extracting ... done! [50 in 2.96s]
bundle 3 - Bundling ... done! [38.56M in 380ms]
bundle 4 - Cleaning & making dirs ... done! [50 in 1.16s]
bundle 4 - Bundling ... done! [42.73M in 90ms]
bundle 5 - Cleaning & making dirs ... done! [50 in 880ms]
bundle 5 - Bundling ... done! [39.81M in 100ms]
bundle 6 - Cleaning & making dirs ... done! [50 in 840ms]
bundle 6 - Bundling ... done! [41.02M in 90ms]
bundle 7 - Cleaning & making dirs ... done! [50 in 890ms]
bundle 7 - Bundling ... done! [37.42M in 80ms]
bundle 8 - Cleaning & making dirs ... done! [50 in 760ms]
bundle 8 - Bundling ... done! [40.49M in 90ms]
bundle 9 - Cleaning & making dirs ... done! [50 in 780ms]
bundle 9 - Bundling ... done! [72.96M in 140ms]
bundle 10 - Cleaning & making dirs ... done! [50 in 810ms]
bundle 10 - Bundling ... done! [28.12M in 60ms]
bundle 11 - Cleaning & making dirs ... done! [50 in 880ms]
bundle 11 - Bundling ... done! [91.83M in 170ms]
bundle 12 - Cleaning & making dirs ... done! [50 in 830ms]
bundle 1 - Uploading ... 95.2% [#################.] [1.76GB in 1m9.2s; ~ETA: 4s]
bundle 2 - Uploading ... 75.7% [#############.....] [1.52GB in 1m5.5s; ~ETA: 22s]
bundle 3 - Uploading ... 96.4% [#################.] [1.74GB in 1m2.4s; ~ETA: 3s]
bundle 4 - Uploading ... 81.8% [##############....] [1.58GB in 59.2s; ~ETA: 14s]
bundle 5 - Uploading ... 73.1% [#############.....] [1.49GB in 55.9s; ~ETA: 21
Thanks for testing! I'll soon make a new release!
When trying to upload data to a (relatively) new irods instance I have the following issue
The issue might be on our side but
iRODS Error (code: '-1', message: '')
does not tell me much what goes wrong... any ideas?