search

ENCODE-DCC / atac-seq-pipeline

ENCODE ATAC-seq pipeline
MIT License
384 stars 172 forks source link

pipeline because javax.net.ssl.SSLException failed #409

Open sjh996 opened 1 year ago

sjh996 commented 1 year ago

Describe the bug pipeline because javax.net.ssl.SSLException failed

OS/Platform OS/Platform: [e.g. Ubuntu 18.04, Google Cloud, Stanford Sherlock/SCG cluster, ...] Conda version: conda 22.9.0. Pipeline version: [e.g. v2.2.1] Caper version: [e.g. v2.2.3]

Caper configuration file backend=local

Local directory for localized files and Cromwell's intermediate files.

If not defined then Caper will make .caper_tmp/ on CWD or local-out-dir.

/tmp is not recommended since Caper store localized data files here.

local-loc-dir=

cromwell=/public/home/jiahsun/.caper/cromwell_jar/cromwell-82.jar womtool=/public/home/jiahsun/.caper/womtool_jar/womtool-82.jar

Input JSON file

{ "atac.pipeline_type" : "atac", "atac.genome_tsv" : "/public/home/jiahsun/02.ATAC_seq/01.horse_data/03.build/horse-annotation.tsv", "atac.fastqs_rep1_R1" : [ "/public/home/jiahsun/02.ATAC_seq/01.horse_data/02atac-cleandata/atac-clean-data-fa/BMM_R1_val_1.fq.gz"], "atac.fastqs_rep1_R2" : [ "/public/home/jiahsun/02.ATAC_seq/01.horse_data/02atac-cleandata/atac-clean-data-fa/BMM_R2_val_2.fq.gz"], "atac.fastqs_rep2_R1" : [ "/public/home/jiahsun/02.ATAC_seq/01.horse_data/02atac-cleandata/atac-clean-data-fa/XFM-2_R1_val_1.fq.gz"], "atac.fastqs_rep2_R1" : [ "/public/home/jiahsun/02.ATAC_seq/01.horse_data/02atac-cleandata/atac-clean-data-fa/XFM-2_R2_val_2.fq.gz"], "atac.paired_end" : true, "atac.auto_detect_adapter" : true, "atac.enable_xcor" : true, "atac.title" : "rep1-BMM_rep2-XFJ", "atac.description" : "ATAC-seq on rep1-BMM_rep2-XFJ 3.15" }

Troubleshooting result cromwell.out 2023-03-30 19:57:06,715 INFO - Running with database db.url = jdbc:hsqldb:mem:77b7f1a5-1eac-4692-8a7c-701c71f90fa5;shutdown=false;hsqldb.tx=mvcc Running Changeset: changesets/db_schema.xml::db_schema_other_table_alldb::scottfrazer Running Changeset: changesets/db_schema.xml::db_schema_symbol_table_otherdb::scottfrazer Running Changeset: changesets/db_schema.xml::db_schema_constraints::scottfrazer Running Changeset: changesets/symbol_iteration_null.xml::symbol_iteration_null::kshakir Running Changeset: changesets/wdl_and_inputs.xml::WORKFLOW_EXECUTION_AUX::mcovarr Running Changeset: changesets/wdl_and_inputs.xml::WE_AUX_CONSTRAINTS::mcovarr Running Changeset: changesets/drop_workflow_uri_and_local_command.xml::drop_workflow_uri_and_local_command::kshakir Running Changeset: changesets/local_job_allow_null.xml::local_job_allow_null::kshakir Running Changeset: changesets/symbol_iteration_not_null.xml::symbol_iteration_not_null::kshakir Running Changeset: changesets/add_unique_constraints.xml::add_unique_constraints::kshakir Running Changeset: changesets/lengthen_wdl_value.xml::LENGTHEN_WDL_VALUE::sfrazer Running Changeset: changesets/add_index_in_execution.xml::add-index-in-execution::tjeandet Running Changeset: changesets/rename_iteration_to_index.xml::rename-iteration-to-index::tjeandet Running Changeset: changesets/sge.xml::sge::scottfrazer Running Changeset: changesets/sge.xml::db_schema_constraints::scottfrazer Running Changeset: changesets/change_execution_unique_constraint.xml::change_execution_unique_constraint::tjeandet Running Changeset: changesets/rc.xml::rc::mcovarr Running Changeset: changesets/workflow_options.xml::workflow-options::sfrazer Running Changeset: changesets/jes_id_update.xml::jes_id_update::chrisl Running Changeset: changesets/optional_sge_ids.xml::optional_sge_ids::chrisl Running Changeset: changesets/add_start_end_time_in_execution.xml::add-start-end-time-in-execution::jgentry Running Changeset: changesets/rename_jes_id.xml::rename-jes-id-to-jes-run-id::sfrazer Running Changeset: changesets/add_workflow_name.xml::workflow-name::mcovarr Running Changeset: changesets/top_level_output.xml::top_level_output::chrisl Running Changeset: changesets/workflow_execution_aux_not_null.xml::workflow_execution_aux_not_null::tjeandet Running Changeset: changesets/call_result_caching.xml::call_result_caching::chrisl Running Changeset: changesets/events_table.xml::execution_event_table::chrisl Running Changeset: changesets/sync_not_null_constraints.xml::job_not_null_columns::kshakir Running Changeset: changesets/sync_not_null_constraints.xml::events_end_dt_not_null::kshakir Running Changeset: changesets/sge_job_execution_unique_key.xml::sge_job_execution_unique_key::kshakir Running Changeset: changesets/add_attempt_in_execution.xml::add-attempt-in-execution::tjeandet Running Changeset: changesets/execution_backend_info.xml::CREATE_EXECUTION_INFO::mcovarr Running Changeset: changesets/execution_backend_info.xml::add_foreign_key_constraints::mcovarr Running Changeset: changesets/execution_backend_info.xml::add_backend_type_column::mcovarr Running Changeset: changesets/execution_backend_info.xml::insert_data_from_backend_tables::mcovarr Running Changeset: changesets/execution_backend_info.xml::set_backend_type::mcovarr Running Changeset: changesets/execution_backend_info.xml::drop_job_tables::mcovarr Running Changeset: changesets/execution_backend_info.xml::execution_info_uniqueness_constraint::mcovarr Running Changeset: changesets/execution_backend_info.xml::execution_backend_type_not_null::mcovarr Running Changeset: changesets/runtime_attributes_table.xml::runtime-attributes-table::tjeandet Running Changeset: changesets/failure_table.xml::failure_table::chrisl Running Changeset: changesets/metadata_journal.xml::metadata_journal::mcovarr Running Changeset: changesets/metadata_journal.xml::metadata_workflow_index::mcovarr Running Changeset: changesets/metadata_journal.xml::metadata_job_index::mcovarr Running Changeset: changesets/metadata_journal.xml::metadata_job_and_key_index::mcovarr Running Changeset: changesets/metadata_typed_values.xml::metadata_typed_values::tjeandet Running Changeset: changesets/workflow_metadata_summary.xml::metadata_journal_id_int_to_big_int::kshakir Running Changeset: changesets/workflow_metadata_summary.xml::workflow_metadata_summary::mcovarr Running Changeset: changesets/workflow_metadata_summary.xml::workflow_metadata_uuid_idx::mcovarr Running Changeset: changesets/workflow_metadata_summary.xml::workflow_metadata_name_idx::mcovarr Running Changeset: changesets/workflow_metadata_summary.xml::workflow_metadata_status_idx::mcovarr Running Changeset: changesets/metadata_journal_subsecond_timestamp.xml::metadata_journal_subsecond_timestamp::mcovarr Running Changeset: changesets/metadata_journal_subsecond_timestamp.xml::metadata_journal_timestamp_not_null::mcovarr Running Changeset: changesets/workflow_store.xml::WORKFLOW_STORE::chrisl Running Changeset: changesets/workflow_store.xml::workflow_store_uuid_index::kshakir Running Changeset: changesets/workflow_store.xml::workflow_store_state_index::chrisl Running Changeset: changesets/backend_KV_Store.xml::BACKEND_KV_STORE::rmunshi Running Changeset: changesets/backend_KV_Store.xml::backend_KV_store_job_key_constraint::rmunshi Running Changeset: changesets/job_store.xml::JOB_STORE::chrisl Running Changeset: changesets/job_store.xml::job_store_uuid_index::chrisl Running Changeset: changesets/job_store.xml::job_store_jobkey_index::chrisl Running Changeset: changesets/callcaching.xml::call_caching_result_metainfo::chrisl Running Changeset: changesets/callcaching.xml::call_caching_result_metainfo_uniqueness::chrisl Running Changeset: changesets/callcaching.xml::call_caching_hashes::chrisl Running Changeset: changesets/callcaching.xml::call_caching_result_hashes_uniqueness::chrisl Running Changeset: changesets/callcaching.xml::call_caching_result_hashes_uniqueness_foreign_key::chrisl Running Changeset: changesets/callcaching.xml::call_caching_result_simpletons::chrisl Running Changeset: changesets/callcaching.xml::call_caching_result_simpletons_uniqueness::chrisl Running Changeset: changesets/callcaching.xml::call_caching_result_simpletons_foreign_key::chrisl Running Changeset: changesets/call_caching_allow_result_reuse_fix.xml::call_caching_fix_boolean::mcovarr Running Changeset: changesets/call_caching_allow_result_reuse_fix.xml::call_caching_reuse_default::mcovarr Running Changeset: changesets/job_store_simpletons.xml::job_store_result_simpletons::mcovarr Running Changeset: changesets/job_store_simpletons.xml::job_store_result_simpletons_uniqueness::mcovarr Running Changeset: changesets/job_store_simpletons.xml::job_store_result_simpletons_foreign_key::mcovarr Running Changeset: changesets/job_store_simpletons.xml::job_store_remove_job_output::mcovarr Running Changeset: changesets/restart_and_recover_migration.xml::WORKFLOW_STORE_MIGRATION::mcovarr Running Changeset: changesets/restart_and_recover_migration.xml::JOB_STORE_MIGRATION::mcovarr Running Changeset: changesets/restart_and_recover_migration.xml::JOB_STORE_RESULT_SIMPLETON_VALUE_NULLABLE::mcovarr Running Changeset: changesets/restart_and_recover_migration.xml::JOB_STORE_RESULT_SIMPLETON_MIGRATION::mcovarr Running Changeset: changesets/restart_and_recover_migration.xml::BACKEND_KV_STORE_MIGRATION::mcovarr Running Changeset: changesets/restart_and_recover_migration.xml::WORKFLOW_OPTIONS_RENAME_MIGRATION::mcovarr Running Changeset: changesets/summary_status_table.xml::summary_status_table::kshakir Running Changeset: changesets/summary_status_table.xml::summary_status_summarized_table_name_index::kshakir Running Changeset: changesets/standardize_column_names.xml::workflow_execution_uuid::rmunshi Running Changeset: changesets/standardize_column_names.xml::call_job_identifiers::rmunshi Running Changeset: changesets/embiggen_metadata_value.xml::entry_or_journal_existence_xor::mcovarr Running Changeset: changesets/embiggen_metadata_value.xml::embiggen_metadata_entry::mcovarr Running Changeset: changesets/embiggen_metadata_value.xml::embiggen_metadata_journal::mcovarr Running Changeset: changesets/call_caching_job_detritus.xml::call_caching_job_detritus::rmunshi Running Changeset: changesets/call_caching_job_detritus.xml::call_caching_job_detritus_key_uniqueness::rmunshi Running Changeset: changesets/call_caching_job_detritus.xml::call_caching_job_detritus_foreign_key::rmunshi Running Changeset: changesets/job_store_tinyints.xml::job_store_fix_job_successful::mcovarr Running Changeset: changesets/job_store_tinyints.xml::job_store_fix_job_retryable_failure::mcovarr Running Changeset: changesets/job_store_tinyints.xml::job_store_not_nullable_job_successful::mcovarr Running Changeset: changesets/standardize_column_names_again.xml::standardize_column_names_again::kshakir Running Changeset: changesets/rename_workflow_options_in_metadata.xml::rename_workflow_options_in_metadata::tjeandet 2023-03-30 19:57:09,626 INFO - Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000 2023-03-30 19:57:09,639 INFO - [RenameWorkflowOptionsInMetadata] 100% Running Changeset: changesets/encrypt_and_clear_workflow_options.xml::encrypt_workflow_store_entry_workflow_options::kshakir Running Changeset: changesets/encrypt_and_clear_workflow_options.xml::clear_metadata_entry_workflow_options::kshakir Running Changeset: changesets/sub_workflow_store.xml::SUB_WORKFLOW_STORE_ENTRY::tjeandet Running Changeset: changesets/sub_workflow_store.xml::sub_workflow_store_uuid_index::tjeandet Running Changeset: changesets/sub_workflow_store.xml::sub_workflow_store_jobkey_index::tjeandet Running Changeset: changesets/sub_workflow_store.xml::sub_workflow_store_root_workflow_fk::tjeandet Running Changeset: changesets/workflow_store_imports_file.xml::workflow-store-imports-file::cjllanwarne Running Changeset: changesets/workflow_store_labels_file.xml::workflow-store-labels-file::cjllanwarne Running Changeset: changesets/embiggen_detritus_value.xml::embiggen_call_caching_detritus_entry::rmunshi Running Changeset: changesets/standardize_column_names_patches.xml::standardize_column_names_patches::kshakir Running Changeset: changesets/standardize_column_names_patches.xml::hsqldb_longvarchar_to_longtext::kshakir Running Changeset: changesets/nullable_lobs.xml::nullable_lobs::kshakir Running Changeset: changesets/add_attempt_in_call_caching_entry.xml::add_attempt_in_call_caching_entry::tjeandet Running Changeset: changesets/replace_empty_custom_labels.xml::custom_labels_not_null::rmunshi Running Changeset: changesets/call_caching_aggregated_hashes.xml::call_caching_aggregation_entry::tjeandet Running Changeset: changesets/call_caching_aggregated_hashes.xml::call_caching_aggregation_entry_keys_and_indexes::tjeandet Running Changeset: changesets/custom_label_entry.xml::custom_LABEL_entry::rmunshi Running Changeset: changesets/custom_label_entry.xml::WMSE_workflow_execution_uuid_foreign_key::rmunshi Running Changeset: changesets/custom_label_entry.xml::custom_label_entry_index::rmunshi Running Changeset: changesets/custom_label_entry.xml::modify label key/value data types::rmunshi Running Changeset: changesets/docker_hash_store.xml::DOCKER_HASH_STORE_ENTRY::tjeandet Running Changeset: changesets/docker_hash_store.xml::docker_hash_store_uuid_unique_constraint::tjeandet Running Changeset: changesets/workflow_store_type_and_version.xml::workflow-store-type-and-version::mcovarr Running Changeset: changesets/remove_pre_pbe_tables.xml::drop-pre-pbe-tables::mcovarr Running Changeset: changesets/move_sql_metadata_changelog.xml::move_metadata_changelog::kshakir Running Changeset: changesets/workflow_store_state_widening.xml::workflow-store-state-widening::tjeandet Running Changeset: changesets/workflow_store_restarted_column.xml::workflow-store-restarted-column::tjeandet Running Changeset: changesets/workflow_store_restarted_column.xml::update-restartable::tjeandet Running Changeset: changesets/workflow_store_workflow_root_column.xml::workflow-store-workflow-root-column::tjeandet Running Changeset: changesets/workflow_store_horizontal_db.xml::workflow-store-horizontal-db::mcovarr Running Changeset: changesets/add_workflow_url_in_workflow_store_entry.xml::add_workflow_url_in_workflow_store_entry::sshah Running Changeset: changesets/change_max_size_workflow_url.xml::change_max_size_for_workflow_url::sshah Running Changeset: changesets/docker_hash_store_add_size_column.xml::docker_hash_store_entry_add_size_column::tjeandet Running Changeset: changesets/enlarge_call_caching_hash_entry_id.xml::restore_auto_increment_call_caching_hash_entry_id::mcovarr Running Changeset: changesets/add_hog_group_in_workflow_store.xml::add_hog_group_in_workflow_store::cjllanwarne Running Changeset: changesets/resync_engine_schema.xml::resync-engine-schema::kshakir Running Changeset: changesets/enlarge_job_store_ids.xml::drop_job_store_simpleton_entry_fk::mcovarr Running Changeset: changesets/enlarge_job_store_ids.xml::enlarge_job_store_entry_id::mcovarr Running Changeset: changesets/enlarge_job_store_ids.xml::enlarge_job_store_simpleton_entry_id::mcovarr Running Changeset: changesets/enlarge_job_store_ids.xml::enlarge_job_store_simpleton_entry_fk::mcovarr Running Changeset: changesets/enlarge_job_store_ids.xml::recreate_job_store_simpleton_entry_fk::mcovarr 2023-03-30 19:57:09,948 INFO - Running with database db.url = jdbc:hsqldb:mem:dffde933-a91c-4eed-b955-5bbcd9e4676f;shutdown=false;hsqldb.tx=mvcc Running Changeset: metadata_changesets/move_sql_metadata_changelog.xml::move_metadata_changelog::kshakir Running Changeset: changesets/change_max_size_label_entry.xml::change_max_size_for_key_and_value_in_label::sshah Running Changeset: metadata_changesets/delete_duplicate_custom_labels.xml::delete_duplicate_custom_labels::kshakir Running Changeset: metadata_changesets/add_submission_timestamp_metadata_summary.xml::add_submission_timestamp_in_metadata_entry::sshah Running Changeset: metadata_changesets/custom_label_entry_key_value_index.xml::custom_label_entry_key_value_index::mcovarr Running Changeset: metadata_changesets/add_parent_and_root_workflow_execution_uuid.xml::add_parent_and_root_workflow_execution_uuid::kshakir Running Changeset: metadata_changesets/add_parent_and_root_workflow_typo_fix.xml::add_parent_and_root_workflow_typo_fix::kshakir Running Changeset: metadata_changesets/metadata_index_removals.xml::metadata_index_removals::mcovarr Running Changeset: metadata_changesets/add_metadata_archive_status.xml::add_metadata_archive_status::cjllanwarne Running Changeset: metadata_changesets/add_metadata_archive_status.xml::metadata_archive_status_index::cjllanwarne Running Changeset: metadata_changesets/summarization_queue_table.xml::summary_queue_table::gsterin Running Changeset: metadata_changesets/summarization_queue_table.xml::summary_queue_initialization::gsterin Running Changeset: metadata_changesets/summarization_queue_table_add_primary_key.xml::summary_queue_add_primary_key::gsterin Running Changeset: metadata_changesets/remove_non_summarizable_metadata_from_queue.xml::delete_non_summarizable_metadata_from_queue::mcovarr Running Changeset: metadata_changesets/update_metadata_archive_index.xml::update_old_metadata_archive_status_index::cjllanwarne Running Changeset: metadata_changesets/reset_archive_statuses_to_null.xml::reset_archive_statuses_to_null::cjllanwarne 2023-03-30 19:57:10,514 INFO - Slf4jLogger started 2023-03-30 19:57:10,719 cromwell-system-akka.dispatchers.engine-dispatcher-5 INFO - Workflow heartbeat configuration: { "cromwellId" : "cromid-4fc1f55", "heartbeatInterval" : "2 minutes", "ttl" : "10 minutes", "failureShutdownDuration" : "5 minutes", "writeBatchSize" : 10000, "writeThreshold" : 10000 } 2023-03-30 19:57:10,815 cromwell-system-akka.dispatchers.service-dispatcher-12 INFO - Metadata summary refreshing every 1 second. 2023-03-30 19:57:10,817 cromwell-system-akka.dispatchers.service-dispatcher-12 INFO - No metadata archiver defined in config 2023-03-30 19:57:10,817 cromwell-system-akka.dispatchers.service-dispatcher-12 INFO - No metadata deleter defined in config 2023-03-30 19:57:10,830 cromwell-system-akka.dispatchers.service-dispatcher-11 INFO - WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds. 2023-03-30 19:57:10,830 cromwell-system-akka.actor.default-dispatcher-3 INFO - KvWriteActor configured to flush with batch size 200 and process rate 5 seconds. 2023-03-30 19:57:10,851 cromwell-system-akka.dispatchers.engine-dispatcher-45 INFO - CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds. 2023-03-30 19:57:10,919 cromwell-system-akka.dispatchers.engine-dispatcher-45 INFO - JobRestartCheckTokenDispenser - Distribution rate: 50 per 1 seconds. 2023-03-30 19:57:10,942 cromwell-system-akka.dispatchers.engine-dispatcher-45 INFO - JobExecutionTokenDispenser - Distribution rate: 1 per 2 seconds. 2023-03-30 19:57:10,971 cromwell-system-akka.dispatchers.engine-dispatcher-5 INFO - SingleWorkflowRunnerActor: Version 82 2023-03-30 19:57:10,979 cromwell-system-akka.dispatchers.engine-dispatcher-5 INFO - SingleWorkflowRunnerActor: Submitting workflow 2023-03-30 19:57:10,997 cromwell-system-akka.dispatchers.engine-dispatcher-45 INFO - 1 new workflows fetched by cromid-4fc1f55: d93f9fdb-5967-4a13-8fd9-d4058768107f 2023-03-30 19:57:11,017 cromwell-system-akka.dispatchers.engine-dispatcher-5 INFO - WorkflowManagerActor: Starting workflow UUID(d93f9fdb-5967-4a13-8fd9-d4058768107f) 2023-03-30 19:57:11,022 cromwell-system-akka.dispatchers.api-dispatcher-48 INFO - Unspecified type (Unspecified version) workflow d93f9fdb-5967-4a13-8fd9-d4058768107f submitted 2023-03-30 19:57:11,024 cromwell-system-akka.dispatchers.engine-dispatcher-5 INFO - WorkflowManagerActor: Successfully started WorkflowActor-d93f9fdb-5967-4a13-8fd9-d4058768107f 2023-03-30 19:57:11,024 cromwell-system-akka.dispatchers.engine-dispatcher-5 INFO - Retrieved 1 workflows from the WorkflowStoreActor 2023-03-30 19:57:11,041 cromwell-system-akka.dispatchers.engine-dispatcher-46 INFO - SingleWorkflowRunnerActor: Workflow submitted UUID(d93f9fdb-5967-4a13-8fd9-d4058768107f) 2023-03-30 19:57:11,045 cromwell-system-akka.dispatchers.engine-dispatcher-40 INFO - WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes. 2023-03-30 19:57:11,267 cromwell-system-akka.dispatchers.engine-dispatcher-40 INFO - MaterializeWorkflowDescriptorActor [UUID(d93f9fdb)]: Parsing workflow as WDL 1.0 2023-03-30 19:57:13,464 cromwell-system-akka.dispatchers.engine-dispatcher-40 INFO - MaterializeWorkflowDescriptorActor [UUID(d93f9fdb)]: Call-to-Backend assignments: atac.overlap -> Local, atac.idr_pr -> Local, atac.fraglen_stat_pe -> Local, atac.bam2ta -> Local, atac.call_peak_pr1 -> Local, atac.preseq -> Local, atac.count_signal_track -> Local, atac.xcor -> Local, atac.align_mito -> Local, atac.annot_enrich -> Local, atac.filter_no_dedup -> Local, atac.call_peak_pr2 -> Local, atac.pool_ta_pr2 -> Local, atac.pool_ta_pr1 -> Local, atac.bam2ta_no_dedup -> Local, atac.overlap_pr -> Local, atac.reproducibility_overlap -> Local, atac.frac_mito -> Local, atac.compare_signal_to_roadmap -> Local, atac.gc_bias -> Local, atac.tss_enrich -> Local, atac.spr -> Local, atac.count_signal_track_pooled -> Local, atac.pool_ta -> Local, atac.read_genome_tsv -> Local, atac.error_input_data -> Local, atac.qc_report -> Local, atac.macs2_signal_track_pooled -> Local, atac.call_peak_ppr1 -> Local, atac.overlap_ppr -> Local, atac.call_peak -> Local, atac.reproducibility_idr -> Local, atac.macs2_signal_track -> Local, atac.idr -> Local, atac.call_peak_ppr2 -> Local, atac.filter -> Local, atac.jsd -> Local, atac.pool_blacklist -> Local, atac.call_peak_pooled -> Local, atac.idr_ppr -> Local, atac.align -> Local 2023-03-30 19:57:13,671 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,673 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,674 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,675 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,675 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,678 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,678 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,679 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,679 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,679 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,679 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,679 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,680 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,680 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,680 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,680 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,680 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,681 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,681 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,681 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,681 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,681 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,681 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,682 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,682 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,682 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,682 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,682 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,683 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,683 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:13,683 cromwell-system-akka.dispatchers.backend-dispatcher-58 WARN - Local [UUID(d93f9fdb)]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2023-03-30 19:57:15,936 cromwell-system-akka.dispatchers.engine-dispatcher-119 INFO - Not triggering log of restart checking token queue status. Effective log interval = None 2023-03-30 19:57:15,955 cromwell-system-akka.dispatchers.engine-dispatcher-83 INFO - Not triggering log of execution token queue status. Effective log interval = None 2023-03-30 19:57:17,939 cromwell-system-akka.dispatchers.engine-dispatcher-83 INFO - WorkflowExecutionActor-d93f9fdb-5967-4a13-8fd9-d4058768107f [UUID(d93f9fdb)]: Starting atac.read_genome_tsv 2023-03-30 19:57:18,966 cromwell-system-akka.dispatchers.engine-dispatcher-83 INFO - Assigned new job execution tokens to the following groups: d93f9fdb: 1 2023-03-30 19:57:21,946 cromwell-system-akka.dispatchers.engine-dispatcher-96 INFO - BT-322 d93f9fdb:atac.read_genome_tsv:-1:1 is eligible for call caching with read = true and write = true 2023-03-30 19:57:22,002 cromwell-system-akka.dispatchers.engine-dispatcher-116 INFO - BT-322 d93f9fdb:atac.read_genome_tsv:-1:1 cache hit copying nomatch: could not find a suitable cache hit. 2023-03-30 19:57:22,004 cromwell-system-akka.dispatchers.engine-dispatcher-116 INFO - d93f9fdb-5967-4a13-8fd9-d4058768107f-EngineJobExecutionActor-atac.read_genome_tsv:NA:1 [UUID(d93f9fdb)]: Could not copy a suitable cache hit for d93f9fdb:atac.read_genome_tsv:-1:1. No copy attempts were made. 2023-03-30 19:57:22,038 cromwell-system-akka.dispatchers.backend-dispatcher-134 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.read_genome_tsv:NA:1]: Unrecognized runtime attribute keys: disks, time, memory, cpu 2023-03-30 19:57:22,150 cromwell-system-akka.dispatchers.backend-dispatcher-134 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.read_genome_tsv:NA:1]: `echo "$(basename /cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/inputs/-117978071/horse-annotation.tsv)" > genome_name

create empty files for all entries

touch ref_fa bowtie2_idx_tar chrsz gensz blacklist blacklist2 touch ref_mito_fa touch bowtie2_mito_idx_tar touch tss tss_enrich # for backward compatibility touch dnase prom enh reg2map reg2map_bed roadmap_meta touch mito_chr_name touch regex_bfilt_peak_chr_name

python <<CODE import os with open('/cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/inputs/-117978071/horse-annotation.tsv','r') as fp: for line in fp: arr = line.strip('\n').split('\t') if arr: key, val = arr with open(key,'w') as fp2: fp2.write(val) CODE` 2023-03-30 19:57:22,550 cromwell-system-akka.dispatchers.backend-dispatcher-134 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.read_genome_tsv:NA:1]: executing: rm -f /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid

if [ 'true' == 'true' ] && [ 'conda' == 'docker' ] || \ [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'encodedcc/atac-seq-pipeline@sha256:4404cfb32a99839b727537547a4c64839b6fa6c7050fb01e3ed2d53a19e40c30' ] then docker run -i --cidfile=/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid --user= --entrypoint=/bin/bash \ --volume=/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv:/cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv:delegated encodedcc/atac-seq-pipeline@sha256:4404cfb32a99839b727537547a4c64839b6fa6c7050fb01e3ed2d53a19e40c30 /cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/script rc=$(docker wait cat /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid) docker rm cat /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid else

recover GID lost due to Cromwell running chmod 777 on CWD

chown :`stat -c '%G' /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv` -R /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv
chmod g+s /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv

if [ 'true' == 'true' ] && [ 'conda' == 'singularity' ] || \
   [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif' ]
then
    mkdir -p $HOME/.singularity/lock/
    flock --exclusive --timeout 600 \
        $HOME/.singularity/lock/`echo -n 'https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif' | md5sum | cut -d' ' -f1` \
        singularity exec --containall https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif echo 'Successfully pulled https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif'

    singularity exec --cleanenv --home=`dirname /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv` \
        --bind=,/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv:/cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv \
         \
        https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif /bin/bash /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/script & echo $! > /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid
else
    # remap paths between inside and outside of a docker container
    shopt -s nullglob
    sed -i 's#/cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv#/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv#g' /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/script `dirname /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/script`/write_*.tmp

    if [ 'true' == 'true' ] && [ 'conda' == 'conda' ] || \
       [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'encd-atac' ]
    then
        conda run --name=encd-atac /bin/bash /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/script & echo $! > /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid
    else
        /bin/bash /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/script & echo $! > /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid
    fi
fi

touch /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid.not_docker
wait `cat /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-read_genome_tsv/execution/docker_cid`
rc=`echo $?`

fi

exit $rc 2023-03-30 19:57:25,865 cromwell-system-akka.dispatchers.backend-dispatcher-134 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.read_genome_tsv:NA:1]: job id: 201284 2023-03-30 19:57:25,872 cromwell-system-akka.dispatchers.backend-dispatcher-133 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.read_genome_tsv:NA:1]: Status change from - to WaitingForReturnCode 2023-03-30 19:57:26,822 ERROR - NIO2 channel closed with error javax.net.ssl.SSLException: SSL Handshake WRAP produced 0 bytes: Status = OK HandshakeStatus = NEED_WRAP bytesConsumed = 0 bytesProduced = 0 at org.http4s.blaze.pipeline.stages.SSLStage.sslHandshakeLoop$1(SSLStage.scala:220) at org.http4s.blaze.pipeline.stages.SSLStage.sslHandshake(SSLStage.scala:245) at org.http4s.blaze.pipeline.stages.SSLStage.org$http4s$blaze$pipeline$stages$SSLStage$$doRead(SSLStage.scala:156) at org.http4s.blaze.pipeline.stages.SSLStage.$anonfun$doRead$1(SSLStage.scala:148) at org.http4s.blaze.pipeline.stages.SSLStage.$anonfun$doRead$1$adapted(SSLStage.scala:145) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at org.http4s.blaze.util.SerialExecutionContext$$anon$1.act(SerialExecutionContext.scala:34) at org.http4s.blaze.util.SerialExecutionContext$$anon$1.act(SerialExecutionContext.scala:33) at org.http4s.blaze.util.Actor$RecycleableRunnable.go$1(Actor.scala:73) at org.http4s.blaze.util.Actor$RecycleableRunnable.run(Actor.scala:86) at org.http4s.blaze.util.Execution$$anon$3.execute(Execution.scala:80) at org.http4s.blaze.util.Actor.$bang(Actor.scala:61) at org.http4s.blaze.util.SerialExecutionContext.execute(SerialExecutionContext.scala:40) at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429) at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete0(Promise.scala:285) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:278) at scala.concurrent.Promise.complete(Promise.scala:57) at scala.concurrent.Promise.complete$(Promise.scala:56) at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:104) at scala.concurrent.Promise.success(Promise.scala:91) at scala.concurrent.Promise.success$(Promise.scala:91) at scala.concurrent.impl.Promise$DefaultPromise.success(Promise.scala:104) at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$2.completed(ByteBufferHead.scala:107) at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$2.completed(ByteBufferHead.scala:88) at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127) at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishRead(UnixAsynchronousSocketChannelImpl.java:437) at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:191) at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213) at java.base/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:306) at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2023-03-30 19:57:34,926 cromwell-system-akka.dispatchers.backend-dispatcher-133 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.read_genome_tsv:NA:1]: Status change from WaitingForReturnCode to Done 2023-03-30 19:57:41,499 cromwell-system-akka.dispatchers.engine-dispatcher-128 INFO - WorkflowExecutionActor-d93f9fdb-5967-4a13-8fd9-d4058768107f [UUID(d93f9fdb)]: Starting atac.align (2 shards) 2023-03-30 19:57:42,955 cromwell-system-akka.dispatchers.engine-dispatcher-116 INFO - Assigned new job execution tokens to the following groups: d93f9fdb: 1 2023-03-30 19:57:44,955 cromwell-system-akka.dispatchers.engine-dispatcher-116 INFO - Assigned new job execution tokens to the following groups: d93f9fdb: 1 2023-03-30 19:57:44,976 cromwell-system-akka.dispatchers.engine-dispatcher-128 INFO - BT-322 d93f9fdb:atac.align:0:1 is eligible for call caching with read = true and write = true 2023-03-30 19:57:44,988 cromwell-system-akka.dispatchers.engine-dispatcher-88 INFO - BT-322 d93f9fdb:atac.align:0:1 cache hit copying nomatch: could not find a suitable cache hit. 2023-03-30 19:57:44,989 cromwell-system-akka.dispatchers.engine-dispatcher-88 INFO - d93f9fdb-5967-4a13-8fd9-d4058768107f-EngineJobExecutionActor-atac.align:0:1 [UUID(d93f9fdb)]: Could not copy a suitable cache hit for d93f9fdb:atac.align:0:1. No copy attempts were made. 2023-03-30 19:57:44,991 cromwell-system-akka.dispatchers.backend-dispatcher-139 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.align:0:1]: Unrecognized runtime attribute keys: preemptible, disks, cpu, time, memory 2023-03-30 19:57:45,092 cromwell-system-akka.dispatchers.backend-dispatcher-139 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.align:0:1]: `set -e

check if pipeline dependencies can be found

if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline environment (docker, singularity or conda) not found." 1>&2 exit 3 fi

trim adapter

python3 $(which encode_task_trim_adapter.py) \ /cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/write_tsv_ac4a429a8891a88c36a56fd35d83f401.tmp \ \ --adapters /cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6

align on trimmed/merged fastqs

if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/inputs/-2047100455/Equus_caballus.EquCab3.0.dna.toplevel.fa.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 6.823947537913918 \ --nth 6 fi

python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name MT \ --chrsz /cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/inputs/-117978071/horse-annotation.chrom.sizes \ --mem-gb 6.823947537913918 \ --nth 6 rm -rf R1 R2` 2023-03-30 19:57:45,113 cromwell-system-akka.dispatchers.backend-dispatcher-139 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.align:0:1]: executing: rm -f /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid

if [ 'true' == 'true' ] && [ 'conda' == 'docker' ] || \ [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'encodedcc/atac-seq-pipeline@sha256:4404cfb32a99839b727537547a4c64839b6fa6c7050fb01e3ed2d53a19e40c30' ] then docker run -i --cidfile=/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid --user= --entrypoint=/bin/bash \ --volume=/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0:/cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0:delegated encodedcc/atac-seq-pipeline@sha256:4404cfb32a99839b727537547a4c64839b6fa6c7050fb01e3ed2d53a19e40c30 /cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/script rc=$(docker wait cat /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid) docker rm cat /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid else

recover GID lost due to Cromwell running chmod 777 on CWD

chown :`stat -c '%G' /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0` -R /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0
chmod g+s /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0

if [ 'true' == 'true' ] && [ 'conda' == 'singularity' ] || \
   [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif' ]
then
    mkdir -p $HOME/.singularity/lock/
    flock --exclusive --timeout 600 \
        $HOME/.singularity/lock/`echo -n 'https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif' | md5sum | cut -d' ' -f1` \
        singularity exec --containall https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif echo 'Successfully pulled https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif'

    singularity exec --cleanenv --home=`dirname /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0` \
        --bind=,/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0:/cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0 \
         \
        https://encode-pipeline-singularity-image.s3.us-west-2.amazonaws.com/atac-seq-pipeline_v2.2.1.sif /bin/bash /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/script & echo $! > /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid
else
    # remap paths between inside and outside of a docker container
    shopt -s nullglob
    sed -i 's#/cromwell-executions/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0#/public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0#g' /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/script `dirname /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/script`/write_*.tmp

    if [ 'true' == 'true' ] && [ 'conda' == 'conda' ] || \
       [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'encd-atac' ]
    then
        conda run --name=encd-atac /bin/bash /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/script & echo $! > /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid
    else
        /bin/bash /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/script & echo $! > /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid
    fi
fi

touch /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid.not_docker
wait `cat /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/call-align/shard-0/execution/docker_cid`
rc=`echo $?`

fi

exit $rc 2023-03-30 19:57:45,848 cromwell-system-akka.dispatchers.backend-dispatcher-138 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.align:0:1]: job id: 202640 2023-03-30 19:57:45,854 cromwell-system-akka.dispatchers.backend-dispatcher-138 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.align:0:1]: Status change from - to WaitingForReturnCode 2023-03-30 23:07:27,775 cromwell-system-akka.dispatchers.backend-dispatcher-180 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d93f9fdb)atac.align:0:1]: Status change from WaitingForReturnCode to Done 2023-03-30 23:07:29,723 cromwell-system-akka.dispatchers.engine-dispatcher-97 INFO - WorkflowManagerActor: Workflow d93f9fdb-5967-4a13-8fd9-d4058768107f failed (during ExecutingWorkflowState): cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align: Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

2023-03-30 23:07:30,864 cromwell-system-akka.dispatchers.engine-dispatcher-42 INFO - WorkflowManagerActor: Workflow actor for d93f9fdb-5967-4a13-8fd9-d4058768107f completed with status 'Failed'. The workflow will be removed from the workflow store. 2023-03-30 23:08:14,986 cromwell-system-akka.dispatchers.engine-dispatcher-99 INFO - SingleWorkflowRunnerActor workflow finished with status 'Failed'. 2023-03-30 23:08:16,194 cromwell-system-akka.dispatchers.engine-dispatcher-39 INFO - SingleWorkflowRunnerActor writing metadata to /public/home/jiahsun/02.ATAC_seq/02.caper_result/.caper_tmp/atac/20230330_195659_549883/metadata.json 2023-03-30 23:08:16,281 INFO - Workflow polling stopped 2023-03-30 23:08:16,449 INFO - 0 workflows released by cromid-4fc1f55 2023-03-30 23:08:16,452 INFO - Shutting down WorkflowStoreActor - Timeout = 5 seconds 2023-03-30 23:08:16,455 INFO - Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds 2023-03-30 23:08:16,543 INFO - Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds 2023-03-30 23:08:16,544 cromwell-system-akka.dispatchers.engine-dispatcher-108 INFO - Aborting all running workflows. 2023-03-30 23:08:16,544 INFO - JobExecutionTokenDispenser stopped 2023-03-30 23:08:16,544 INFO - WorkflowStoreActor stopped 2023-03-30 23:08:16,550 INFO - WorkflowLogCopyRouter stopped 2023-03-30 23:08:16,550 INFO - Shutting down WorkflowManagerActor - Timeout = 3600 seconds 2023-03-30 23:08:16,551 cromwell-system-akka.dispatchers.engine-dispatcher-110 INFO - WorkflowManagerActor: All workflows finished 2023-03-30 23:08:16,551 INFO - WorkflowManagerActor stopped 2023-03-30 23:08:16,800 INFO - Connection pools shut down 2023-03-30 23:08:16,804 INFO - Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds 2023-03-30 23:08:16,804 INFO - Shutting down JobStoreActor - Timeout = 1800 seconds 2023-03-30 23:08:16,804 INFO - Shutting down CallCacheWriteActor - Timeout = 1800 seconds 2023-03-30 23:08:16,804 INFO - Shutting down ServiceRegistryActor - Timeout = 1800 seconds 2023-03-30 23:08:16,804 INFO - Shutting down DockerHashActor - Timeout = 1800 seconds 2023-03-30 23:08:16,804 INFO - Shutting down IoProxy - Timeout = 1800 seconds 2023-03-30 23:08:16,804 INFO - SubWorkflowStoreActor stopped 2023-03-30 23:08:16,805 INFO - CallCacheWriteActor Shutting down: 0 queued messages to process 2023-03-30 23:08:16,805 INFO - JobStoreActor stopped 2023-03-30 23:08:16,805 INFO - KvWriteActor Shutting down: 0 queued messages to process 2023-03-30 23:08:16,805 INFO - WriteMetadataActor Shutting down: 0 queued messages to process 2023-03-30 23:08:16,805 INFO - CallCacheWriteActor stopped 2023-03-30 23:08:16,806 INFO - IoProxy stopped 2023-03-30 23:08:16,807 INFO - DockerHashActor stopped 2023-03-30 23:08:16,808 INFO - ServiceRegistryActor stopped 2023-03-30 23:08:16,814 INFO - Database closed 2023-03-30 23:08:16,815 INFO - Stream materializer shut down 2023-03-30 23:08:16,818 INFO - WDL HTTP import resolver closed 2023-03-30 23:08:16,826 INFO - Shutting down connection pool: curAllocated=1 idleQueues.size=1 waitQueue.size=0 maxWaitQueueLimit=256 closed=false 2023-03-30 23:08:16,826 INFO - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false 2023-03-30 23:08:16,826 INFO - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false

stderr output 023-03-30 19:56:59,525|caper.cli|INFO| Cromwell stdout: /public/home/jiahsun/02.ATAC_seq/04.command/cromwell.out 2023-03-30 19:56:59,577|caper.caper_base|INFO| Creating a timestamped temporary directory. /public/home/jiahsun/02.ATAC_seq/02.caper_result/.caper_tmp/atac/20230330_195659_549883 2023-03-30 19:56:59,577|caper.caper_runner|INFO| Localizing files on work_dir. /public/home/jiahsun/02.ATAC_seq/02.caper_result/.caper_tmp/atac/20230330_195659_549883 2023-03-30 19:57:00,326|caper.caper_workflow_opts|INFO| Conda environment name found in WDL metadata. wdl=/public/home/jiahsun/GitPublish/atac-seq-pipeline-2.2.1/atac.wdl, s=encd-atac 2023-03-30 19:57:00,331|caper.cromwell|INFO| Validating WDL/inputs/imports with Womtool... 2023-03-30 19:57:04,323|caper.nb_subproc_thread|INFO| Subprocess finished successfully. 2023-03-30 19:57:04,324|caper.cromwell|INFO| Passed Womtool validation. 2023-03-30 19:57:04,325|caper.caper_runner|INFO| launching run: wdl=/public/home/jiahsun/GitPublish/atac-seq-pipeline-2.2.1/atac.wdl, inputs=/public/home/jiahsun/02.ATAC_seq/04.command/BMM_R1_R2_XFM-2_R1_R2_input.json, backend_conf=/public/home/jiahsun/02.ATAC_seq/02.caper_result/.caper_tmp/atac/20230330_195659_549883/backend.conf 2023-03-30 19:57:11,022|caper.cromwell_workflow_monitor|INFO| Workflow: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, status=Submitted 2023-03-30 19:57:11,024|caper.cromwell_workflow_monitor|INFO| Workflow: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, status=Running 2023-03-30 19:57:25,866|caper.cromwell_workflow_monitor|INFO| Task: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, task=atac.read_genome_tsv:-1, retry=0, status=Started, job_id=201284 2023-03-30 19:57:25,872|caper.cromwell_workflow_monitor|INFO| Task: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, task=atac.read_genome_tsv:-1, retry=0, status=WaitingForReturnCode 2023-03-30 19:57:34,926|caper.cromwell_workflow_monitor|INFO| Task: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, task=atac.read_genome_tsv:-1, retry=0, status=Done 2023-03-30 19:57:45,849|caper.cromwell_workflow_monitor|INFO| Task: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, task=atac.align:0, retry=0, status=Started, job_id=202640 2023-03-30 19:57:45,855|caper.cromwell_workflow_monitor|INFO| Task: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, task=atac.align:0, retry=0, status=WaitingForReturnCode 2023-03-30 23:07:27,795|caper.cromwell_workflow_monitor|INFO| Task: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, task=atac.align:0, retry=0, status=Done 2023-03-30 23:07:29,723|caper.cromwell_workflow_monitor|INFO| Workflow: id=d93f9fdb-5967-4a13-8fd9-d4058768107f, status=Failed 2023-03-30 23:08:17,253|caper.cromwell_metadata|INFO| Wrote metadata file. /public/home/jiahsun/02.ATAC_seq/02.caper_result/atac/d93f9fdb-5967-4a13-8fd9-d4058768107f/metadata.json 2023-03-30 23:08:17,254|caper.cromwell|INFO| Workflow failed. Auto-troubleshooting... 2023-03-30 23:08:17,254|caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1 2023-03-30 23:08:17,255|caper.cli|ERROR| Check stdout in /public/home/jiahsun/02.ATAC_seq/04.command/cromwell.out

The output (if any) follows:

==== NAME=atac.align, STATUS=Failed, PARENT= SHARD_IDX=1, RC=None, JOB_ID=None START=None, END=None STDOUT=None STDERR=None

leepc12 commented 1 year ago

It looks like FASTQs definition (e.g. atac.fastqs_rep1_R1) in your input JSON is wrong. Please post your input JSON file.