search

projectglow / glow

An open-source toolkit for large-scale genomic analysis
https://projectglow.io
Apache License 2.0
262 stars 107 forks source link

AnalysisException: Column 'num_workers' does not exist. #528

Closed mwarqee closed 5 months ago

mwarqee commented 1 year ago

While running 0_ingest_vcf2delta I get the following error message:

The execution of this command did not finish successfully

AnalysisException: Column 'num_workers' does not exist. Did you mean one of the following? [jdbc_port, cluster_id, node_type_id, state, cluster_name, cluster_source, spark_version, start_time, runtime_engine, state_message, cluster_memory_mb, creator_user_name, single_user_name, spark_context_id, terminated_time, data_security_mode, driver_node_type_id, last_restarted_time, effective_spark_version, last_activity_time, last_state_loss_time, autotermination_minutes];

williambrandler commented 1 year ago

hi @mwarqee, this notebook depends on the four upstream ETL notebooks being run in order beforehand

tertiary/0_ingest_vcf2delta depends on etl/3_delta_to_vcf

which in turn depends on the other three etl notebooks, (etl/data/download_1000G, etl/1_simulate_covariates_phenotypes_offset, etl/2_simulate_delta_pvcf)