AlexsLemonade / refinebio

Refine.bio harmonizes petabytes of publicly available biological data into ready-to-use datasets for cancer researchers and AI/ML scientists.
https://www.refine.bio/
Other
126 stars 19 forks source link

Fix affymetrix "It appears that the file ... is corrupted" error #3236

Closed arkid15r closed 1 year ago

arkid15r commented 1 year ago

Context

In order to restore staging/prod deploy process CI/CD tests need to be fixed.

Problem or idea

Reading in : /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL
R[write to console]: Error in { :
  task 1 failed - "It appears that the file /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL is corrupted.
"

2023-03-09 16:08:14,433 local data_refinery_workers.processors.array_express ERROR [processor_job: 2]: Encountered error in R code while running AFFY_TO_PCL pipeline during processing of /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL: Error in { :
  task 1 failed - "It appears that the file /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL is corrupted.
"

2023-03-09 16:08:14,434 local data_refinery_workers.processors.utils ERROR [failure_reason: Encountered error in R code while running AFFY_TO_PCL pipeline during processing of /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL: Error in { :
  task 1 failed - "It appears that the file /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL is corrupted.
"
] [processor_job: 2]: Processor function _run_scan_upc failed. Terminating pipeline.
2023-03-09 16:08:14,464 local data_refinery_workers.processors.utils ERROR [pipeline_applied: AFFY_TO_PCL] [failure_reason: Encountered error in R code while running AFFY_TO_PCL pipeline during processing of /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL: Error in { :
  task 1 failed - "It appears that the file /home/user/data_store/raw/TEST/CEL/GSM1364667_U_110208_7-02-10_S2.CEL is corrupted.
"
] [processor_job: 2] [no_retry: True]: Processor job failed!

Solution or next step

Fix the corrupted file error.

arkid15r commented 1 year ago

Obsolete