EBISPOT / gwas-sumstats-service

Summary statistics service
2 stars 0 forks source link

Fix new Celery ConnectTimeoutError on request Error #286

Closed sprintell closed 4 months ago

sprintell commented 7 months ago

globus_sdk.exc.convert.GlobusConnectionTimeoutError: ConnectTimeoutError on request Task was called with args: [{'callbackID': 'YAkVbr5r', 'studyList': [{'id': '65bbe83d53c4ef0001851868', 'gcst': 'GCST90310248', 'pmid': None, 'file_path': 'EWAS_PUMMA_SBP-update.tsv.gz', 'assembly': 'GRCh37', 'callback_id': 'YAkVbr5r', 'readme': 'README_GWAScatalog.txt', 'entryUUID': '7ce9dd58-04e3-4518-ae26-971fe35ac474', 'author_name': None, 'rawSS': None, 'md5': '809bec5de962bfb69af8ef221b18dc77'}, {'id': '65bbe83d53c4ef0001851869', 'gcst': 'GCST90310249', 'pmid': None, 'file_path': 'EWAS_PUMMA_DBP-update.tsv.gz', 'assembly': 'GRCh37', 'callback_id': 'YAkVbr5r', 'readme': 'README_GWAScatalog.txt', 'entryUUID': '7ce9dd58-04e3-4518-ae26-971fe35ac474', 'author_name': None, 'rawSS': None, 'md5': 'd16c11d094a93e72f994b7d92225aebc'},...

karatugo commented 5 months ago

When trying to read or write, like deleting a Globus collection, requests time out. This issue doesn't happen in my local development environment, suggesting a network problem in HPC. So, I contacted the HPC team for help with these network issues.

sprintell commented 5 months ago

@karatugo will create a service-now ticket so the issue can be properly tracked with the TSC.

karatugo commented 5 months ago

The Codon team responded on Slack, so I didn't create a ServiceNow ticket. They'll consult with the Globus team about a possible restriction. The Slack thread is available here: https://gwas-catalog.slack.com/archives/C042J8B3F33/p1712161422416529.

karatugo commented 5 months ago

The Codon team asked me to create a ServiceNow ticket: https://embl.service-now.com/esc?id=ticket&table=incident&sys_id=f956d0581bad42d0a32b4043b24bcbfa

karatugo commented 5 months ago
karatugo commented 5 months ago

Tested HTTP traffic issues on Codon and local environments; the commands work as expected in the terminal. But still facing the problem with Celery workers when performing get_collection() or delete_collection() operations, resulting in a ConnectionResetError during task execution. Asked for help from the Codon team.