search

fabric8-analytics / fabric8-analytics-worker

fabric8-analytics worker for gathering raw data
GNU General Public License v3.0
8 stars 45 forks source link

IndexError: list index out of range for file_url = release_files[0]['url'] #81

Open fridex opened 7 years ago

fridex commented 7 years ago

I can see following errors on our staging instance when scheduled selective flows that run just GraphImorterTask:

{"queue": "STAGE_ingestion_InitAnalysisFlow_v0", "task_name": "InitAnalysisFlow", "flow_name": "bayesianFlow", "dispatcher_id": "54a8e41e-b146-44f6-9452-cf2dab483e14", "parent": {}, "node_args": {"version": "0.1", "force": true, "ecosystem": "pypi", "name": "countrysubdomains"}, "task_id": "762ac67a-0c50-432f-8443-bf3e8fca2014", "event": "TASK_START"}
2017-06-26 06:16:52,988 [ERROR] cucoslib.object_cache: Failed to retrieve pypi/countrysubdomains/0.1/meta.json
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/cucoslib/object_cache.py", line 69, in _get_meta
    self._meta = self._s3.retrieve_dict(self._meta_json_object_key)
  File "/usr/lib/python3.4/site-packages/cucoslib/storages/s3.py", line 195, in retrieve_dict
    return json.loads(self.retrieve_blob(object_key).decode())
  File "/usr/lib/python3.4/site-packages/cucoslib/storages/s3.py", line 191, in retrieve_blob
    return self._s3.Object(self.bucket_name, object_key).get()['Body'].read()
  File "/usr/lib/python3.4/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/lib/python3.4/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/usr/lib/python3.4/site-packages/botocore/client.py", line 253, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3.4/site-packages/botocore/client.py", line 543, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
{"queue": "STAGE_ingestion_InitAnalysisFlow_v0", "node_args": {"version": "0.1", "force": true, "ecosystem": "pypi", "name": "countrysubdomains"}, "flow_name": "bayesianFlow", "parent": {}, "what": "Traceback (most recent call last):\n  File \"/usr/lib/python3.4/site-packages/selinon/selinonTaskEnvelope.py\", line 114, in run\n    result = task.run(node_args)\n  File \"/usr/lib/python3.4/site-packages/cucoslib/base.py\", line 38, in run\n    result = self.execute(node_args)\n  File \"/usr/lib/python3.4/site-packages/cucoslib/workers/init_analysis_flow.py\", line 47, in execute\n    target_dir=cache_path\n  File \"/usr/lib/python3.4/site-packages/cucoslib/process.py\", line 260, in fetch_artifact\n    file_url = release_files[0]['url']\nIndexError: list index out of range\n", "retried_count": 0, "task_name": "InitAnalysisFlow", "dispatcher_id": "54a8e41e-b146-44f6-9452-cf2dab483e14", "task_id": "762ac67a-0c50-432f-8443-bf3e8fca2014", "event": "TASK_FAILURE"}
{"node_args": {"version": "0.1", "force": true, "ecosystem": "pypi", "name": "countrysubdomains"}, "flow_name": "bayesianFlow", "storage_task_name": "InitAnalysisFlow", "record_id": null, "task_name": "InitAnalysisFlow", "error_traceback": "  File \"/usr/lib/python3.4/site-packages/selinon/selinonTaskEnvelope.py\", line 114, in run\n    result = task.run(node_args)\n  File \"/usr/lib/python3.4/site-packages/cucoslib/base.py\", line 38, in run\n    result = self.execute(node_args)\n  File \"/usr/lib/python3.4/site-packages/cucoslib/workers/init_analysis_flow.py\", line 47, in execute\n    target_dir=cache_path\n  File \"/usr/lib/python3.4/site-packages/cucoslib/process.py\", line 260, in fetch_artifact\n    file_url = release_files[0]['url']\n", "storage_name": "BayesianPostgres", "task_id": "762ac67a-0c50-432f-8443-bf3e8fca2014", "error_value": "list index out of range", "event": "STORAGE_STORE_ERROR", "error_type": "<class 'IndexError'>"}
2017-06-26 06:16:53,209 [ERROR] celery.app.trace: Task selinon.SelinonTaskEnvelope[762ac67a-0c50-432f-8443-bf3e8fca2014] raised unexpected: IndexError('list index out of range',)
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/celery/app/trace.py", line 367, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/celery/app/trace.py", line 622, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/selinon/selinonTaskEnvelope.py", line 169, in run
    raise self.retry(max_retries=0, exc=exc)
  File "/usr/lib/python3.4/site-packages/celery/app/task.py", line 668, in retry
    raise_with_context(exc)
  File "/usr/lib/python3.4/site-packages/selinon/selinonTaskEnvelope.py", line 114, in run
    result = task.run(node_args)
  File "/usr/lib/python3.4/site-packages/cucoslib/base.py", line 38, in run
    result = self.execute(node_args)
  File "/usr/lib/python3.4/site-packages/cucoslib/workers/init_analysis_flow.py", line 47, in execute
    target_dir=cache_path
  File "/usr/lib/python3.4/site-packages/cucoslib/process.py", line 260, in fetch_artifact
    file_url = release_files[0]['url']
IndexError: list index out of range
fridex commented 7 years ago

It looks like the artifact was deleted from PyPI - https://pypi.python.org/pypi/CountrySubdomains/0.1

pip3 install CountrySubdomains==0.1
Collecting CountrySubdomains==0.1
  Could not find a version that satisfies the requirement CountrySubdomains==0.1 (from versions: )
No matching distribution found for CountrySubdomains==0.1