Start failed after "Downloading data for deleted images on amazon"

st-man commented 9 months ago

loaded 16277 items from cache (/home/user/.cache/scc-tools/scc_data.json) loaded 5 items from cache (/home/user/.cache/scc-tools/public_cloud_amazon.json)

cached data for amazon is too old, downloading -- Downloading data for active images on amazon... -- Downloading data for inactive images on amazon... -- Downloading data for deprecated images on amazon... -- Downloading data for deleted images on amazon... got non-fatal reply (502) from server, trying again in 5 seconds got non-fatal reply (502) from server, trying again in 5 seconds got non-fatal reply (502) from server, trying again in 5 seconds got non-fatal reply (502) from server, trying again in 5 seconds got non-fatal reply (502) from server, trying again in 5 seconds Traceback (most recent call last): File "/home/user/Downloads/scc-tools/vercheck.py", line 2006, in
main() File "/home/user/Downloads/scc-tools/vercheck.py", line 1435, in main pc = PublicCloudCheck() ^^^^^^^^^^^^^^^^^^ File "/home/user/Downloads/scc-tools/vercheck.py", line 1794, in init self.aws_cm = PublicImageCacheManager(provider='amazon') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/Downloads/scc-tools/vercheck.py", line 1676, in init self.cache_data = self.get_image_states(provider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/Downloads/scc-tools/vercheck.py", line 1778, in get_image_states image_data['deleted'] = self.fetch_image_states(provider, 'deleted') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/Downloads/scc-tools/vercheck.py", line 1769, in fetch_image_states return return_data['images']
```
KeyError: 'images'
```

doccaz commented 9 months ago

Indeed, I never considered that the pint database could be offline. I'll change the logic to ignore that on the first try and use the cached data instead.

doccaz commented 9 months ago

There. I simulated non-fatal and fatal errors while downloading cloud data, and treated them accordingly.

st-man commented 8 months ago

Now it's like this: scc-tools/vercheck.py -dvf ./scc_dir/ loaded 8613 items from cache (/home/user/.cache/scc-tools/scc_data.json)

cached data for amazon does not exist, downloading -- Downloading data for active images on amazon... -- Downloading data for inactive images on amazon... -- Downloading data for deprecated images on amazon... -- Downloading data for deleted images on amazon... got non-fatal reply (502) from server, trying again in 5 seconds cannot download cloud data at the moment, will use cached data.
cached data for google does not exist, downloading -- Downloading data for active images on google... -- Downloading data for inactive images on google... -- Downloading data for deprecated images on google... -- Downloading data for deleted images on google...
cached data for microsoft does not exist, downloading -- Downloading data for active images on microsoft... -- Downloading data for inactive images on microsoft... -- Downloading data for deprecated images on microsoft... -- Downloading data for deleted images on microsoft... Traceback (most recent call last): File "/home/user/scc-tools/vercheck.py", line 2030, in
main() File "/home/user/scc-tools/vercheck.py", line 1435, in main pc = PublicCloudCheck() ^^^^^^^^^^^^^^^^^^ File "/home/user/scc-tools/vercheck.py", line 1826, in init print(f"--- AMAZON data as of {self.aws_image_data['timestamp']}")
```
TypeError: list indices must be integers or slices, not str
```

doccaz commented 8 months ago

Sorry, reopening issue.

doccaz commented 8 months ago

Interesting, we have a unique situation here: only ONE report is causing a permanent 502 error while downloading the public image information from PINT. I'm adding proper error handling on this case.

doccaz commented 8 months ago

Could you test this again, @st-man ?

doccaz commented 8 months ago

Haven't had error reports after the change, closing the issue.

doccaz / scc-tools

Start failed after "Downloading data for deleted images on amazon" #33