search

aaronwmorris / indi-allsky

Software to manage a Linux-based All Sky Camera.
GNU General Public License v3.0
256 stars 42 forks source link

Error for GCP Upload Bucket after upgrading to indi_v2024.05.1 #1420

Closed gaitskell closed 4 months ago

gaitskell commented 4 months ago

After after upgrading to indi_v2024.05.1 the fits files are no longer uploading to GCP (Google Cloud Platform).

I am getting the following errors associated with the GCP upload process: 

Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception: ValueError: PosixPath('ccd_f3ee6189-ab30-44e0-a7a7-75ec905af83c/fits/20240725/day/25_12/ccd1_20240725_125944.fit') could not be converted to unicode

...

Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     blob = gcp_bucket.blob(key)

The credentials file and GCP permissions are unchanged and appear to be in order.

Is there something I can do to help diagnoze this issue?

Do you know what the new cause may be?

Rick

The full relevant details from the log are here

VERSION

Jul 25 12:59:06 PiAllSky [INFO] MainProcess-4178/MainThread allsky._startup() [281]: Python version: 3.9.2
Jul 25 12:59:06 PiAllSky [INFO] MainProcess-4178/MainThread allsky._startup() [279]: indi-allsky config level: 20240620.0
Jul 25 12:59:06 PiAllSky [INFO] MainProcess-4178/MainThread allsky._startup() [278]: indi-allsky release: indi_v2024.05.1

FULL LOG DURING GCP UPLOAD ERROR

Jul 25 12:59:59 PiAllSky [INFO] Capture-1-4183/MainThread indi.getTelescopeRaDec() [874]: Telescope Coord: RA 8.51, Dec 41.74
Jul 25 12:59:59 PiAllSky [WARNING] Capture-1-4183/MainThread indi.getCcdTemperature() [884]: Camera temperature not supported
Jul 25 12:59:59 PiAllSky [INFO] MainProcess-4178/MainThread allsky._queueManualTasks() [1211]: Checking for manually submitted tasks
Jul 25 12:59:59 PiAllSky [INFO] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [577]: Starting Upload-5 worker
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception: 
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception: ValueError: PosixPath('ccd_f3ee6189-ab30-44e0-a7a7-75ec905af83c/fits/20240725/day/25_12/ccd1_20240725_125944.fit') could not be converted to unicode
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     raise ValueError("%r could not be converted to unicode" % (value,))
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:   File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/google/cloud/_helpers/__init__.py", line 352, in _bytes_to_unicode
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     name = _bytes_to_unicode(name)
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:   File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/google/cloud/storage/blob.py", line 216, in __init__
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     return Blob(
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:   File "/home/admin/indi-allsky/virtualenv/indi-allsky/lib/python3.9/site-packages/google/cloud/storage/bucket.py", line 793, in blob
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     blob = gcp_bucket.blob(key)
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:   File "/home/admin/indi-allsky/indi_allsky/filetransfer/gcp_storage.py", line 79, in put
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     response = client.put(**put_kwargs)
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:   File "/home/admin/indi-allsky/indi_allsky/uploader.py", line 454, in processUpload
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     self.processUpload(u_dict)
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:   File "/home/admin/indi-allsky/indi_allsky/uploader.py", line 139, in saferun
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:     self.saferun()
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception:   File "/home/admin/indi-allsky/indi_allsky/uploader.py", line 108, in run
Jul 25 12:59:59 PiAllSky [ERROR] MainProcess-4178/MainThread allsky._fileUploadWorkerStart() [570]: Upload worker exception: Traceback (most recent call last):
Jul 25 12:59:59 PiAllSky [INFO] Capture-1-4183/MainThread capture.detectNight() [1322]: Moon altitude: -27.1, phase 76.6%
Jul 25 12:59:59 PiAllSky [INFO] Capture-1-4183/MainThread capture.detectNight() [1315]: Sun altitude: 67.6
Jul 25 12:59:59 PiAllSky [INFO] Capture-1-4183/MainThread capture.saferun() [219]: Exposure state: OK
Jul 25 12:59:59 PiAllSky [INFO] Capture-1-4183/MainThread capture.saferun() [218]: Camera last ready: 14.5s
Jul 25 12:59:55 PiAllSky [WARNING] MainProcess-4178/Sensor-1 sensor.night_day_change() [157]: Day/Night change
Jul 25 12:59:48 PiAllSky [INFO] Capture-1-4183/MainThread indi.getTelescopeRaDec() [874]: Telescope Coord: RA 8.50, Dec 41.74
Jul 25 12:59:48 PiAllSky [WARNING] Capture-1-4183/MainThread indi.getCcdTemperature() [884]: Camera temperature not supported
Jul 25 12:59:48 PiAllSky [INFO] Capture-1-4183/MainThread capture.detectNight() [1322]: Moon altitude: -27.0, phase 76.6%
Jul 25 12:59:48 PiAllSky [INFO] Capture-1-4183/MainThread capture.detectNight() [1315]: Sun altitude: 67.6
Jul 25 12:59:48 PiAllSky [INFO] Capture-1-4183/MainThread capture.saferun() [219]: Exposure state: OK
Jul 25 12:59:48 PiAllSky [INFO] Capture-1-4183/MainThread capture.saferun() [218]: Camera last ready: 3.2s
Jul 25 12:59:47 PiAllSky [INFO] MainProcess-4178/Upload-3 generic.put() [66]: Uploading /var/www/html/allsky/images/ccd_f3ee6189-ab30-44e0-a7a7-75ec905af83c/fits/20240725/day/25_12/ccd1_20240725_125944.fit
Jul 25 12:59:46 PiAllSky [INFO] MainProcess-4178/MainThread allsky._queueManualTasks() [1211]: Checking for manually submitted tasks
Jul 25 12:59:46 PiAllSky [INFO] Image-1-4185/MainThread image.write_img() [1337]: Daytime timelapse is disabled
Jul 25 12:59:46 PiAllSky [INFO] Image-1-4185/MainThread image.write_img() [1312]: Image compressed in 0.0451 s
Jul 25 12:59:46 PiAllSky [INFO] Image-1-4185/MainThread image.processImage() [625]: Image processed in 1.3636 s
Jul 25 12:59:46 PiAllSky [INFO] Image-1-4185/MainThread processing.apply_image_circle_mask() [1479]: Image circle mask in 0.0535 s
Jul 25 12:59:45 PiAllSky [WARNING] Image-1-4185/MainThread processing._generate_image_circle_mask() [2634]: Opacity disabled for image circle outline
Jul 25 12:59:45 PiAllSky [WARNING] Image-1-4185/MainThread image.recalculate_exposure() [1660]: New calculated exposure: 0.32327586
Jul 25 12:59:45 PiAllSky [INFO] Image-1-4185/MainThread processing._convert_16bit_to_8bit() [1084]: Resampling image from 16 to 8 bits
Jul 25 12:59:45 PiAllSky [INFO] MainProcess-4178/Upload-3 generic.connect() [52]: Connecting to storage.googleapis.com (443) as * with gcp_storage
Jul 25 12:59:45 PiAllSky [INFO] Image-1-4185/MainThread processing.contrast_clahe_16bit() [1395]: Performing 16-bit CLAHE contrast enhance
gaitskell commented 4 months ago

The bucket itself looks like this

image

aaronwmorris commented 4 months ago

In my laziness, I was just passing a Pathlib object as the s3 key and I guess some versions of the library dealt with that properly.

Merged #1422 to ensure the key is a str

gaitskell commented 4 months ago

As also said in meged thread - Thank you for resolving this - working