search

geodesymiami / insarmaps

3 stars 0 forks source link

new bug in json_mbtiles2insarmaps.py #35

Closed falkamelung closed 3 years ago

falkamelung commented 3 years ago

When I try to ingest after updating the ingest scripts i get the following message. It seems to be checking whetehr the dataset exist what it should not do.

json_mbtiles2insarmaps.py -u insaradmin -p Insar123 --host insarmaps.miami.edu -P rsmastest -U rsmas\@gmail.com --json_folder /scratch/05861/tg851601/unittestGalapagosSenDT128/mintpy/JSON --mbtiles_file /scratch/05861/tg851601/unittestGalapagosSenDT128/mintpy/JSON/S1_IW1_128_0596_0597_20160605_XXXXXXXX.mbtiles
Uploading json chunks...
Clearing old dataset, if it is there
Traceback (most recent call last):
  File "/scratch/05861/tg851601/code/rsmas_insar/sources/insarmaps_scripts/json_mbtiles2insarmaps.py", line 205, in <module>
    main()
  File "/scratch/05861/tg851601/code/rsmas_insar/sources/insarmaps_scripts/json_mbtiles2insarmaps.py", line 161, in main
    upload_json(parseArgs.json_folder)
  File "/scratch/05861/tg851601/code/rsmas_insar/sources/insarmaps_scripts/json_mbtiles2insarmaps.py", line 83, in upload_json
    attributesController.remove_dataset_if_there(area_name)
  File "/scratch/05861/tg851601/code/rsmas_insar/sources/insarmaps_scripts/add_attribute_insarmaps.py", line 200, in remove_dataset_if_there
    % (unavco_name))
Exception: Dataset S1_IW1_128_0596_0597_20160605_XXXXXXXX is not on insarmaps, so couldn't remove it
//login3/scratch/05861/tg851601/unittestGalapagosSenDT128[1075]
stackTom commented 3 years ago

About to work on this. I am using this dataset to test: S1_IW1_128_0593_0597_20141213_XXXXXXXX

I have an old copy of the he5 file on our server. I will use that to test. Not sure if it is the latest data for that dataset, so you might want to re-upload after I fix this bug. I don't know if I ever got access to pegasus so I can get a more recent file.

stackTom commented 3 years ago

Actually, the files on the server are too old and dont seem to work. Can you transfer one of the latest h5 files that works with the latest scripts. Ideally a very small one. Just put it on dropbox or google drive, etc, and please send me a link so I can debug this code with the scripts.

stackTom commented 3 years ago

I just pushed a potential fix. Please try it. I couldn't try it as I don't have an h5 file, but it was simple enough that it might do the trick. If it works, we can close this issue. If not, please let me know.

Also please send me a small h5 file that I can have so I can run these scripts and be able to debug them.

falkamelung commented 3 years ago

It works. Thank you! The data are on jetstream but this will be down until monday, probably. If you need before let me know and I put on Google Drive.

stackTom commented 3 years ago

Sounds good. I can just wait for jet stream. I only really need an h5 file if more work on the scripts is needed.