SONN1 or SONN3 as sources

[keenonkites]

I'm not sure if the downloadpart from Sonny is already running or if I cought it in an intermediate state.

It looks like I got the google drive authentication running, at least I do not get any errormessages related to this.

But when running the tool I get the following errormessages:

(venv_pyhgtmap) xyz@myhostname:~/develop/fzk-utils/elevation$ pyhgtmap --step=20 --osm-version=0.6 --jobs=1 --line-cat=500,100 --start-node-id=750000000000 --start-way-id=750000000000 --max-nodes-per-tile=0 --pbf --polygon=./poly.work/Freizeitkarte_LUX.poly --source=SONN3 --write-timestamp --output-prefix=./pbf/ele_20_100_500/Hoehendaten_Freizeitkarte_LUX
N49E005: trying sonn3 ...
Traceback (most recent call last):
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/sources/__init__.py", line 68, in get_file
    self.check_cached_file(file_name, resolution)
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/sources/__init__.py", line 49, in check_cached_file
    found_size: int = os.path.getsize(file_name)
  File "/usr/lib/python3.8/genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: 'hgt/SONN3/N49E005.hgt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xyz/venv_pyhgtmap/bin/pyhgtmap", line 8, in <module>
    sys.exit(main())
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/main.py", line 593, in main
    main_internal(sys.argv[1:])
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/main.py", line 574, in main_internal
    hgtDataFiles = NASASRTMUtil.getFiles(
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/NASASRTMUtil.py", line 915, in getFiles
    saveFilename = sources_pool.get_file(opener, area, source)
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/NASASRTMUtil.py", line 886, in get_file
    file_name = self._real_pool.get_source("sonn").get_file(
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/sources/__init__.py", line 77, in get_file
    self.download_missing_file(area, resolution, file_name)
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/sources/sonny.py", line 75, in download_missing_file
    files: List[GoogleDriveFile] = self.gdrive.ListFile(
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pyhgtmap/sources/sonny.py", line 61, in gdrive
    gauth.CommandLineAuth()
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pydrive2/auth.py", line 122, in _decorated
    self.LoadCredentials()
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pydrive2/auth.py", line 391, in LoadCredentials
    self.LoadCredentialsFile()
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/pydrive2/auth.py", line 418, in LoadCredentialsFile
    self.credentials = self._default_storage.get()
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/oauth2client/client.py", line 407, in get
    return self.locked_get()
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/oauth2client/file.py", line 54, in locked_get
    credentials = client.Credentials.new_from_json(content)
  File "/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/oauth2client/client.py", line 302, in new_from_json
    module_name = data['_module']
KeyError: '_module'
(venv_pyhgtmap) xyz@myhostname:~/develop/fzk-utils/elevation$

Can you tell me if and what I'm doing wrong ? ....

Cheers Patrik

[agrenott]

From the stack trace it seems it's failing while leading the client secrets file. Did you download it and put it in ~/.pyhgtmap/client-secret.json ? If it works you should get a request from the CLI to open a link to continue the OAuth process.

[keenonkites]

Downloaded, yes.... but if I have it with the name client-secret.json the system is complaining about missing gdrive-credentials.json and asks me for opening something on the browser and for verification code.... as I'm running on a server without gui, I can't open that there.... and if I do that on my local windows:

/home/xyz/venv_pyhgtmap/lib/python3.8/site-packages/oauth2client/_helpers.py:255: UserWarning: Cannot access /home/xyz/.pyhgtmap/gdrive-credentials.json: No such file or directory
  warnings.warn(_MISSING_FILE_MESSAGE.format(filename))
Go to the following link in your browser:

    XXXX

Enter verification code:

I most probably did something wrong, but can't figure out what.

[agrenott]

What you describe is the CLI OAuth workflow. The grdive-credentials file is where the oauth details are cached after first successful auth (to avoid doing it at each run). You have to copy paste the displayed link into any browser. This will redirect you too google OAuth process where you have to agree to have gdrive permission to your insurance of pyhgtmap. It will then provide you a code you have to copy paste into the prompt of pyhgtmap CLI.

[agrenott]

PS: as you warned me in the previous issue, gdrive is indeed a pain to work with... I couldn't find an easier way.

[keenonkites]

When I copy that to my browser on the local workstation I get following error message:

Access blocked: This app’s request is invalid
...
You can’t sign in because this app sent an invalid request. You can try again later or contact the developer about this issue. [Learn more about this error](https://developers.google.com/identity/protocols/oauth2/openid-connect#setredirecturi)
If you are a developer of this app, see [error details](https://accounts.google.com/).
Error 400: redirect_uri_mismatch

I'm sure I messed it up with the client-secret.json while creating it.... but I can't figure out what... Yes, looks like some sort of complicated... ;-)

BTW: thanks for your support anyway.... hopefully others can profit also...

[agrenott]

Did you follow those instructions to create your API credentials? https://docs.iterative.ai/PyDrive2/quickstart/#authentication The redirect_uri_mismatch makes me think you either didn't declare a desktop application, or you didn't out localhost as url.

[keenonkites]

I did follow the instructions.... probably too close:

• Select ‘Application type’ to be Web application.
• Enter an appropriate name.
• Input http://localhost:8080/ for ‘Authorized redirect URIs’.
• Click ‘Create’.

What did the trick is to create a desktop Application, you probably should mention that explicitly in the README.md of the repo.

Now I was able to create the code with a browser on my local machine, finish the autenthication properly and download some test data.

Thanks for your help again. I probably do further testing this evening (CET). Cheers Patrik