djay / covidthailand

Thailand Covid testing and case data gathered and combined from various sources for others to download or view
126 stars 15 forks source link

test plots out of date as source of testing reports switched from nextcloud/webdav to google drive #169

Closed djay closed 2 years ago

djay commented 2 years ago

Links are on https://www3.dmsc.moph.go.th/

Drive location is https://drive.google.com/drive/folders/1yUVwstf5CmdvBVtKBs0uReV0BTbjQYlT

Looks like the reports are the same so should be just a matter of switching out the nextcloud webdav code for google drive api code and ensuring the files are returned latest first.

reduxionist commented 2 years ago

I will get to this on the weekend... 🙂

reduxionist commented 2 years ago

Since there is talk of a refactor, I would like to make a note that this is the kind of task that would be make more comfortable for the dev if there were unit tests to back up what he was trying to do... 🙂

reduxionist commented 2 years ago

As you suspected @djay , changing from a webdav API to the official Google Drive API requires use to use a GCP project with the Drive API enabled. While I don't expect our volume of usage to incur any cost (if it does I'll be happy to pay reasonable fees myself), it does mean we need to create a profile with the appropriate ownership for this project. I think you should own the profile used in this case, don't you agree? If so, please create the needed profile and either setup Drive API access or just share the credentials with me via Discord DM and I'll setup the rest. If you don't care who owns it, then I'll be happy to create the account myself and then just pass the credentials along to you once it's all setup... Thanks! 🙂

reduxionist commented 2 years ago

For now I've just gone ahead and created a new Google developer project under my own account to create API access. 🙂

djay commented 2 years ago

seems it needs any any key only https://stackoverflow.com/questions/47041929/does-google-drive-api-allow-us-to-download-public-file-without-authentication

djay commented 2 years ago

@reduxionist actually it doesn't need any api key. you can browse and download in an anonymous session so I'm sure there is a way with no account or at the very least any api key - https://stackoverflow.com/questions/18116152/how-do-i-get-a-file-list-for-a-google-drive-public-hosted-folder

reduxionist commented 2 years ago

@djay That's great if you enjoy spelunking and figuring things out on your own; I was following the instructions in Google's documentation which very much wants you to use an API key. I think we have different ways of learning new things. In any case, that link you posted still requires you to generate an API key in exactly the same manner I just said I did.

reduxionist commented 2 years ago

Also, yes, I'm sure we could scrape and download the files anonymously without an API key -- but then we'd be scraping again and at the mercy of random changes in wording or layout of links, whereas by using the API method we're (somewhat) protected from such changes as they at least have to announce that they're making breaking changes to their API when they do so.

djay commented 2 years ago

@reduxionist but no paymet required and switch the api key anytime

reduxionist commented 2 years ago

Right I didn't make any payment, and switching the API is the same -- you're still doing it under an account -- I only asked whether you wanted to the account to be yours or mine. It's not "any api key" as in an anonymous API key, just because the resource is anonymous does not mean the API key doesn't still have to belong to someone...

djay commented 2 years ago

Sure. Once it's working I'll switch it to mine

djay commented 2 years ago

Will do this

djay commented 2 years ago

fixed in #171