capture app flavours should be built automatically by CI/CD

[sync-by-unito[bot]]

ALL flavours should be built automatically by CI/CD flavors :

• storage-qa-bubble-qa
• storage-qa-bubble-prod
• storage-prod-bubble-qa
• storage-prod-bubble-prod
• storage-prod-network action-qa

user story as Capture App qa tester, I want an ALL flavours capture app so that i can be tested under different branch at backend , ifram ,network action .

[Capture App] build flavour

┆Issue is synchronized with this Asana task by Unito ┆Created By: Scott Yan

[sync-by-unito[bot]]

➤ Sam commented:

Tammy Yang, just to clarify do we need those in firebase app distribution phase?

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Instead of building multiple flavors, here is my proposal. How does everyone think?

[sync-by-unito[bot]]

➤ Tammy Yang commented:

Sam no, only need AFTER code review (no need in Firebase phase)

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, interesting approach. It will cut github CI build times, google storage etc. I think it's good.

[sync-by-unito[bot]]

➤ Sam commented:

Before

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, do I need extra permissions to change GitHub CI related configs?

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Sam Yes, you can share the new key/value of the configs with me, and I will add them to the GitHub settings

[sync-by-unito[bot]]

➤ Bofu Chen commented:

The purpose of the multiple flavors https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255 ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255 )

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, by "key/value of the configs" do you mean NETWORK_ACTION_URL = "..."

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Next actions for Sam

1. This sprint

   1. Updating the CI script ( https://github.com/numbersprotocol/capture-lite/blob/master/.github/workflows/build-apks.yml ) for building the flavors in row 2 - 4 ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255 ), and sending the PR to Bofu
   2. Providing the new GitHub configs of Network Action to Bofu

2. Next sprint

   1. Updating the Capture App to use the new Network Action configs
   2. Updating the CI script ( https://github.com/numbersprotocol/capture-lite/blob/master/.github/workflows/build-apks.yml ) for building the flavors in row 5 ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255 ), and sending the PR to Bofu

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Sam You can share the value of the 2 new keys with me

NUMBERS_BUBBLE_NETWORK_ACTION_URL NUMBERS_BUBBLE_NETWORK_ACTION_QA_URL

[sync-by-unito[bot]]

➤ Scott Yan commented:

NETWORK_ACTION_URL : https://network-app.numbersprotocol.io/ ( https://network-app.numbersprotocol.io/ ) NETWORK_ACTION_QA_URL : https://network-app.numbersprotocol.io/version-qa-release/ ( https://network-app.numbersprotocol.io/version-qa-release/ )

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, regarding

Next actions for Sam

1. This sprint
   1. Updating the CI script ( https://github.com/numbersprotocol/capture-lite/blob/master/.github/workflows/build-apks.yml ) for building the flavors in row 2 - 4 ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255 ), and sending the PR to Bofu

Should CI script ( https://github.com/numbersprotocol/capture-lite/blob/master/.github/workflows/build-apks.yml ) build row 3 and 4 only? Because row 2 is also build here ( https://github.com/numbersprotocol/capture-lite/blob/abcef5b06bead1ba3abae482fca6a87573331665/.github/workflows/pre-release.yml#L9 ).

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Sam We cap keep row 2 - 4 because it will be convenient for QA to get all the flavors in one place.

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, should we keep PR ( https://github.com/numbersprotocol/capture-lite/pull/2554 ) for this task separate from milestone branch? Directly merge to develop?

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, can you please clarify what you mean by

1. Providing the new GitHub configs of Network Action to Bofu

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Sam Please double check if it's expected

NUMBERS_BUBBLE_NETWORK_ACTION_URL=https://network-app.numbersprotocol.io/ ( https://network-app.numbersprotocol.io/ ) NUMBERS_BUBBLE_NETWORK_ACTION_QA_URL=https://network-app.numbersprotocol.io/version-qa-release/ ( https://network-app.numbersprotocol.io/version-qa-release/ )

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, expected where? I'm bit confused.

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Sam For the PR, please set the merging target to the milestone branch.

We can trigger Capture CI by adding a tag to any branch. So merging to the milestone branch will not affect triggering Capture CI.

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Q:

Bofu Chen, can you please clarify what you mean by

1. Providing the new GitHub configs of Network Action to Bofu

A: NUMBERS_BUBBLE_NETWORK_ACTION_URL=https://network-app.numbersprotocol.io/ ( https://network-app.numbersprotocol.io/ ) NUMBERS_BUBBLE_NETWORK_ACTION_QA_URL=https://network-app.numbersprotocol.io/version-qa-release/ ( https://network-app.numbersprotocol.io/version-qa-release/ )

—- Please double-check if the values are correct. You will need to use the two configs in Capture App in the next sprint.

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, they are provided by Scott so I assume they are correct. Also in next sprint before PRs I will check if capture app can work with given urls.

Sorry for confusions 😅

[sync-by-unito[bot]]

➤ Tammy Yang commented:

Sam just want to confirm if this is done? (found it in the old sprint but still open)

[sync-by-unito[bot]]

➤ Sam commented:

Tammy Yang, from previous comment ( https://app.asana.com/0/0/1203756670523020/1203881285492937/f ). Next actions for Sam

1. This sprint (DONE)

   1. Updating the CI script ( https://github.com/numbersprotocol/capture-lite/blob/master/.github/workflows/build-apks.yml ) for building the flavors in row 2 - 4 ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255 ), and sending the PR to Bofu
   2. Providing the new GitHub configs of Network Action to Bofu

2. Next sprint (NOT DONE)

   1. Updating the Capture App to use the new Network Action configs
   2. Updating the CI script ( https://github.com/numbersprotocol/capture-lite/blob/master/.github/workflows/build-apks.yml ) for building the flavors in row 5 ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255 ), and sending the PR to Bofu

Part 2 is not done. Should we move it to this sprint?

[sync-by-unito[bot]]

➤ Tammy Yang commented:

Sam if part 2 is not done, let's move this to the next sprint since the schedule of this sprint is kickoffed.

[sync-by-unito[bot]]

➤ Sam commented:

Scott Yan, need to clarify where these variables are coming from? NETWORK_ACTION_URL : https://network-app.numbersprotocol.io/ ( https://network-app.numbersprotocol.io/ ) NETWORK_ACTION_QA_URL : https://network-app.numbersprotocol.io/version-qa-release/ ( https://network-app.numbersprotocol.io/version-qa-release/ )

I checked GitHub workflow build-android-prod it already uses ( https://github.com/numbersprotocol/capture-lite/blob/2c617d01716fd309fe5341e3d4b71f85f0e414f2/.github/workflows/pre-release.yml#L99 )secrets.NUMBERS_BUBBLE_DB_URL which is https://network-app.numbersprotocol.io/ ( https://network-app.numbersprotocol.io/ ) (aka same as NETWORK_ACTION_URL).

I don't have permission to see https://github.com/numbersprotocol/capture-lite/settings/variables/actions ( https://github.com/numbersprotocol/capture-lite/settings/variables/actions ) but I think it has NUMBERS_BUBBLE_DB_URL which is https://network-app.numbersprotocol.io/ ( https://network-app.numbersprotocol.io/ ) (can you confirm this)?

https://app.asana.com/app/asana/-/get_asset?asset_id=1204201013228949 ( https://app.asana.com/app/asana/-/get_asset?asset_id=1204201013228949 )

[sync-by-unito[bot]]

➤ Tammy Yang commented:

https://github.com/numbersprotocol/capture-lite/settings/variables/actions ( https://github.com/numbersprotocol/capture-lite/settings/variables/actions ) => no variables

[sync-by-unito[bot]]

➤ Sam commented:

Tammy Yang then probably it should be here? https://github.com/numbersprotocol/capture-lite/settings/secrets/actions ( https://github.com/numbersprotocol/capture-lite/settings/secrets/actions )

[sync-by-unito[bot]]

➤ Tammy Yang commented:

Sam what do you need? There is no network action related varlables

[sync-by-unito[bot]]

➤ Sam commented:

Tammy Yang, I'm just trying to automatically build these ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255&range=A2:F5 ) flavors on git push tags. Can we huddle? Would be much faster.

[sync-by-unito[bot]]

➤ Sam commented:

Scott Yan, what is the QA version for NUMBERS_BUBBLE_DB_URL?

[sync-by-unito[bot]]

➤ Tammy Yang commented:

Sam my gut feeling is, that's not something Scott would know. We need to check in the github side which is never controlled by QA. They are in charge of the quality control, not dev.

Can you please share what are you looking for and what's missing?

[sync-by-unito[bot]]

➤ Sam commented:

Tammy Yang, thank you for clarification. Currently I'm configuring github workflow to build these flavors ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255&range=A10:F14 ).

So end result will build flavors and put them in google folders:

• test-official (probably we might not need this since same flavor will be available at PlayStore before publish)
• test-storage-backend
• test-bubble-iframe
• test-network-actions

For that I wrote these configs and want to make sure that env var names exists and match with env vars defined in GitHub secrets or variables.

https://app.asana.com/app/asana/-/get_asset?asset_id=1204200922531146 ( https://app.asana.com/app/asana/-/get_asset?asset_id=1204200922531146 )

[sync-by-unito[bot]]

➤ Sam commented:

Kenny Hung Old CI/CD generated 6 flavors (APKs uploaded to google drive):

• storage-dev-bubble-qa
• storage-qa-bubble-qa
• storage-prod-bubble-qa
• storage-dev-bubble-prod
• storage-qa-bubble-prod
• storage-prod-bubble-prod

According this table ( https://docs.google.com/spreadsheets/d/14xP-HuGdiS5UExmn3ebMKpjGEqqbTPEa5GJCOLM0WSU/edit?pli=1#gid=926369255&range=A10:F14 ) QA need only 4 flavors now? So if New CI/CD will generate 4 flavors (APKs uploaded to google drive):

• test-official (probably we might not need this since same flavor will be available at PlayStore before publish)
• test-storage-backend
• test-bubble-iframe
• test-network-actions

Kenny HungIs New CI/CD what QA expects?

[sync-by-unito[bot]]

➤ Kenny Hung commented:

Sam Yes!!👍

[sync-by-unito[bot]]

➤ Kenny Hung commented:

Sam(cc Scott Yan)

Just confirm, where will QA get these 4 flavors after new CI/CD?

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, (I push tag android-0.77.2 to generate APKs for QA) but it didn't worked as expected (here are the logs ( https://github.com/numbersprotocol/capture-lite/actions/runs/4923099513/jobs/8794540115 )).

Can you please help to verify in GitHub secrets if secrets.NUMBERS_PQINA_NPM_KEY is available during build-apks.yml ( https://github.com/numbersprotocol/capture-lite/blob/9f490be95cf0f1c436a284ca13cacef77ee0a0b7/.github/workflows/build-apks.yml#LL58C11-L58C32 ) run?

Because in pre-release.yml ( https://github.com/numbersprotocol/capture-lite/blob/9f490be95cf0f1c436a284ca13cacef77ee0a0b7/.github/workflows/pre-release.yml#L32 ) run secrets.NUMBERS_PQINA_NPM_KEY is available but it seems like during build-apks.yml ( https://github.com/numbersprotocol/capture-lite/blob/9f490be95cf0f1c436a284ca13cacef77ee0a0b7/.github/workflows/build-apks.yml#LL58C11-L58C32 )secrets.NUMBERS_PQINA_NPM_KEY run is not available and therefore throwing Unable to authenticate, your authentication token seems to be invalid..

NOTE: We use NUMBERS_PQINA_NPM_KEY auth token to install pintura image editor node module.

[sync-by-unito[bot]]

➤ Bofu Chen commented:

Sam I think NUMBERS_PQINA_NPM_KEY is available in build-apks.yml because there is no difference between it and other environment variables.

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, thank you for clarification.

Last time QA asked APK I generated ( https://dt42-numbers.slack.com/archives/C0323488MEJ/p1683708233489199?thread_ts=1683608934.248819&cid=C0323488MEJ ) manually so nothing is blocked for now.

Kenny Hung, I think we need to revisit this task because bubble version control system introduced braking changes anyway.

[sync-by-unito[bot]]

➤ Kenny Hung commented:

Sam (cc Bofu ChenTammy YangScott Yan)

Okay, maybe we could add the same task on current sprint or next sprint?

[sync-by-unito[bot]]

➤ Sam commented:

Kenny Hung, next spring please if possible. 🙏

[sync-by-unito[bot]]

➤ Kenny Hung commented:

Sam I create it. ✓ (for buuble branch control updated) capture app flavours should be built automatically by CI/CD ( https://app.asana.com/0/1201016280880500/1204576787140259/f )

[sync-by-unito[bot]]

➤ Sam commented:

Kenny Hung, same place as before (google drive).

When this task is completed in google drive there will be folders

• test-storage-backend
• test-bubble-iframe
• test-network-actions

[sync-by-unito[bot]]

➤ Sam commented:

Bofu Chen, I found the issue why this error ( https://github.com/numbersprotocol/capture-lite/actions/runs/4923099513/jobs/8794540115 ) happened on Build Ionic phase.

• pre-release.yml is working because it runs npm run preconfig.nprc ( https://github.com/numbersprotocol/capture-lite/blob/02ae4e74227cfc46f537ad1ead006ddf27d5eb18/.github/workflows/pre-release.yml#L35 ).
• build-apks.yml is missing npm run preconfig.nprc ( https://github.com/numbersprotocol/capture-lite/blob/02ae4e74227cfc46f537ad1ead006ddf27d5eb18/.github/workflows/build-apks.yml#L54 ) therefore we are getting Unable to authenticate

Here is what I did

1. create draft-fix-build-apks-CI-CD ( https://github.com/numbersprotocol/capture-lite/tree/draft-fix-build-apks-CI-CD ).
2. add missing missing npm run preconfig.nprc to build-apks.yml
3. create tag 0.79.1 and pushed (for testing purposes)
4. now Build Ionic phase is passing however I got new error ( https://github.com/numbersprotocol/capture-lite/actions/runs/5397618186/jobs/9802491341 ) occurred in Upload matrix build outputs to Google Drive phase.

My question to Bofu Chen, do we need to refresh tokens for Google drive related github action?

• If YES then I need help from dev with admin privileges.

I also ask GPT why error ( https://github.com/numbersprotocol/capture-lite/actions/runs/5397618186/jobs/9802491341 ) could occur:

The error message suggests that there was an issue with the authentication and authorization process when trying to create a Google Drive client for the action-google-drive GitHub Action.

The specific error "invalid_grant" typically occurs when the provided credentials are invalid or expired. Here are a few steps you can take to troubleshoot the issue:

1. Verify Credentials: Double-check the credentials you provided (SKICKA_GOOGLE_CLIENT_ID and SKICKA_GOOGLE_CLIENT_SECRET) in your GitHub repository's secrets. Ensure that they are correct and properly formatted.

2. Check Permissions: Make sure that the credentials have the necessary permissions to access and manage files on Google Drive. Ensure that the associated Google account has the required permissions.

3. Refresh Tokens: If you suspect that the credentials or access token might have expired, you can try regenerating the credentials and updating them in your GitHub repository's secrets.

4. Confirm API Access: Ensure that the Google Drive API is enabled in the Google Cloud Console. Without proper API access, the action-google-drive action won't be able to interact with Google Drive.

If you continue to encounter the same error after checking the above points, it might be helpful to review the documentation or reach out to the maintainers of the action-google-drive GitHub Action for further assistance.

[sync-by-unito[bot]]

➤ Bofu Chen commented:

James Chien Please help Sam check the error above. Thanks!

[sync-by-unito[bot]]

➤ James Chien commented:

Sam The last Build APKs job runs on last year Merge pull request #1859 from numbersprotocol/hotfix-undo-truncate-ap… · numbersprotocol/capture-lite@b7cb9b1 (github.com) ( https://github.com/numbersprotocol/capture-lite/actions/runs/2768600422 )

And it seems Google oauth2 refresh token expires if not used for 6 months (Using OAuth 2.0 to Access Google APIs | Authorization | Google for Developers ( https://developers.google.com/identity/protocols/oauth2 )). So yes it seems the client_id and client_secret needs to be updated.

I'm trying to create new tokens and update it. Might need to do some additional permission settings in GCP console or generate the tokecache.json again.

numbersprotocol/action-google-drive: GitHub Action to interact with Google Drive ( https://github.com/numbersprotocol/action-google-drive )

[sync-by-unito[bot]]

➤ James Chien commented:

Sam

When trying to authenticate with Skicka, I received an error showing

錯誤 400：invalid_request

The out-of-band (OOB) flow has been blocked in order to keep users secure. Follow the Out-of-Band (OOB) flow migration guide linked in the developer docs below to migrate your app to an alternative method.

要求詳情： redirect_uri=urn:ietf:wg:oauth:2.0:oob

It turns out Google has blocked OOB flow since ( https://developers.google.com/identity/protocols/oauth2/resources/oob-migration#key-compliance-dates )

Skicka package also goes to public archived ( https://github.com/google/skicka ) status so it is already a package that no longer being maintained.

I'd suggest using the more popular and actively maintained tool rclone ( https://rclone.org/ ) to upload file to Google Drive.

I've setuped repo secrets and opened a pull reques ( https://github.com/numbersprotocol/capture-lite/pull/2871 )t to use rclone to upload to google drive. The testing action ( https://github.com/numbersprotocol/capture-lite/actions/runs/5401632028 ) seems working just fine. If you need to change anything else (e.g. folder names) feel free to do it directly before or after merge.

[sync-by-unito[bot]]

➤ Bofu Chen commented:

James Chien After confirming the CI is fixed, please share the updated secrets with me when you have time. Thanks for helping resolve this challenging issue!

[sync-by-unito[bot]]

➤ James Chien commented:

Bofu Chen Shared in slack dm

[sync-by-unito[bot]]

➤ James Chien commented:

I'll also share steps to create the rclone config here.

1. Go to GCP console credentials page ( https://console.cloud.google.com/apis/credentials ) with capture-dev-push-notification project, create OAuth 2.0 Client IDs, select Desktop application as option
2. Save the client_id and client_secret for later usage
3. On API tab, enable Google Drive API
4. Install rclone locally (on any workspace) following official doc instructions ( https://rclone.org/install/ )
5. Run rclone config (interactive), select Google drive as backend. When asked about auto-opening a browser to auth, select no. Input the client_id and client_secret in early step, but no need for application json. Use gdrive as remote name. Enable advanced config and enter root folder id 1NH-4mruBBubxHMfXF6VXaKRCQ76Ldg_n (this is the capture-lite/apk folder in hi@numbersprotocol.io ( mailto:hi@numbersprotocol.io ) drive)
6. After the config is done, base64 .config/rclone/rclone.conf to get the base64 value of the config. The GitHub Action repo secret RCLONE_CONFIG_BASE64 uses base64 encoded config value.

[sync-by-unito[bot]]

➤ Bofu Chen commented:

James Chien I cannot find the capture-dev-push-notification project by using the marvin account. Did I miss something?