Flank / flank

:speedboat: Massively parallel Android and iOS test runner for Firebase Test Lab
https://firebase.community/
Apache License 2.0
682 stars 118 forks source link

Clean flank not authorized error messages #874

Closed bootstraponline closed 4 years ago

bootstraponline commented 4 years ago

Describe the bug

A clear and concise description of what the bug is.

flank android run and flank ios run produce large error messages that aren't user friendly when the service account isn't authenticated correctly. Let's clean up the error messages to improve the user experience.

To Reproduce

Steps to reproduce the behavior:

  1. Update flank.yml to point to a project that flank isn't authorized for
  2. Run flank android run

Expected behavior

A clear and concise description of what you expected to happen.

A clean error message, something like this:

$ flank android run
version: v20.06.2
revision: 08de9c3f4f8ee1ecc3684ecb06391b8521a08f4d

Flank encountered a 403 error when running on project foo-bar. Please verify this credential is authorized for the project.

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "The caller does not have permission",
    "reason" : "forbidden"
  } ],
  "message" : "The caller does not have permission",
  "status" : "PERMISSION_DENIED"
}

Details (please complete the following information):

Have you tested on the latest Flank snapshot?

Post the output of flank --version.

Yes

Additional context

Add any other context about the problem here.

Here's the current error message:

flank android run
version: v20.06.2
revision: 08de9c3f4f8ee1ecc3684ecb06391b8521a08f4d
java.io.IOException: Request failed
    at ftl.http.ExecuteWithRetryKt$executeWithRetry$$inlined$withRetry$1.invokeSuspend(ExecuteWithRetry.kt:36)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.ResumeModeKt.resumeMode(ResumeMode.kt:67)
    at kotlinx.coroutines.DispatchedKt.resume(Dispatched.kt:319)
    at kotlinx.coroutines.DispatchedKt.dispatch(Dispatched.kt:301)
    at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:250)
    at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:260)
    at kotlinx.coroutines.CancellableContinuationImpl.resumeUndispatched(CancellableContinuationImpl.kt:332)
    at kotlinx.coroutines.EventLoopImplBase$DelayedResumeTask.run(EventLoop.common.kt:483)
    at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:270)
    at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:79)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
    at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
    at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
    at ftl.http.ExecuteWithRetryKt.executeWithRetry(ExecuteWithRetry.kt:41)
    at ftl.gc.GcToolResults.getDefaultBucket(GcToolResults.kt:118)
    at ftl.args.ArgsHelper.createGcsBucket(ArgsHelper.kt:124)
    at ftl.args.CreateCommonArgsKt.createCommonArgs(CreateCommonArgs.kt:15)
    at ftl.args.CreateAndroidArgsKt.createAndroidArgs$default(CreateAndroidArgs.kt:12)
    at ftl.args.AndroidArgsCompanion.load$flank(AndroidArgsCompanion.kt:35)
    at ftl.args.AndroidArgsCompanion.load(AndroidArgsCompanion.kt:31)
    at ftl.cli.firebase.test.android.AndroidRunCommand.run(AndroidRunCommand.kt:46)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1769)
    at picocli.CommandLine.access$900(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2150)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2144)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2108)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1975)
    at picocli.CommandLine.execute(CommandLine.java:1904)
    at ftl.Main$Companion$main$1.invoke(Main.kt:53)
    at ftl.Main$Companion$main$1.invoke(Main.kt:43)
    at ftl.util.Utils.withGlobalExceptionHandling(Utils.kt:130)
    at ftl.Main$Companion.main(Main.kt:49)
    at ftl.Main.main(Main.kt)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "The caller does not have permission",
    "reason" : "forbidden"
  } ],
  "message" : "The caller does not have permission",
  "status" : "PERMISSION_DENIED"
}
    at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:443)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1092)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:541)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
    at ftl.http.ExecuteWithRetryKt$executeWithRetry$$inlined$withRetry$1.invokeSuspend(ExecuteWithRetry.kt:41)
    ... 34 more
pawelpasterz commented 4 years ago

Do we also want to print proposal to use flank auth login ?

Flank encountered a 403 error when running on project foo-bar. Please verify this credential is authorized for the project.
You can also login with different credentials with `flank auth login`

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "The caller does not have permission",
    "reason" : "forbidden"
  } ],
  "message" : "The caller does not have permission",
  "status" : "PERMISSION_DENIED"
}
bootstraponline commented 4 years ago

I think we should encourage service accounts for CI. flank auth login is okay for local runs though.

pawelpasterz commented 4 years ago

I think we should encourage service accounts for CI. flank auth login is okay for local runs though.

Good point 👍

pawelpasterz commented 4 years ago

@bootstraponline

Flank encountered a 403 error when running on project foo-project. Please verify this credential is authorized for the project.
Consider authentication with Service Account https://github.com/Flank/flank#authenticate-with-a-service-account
or with Google account https://github.com/Flank/flank#authenticate-with-a-google-account

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "The caller does not have permission",
    "reason" : "forbidden"
  } ],
  "message" : "The caller does not have permission",
  "status" : "PERMISSION_DENIED"
}

EDIT

I think the second common mistake is wrong projectId, we could enhance logs for that error as well

Flank was unable to find project foo-project. Please verify project id.

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
{
  "code" : 404,
  "errors" : [ {
    "domain" : "global",
    "message" : "Project not found: aitest-b9214asdasd",
    "reason" : "notFound"
  } ],
  "message" : "Project not found: aitest-b9214asdasd",
  "status" : "NOT_FOUND"
}
bootstraponline commented 4 years ago

That looks awesome!