Allow users to skip permission errors

[malomarrec]

Current behavior:

If the spec includes a repository I don't have read access to, the entire preview execution fails. To reproduce:

• Create a spec with a named list of repositories using on.repository.
• Include a repository you don't have permission to read.
• Run src batch preview with the --skip-errors flag

Expected behavior:

**On src-cli*: If you use --skip-errors, repositories to which you don't have read access are skipped, with a warning about the reason. That does not fail the entire preview, just skips the repository, with a warning. On SSBC:** same behaviour as above by default.

[github-actions[bot]]

Hey, @sourcegraph/batchers (@eseliger @mrnugget @LawnGnome @malomarrec @chrispine @courier-new) - we have been mentioned. Let's take a look.

[malomarrec]

Experienced by https://github.com/sourcegraph/customers/issues/1

[eseliger]

This is not strictly speaking a bug, we scoped this flag to execution errors, so errors that happen inside your specified steps, which cause the container to exit non-zero. I agree this flag doesn't have the best name, though. And still think this is a valid feature request regardless.

[malomarrec]

Should we include this in skip-errors? Or is there a reason to solve it separately? I think the ideal state would be it's treated as all the other errors, with a useful error message that this is because of missing permissions on this repo. WDYT @eseliger ?

[eseliger]

Agree, I think we should extend the scope of --skip-errors.

[mrnugget]

Just leaving a +1 here: makes sense to me.

[malomarrec]

https://docs.google.com/spreadsheets/d/1OEhzdMSlkGOaWyGKwdiAGlirsKKj9EN45Izn7kdKNTg/edit#gid=0&range=A5930

[malomarrec]

https://github.com/sourcegraph/customers/issues/1 +1 from customer

[malomarrec]

Sync discussion:

• estimate: 0.5 days for the CLI
• on SSBC, need to define the behaviour some more. TBD. Note that this only happens when repositories are hardcoded with on.repository, see #24307 that addresses what happens when a user has read but not write permissions.

[malomarrec]

@eseliger is this still relevant for SSBC? Or is it now a src-cli only issue (so only a 0.5 days)?

[eseliger]

Pretty much the same state as above: In SSBC, currently, we skip repos that you can't see in repositoriesMatchingQuery. If a repo is specified using repository: name we still error, because we thought that when you specify it directly, that you are VERY sure you want to run over that repo so just skipping it seems weird. For warnings, we currently don't have a mechanism. Either we fail and have an error message, or we pass and we don't. If we want to add warnings, that's more than 0.5d.

Is that what you are looking for?

[malomarrec]

And when you say fail it means fail on that one workspace, not fail the batch change correct?

[eseliger]

Fail the preview. So you cannot execute.

[malomarrec]

I think we should add warnings in that case indeed

[eseliger]

Is that a requirement for beta?

[malomarrec]

No, but I'll propose adding this to next quarter's roadmap

[eseliger]

sounds good!