Support download of setup-files from private GitLab repositories

[AresEkb]

It works fine for public repositories with anonymous access. For instance, a setup-file can be downloaded using such an URL https://gitlab.eclipse.org/eclipse/ease/ease/-/raw/main/releng/org.eclipse.ease.releng/oomph/ease.setup

But it doesn't work for private GitLab repositories.

It seems that GitLab doesn't support basic authentication. Here is an open issue. And also basic authentication is not mentioned in the documentation.

So the following URL will not work: https://gitlab.eclipse.org/eclipse/ease/ease/-/raw/main/releng/org.eclipse.ease.releng/oomph/ease.setup?oomph_basic_auth=true

GitLab will redirect to an HTML login form.

It's possible to clone a Git repository using ACCESS_TOKEN:

git clone https://oauth2:ACCESS_TOKEN@gitlab.eclipse.org/eclipse/ease/ease.git

But it seems that it's impossible to use such an URL to download a specific file.

Probably the only way to download a file from a private GitLab repository is to use Files API. In that case the followin URL should be used (you can ommit ACCESS_TOKEN if a repository allows anonymous access): https://gitlab.eclipse.org/api/v4/projects/3814/repository/files/releng%2Forg.eclipse.ease.releng%2Foomph%2Fease.setup/raw?private_token=ACCESS_TOKEN

It works fine. But Eclipse Installer throws the following exception:

java.lang.ClassCastException: class org.eclipse.emf.ecore.xmi.impl.XMIResourceImpl cannot be cast to class org.eclipse.oomph.base.util.BaseResource (org.eclipse.emf.ecore.xmi.impl.XMIResourceImpl is in unnamed module of loader org.eclipse.osgi.internal.loader.EquinoxClassLoader @58dea0a5; org.eclipse.oomph.base.util.BaseResource is in unnamed module of loader org.eclipse.osgi.internal.loader.EquinoxClassLoader @7446d8d5)
        at org.eclipse.oomph.base.util.BaseUtil.loadResourceSafely(BaseUtil.java:97)
        at org.eclipse.oomph.setup.internal.installer.ProductPage$AddUserProductDialog.processResources(ProductPage.java:2073)
        at org.eclipse.emf.common.ui.dialogs.ResourceDialog.okPressed(ResourceDialog.java:372)
        at org.eclipse.jface.dialogs.Dialog.buttonPressed(Dialog.java:468)
        at org.eclipse.jface.dialogs.Dialog.lambda$0(Dialog.java:619)
        at org.eclipse.swt.events.SelectionListener$1.widgetSelected(SelectionListener.java:84)
        at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:252)
        at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
        at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5854)
        at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1529)
        at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:5064)
        at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4516)
        at org.eclipse.jface.window.Window.runEventLoop(Window.java:823)
        at org.eclipse.jface.window.Window.open(Window.java:799)
        at org.eclipse.oomph.setup.internal.installer.ProductPage$15.widgetSelected(ProductPage.java:591)
        at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:252)
        at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
        at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5854)
        at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1529)
        at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:5064)
        at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4516)
        at org.eclipse.oomph.setup.internal.installer.Installer.runEventLoop(Installer.java:359)
        at org.eclipse.oomph.setup.internal.installer.InstallerDialog.show(InstallerDialog.java:434)
        at org.eclipse.oomph.setup.internal.installer.InstallerApplication.run(InstallerApplication.java:295)
        at org.eclipse.oomph.setup.internal.installer.InstallerApplication.start(InstallerApplication.java:401)
        at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:208)
        at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
        at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
        at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
        at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:651)
        at org.eclipse.equinox.launcher.Main.basicRun(Main.java:588)
        at org.eclipse.equinox.launcher.Main.run(Main.java:1459)
        at org.eclipse.equinox.launcher.Main.main(Main.java:1432)

I guess the cause is a wrong file name - raw (because the last segment of URL is raw). The file doesn't have setup extension.

It also seems that not all versions of GitLab returns correct HTTP headers:

Content-Disposition: inline; filename=blob
X-Gitlab-File-Name: ease.setup

Content-Disposition header should contain a valid file name.

Probably one of the ways to fix the problem is to register a Setup Resource Factory as a fallback one.

A discussion is available here.

[merks]

I'd personally be concerned about having my PAT visible in the file system, but that's the user's choice. Better would be to find a way to prompt for credentials, store then in secure storage, and produce the appropriate request headers from that information...

[AresEkb]

An access token can be send in Authorization or Private-Token HTTP header:

curl --header 'Authorization: Bearer ACCESS_TOKEN' 'https://gitlab.eclipse.org/api/v4/projects/3814/repository/files/releng%2Forg.eclipse.ease.releng%2Foomph%2Fease.setup/raw'

curl --header 'Private-Token: ACCESS_TOKEN' 'https://gitlab.eclipse.org/api/v4/projects/3814/repository/files/releng%2Forg.eclipse.ease.releng%2Foomph%2Fease.setup/raw'

Maybe support of bearer authorization can be added in addition to basic authorization?

[merks]

Yes, that would be cool. That being said, we are using ECF under the covers and the header stuff is handled in that technology stack so not so trivial to enhance. Then there is also the problem of reproducibility. I don't have access to a gitlab instance with a private repo where I can really properly test what's working and what isn't. When I tried this stuff with gitlab.eclipse.org using a bogus/wrong token, it prompted me for credentials, though I did not try to enter any but rather created a proper token. And finally, there is the problem of time investment, i.e., not having time. 😱