search

jgraph / drawio

draw.io is a JavaScript, client-side editor for general diagramming.
https://www.drawio.com
Other
40.69k stars 7.57k forks source link

Add private gitlab storage #493

Closed agaget closed 5 years ago

agaget commented 5 years ago

Hi, Thanks for the gitlab storage feature, Is there a way to use a private gitlab ? When I click on "Authorize" it goes to "gitlab.com" but I would like it to store on my gitlab company (https://gitlab.example.fr/) ? If not, can it be integrated ?

Thanks

davidjgraph commented 5 years ago

Happy to look at PR to add it.

fhemberger commented 5 years ago

References to 'gitlab.com' are in the following files:

You could for example modify the current Dockerfile like this:

FROM frekele/ant:1.10.3-jdk8 as BUILD

ARG GITLAB_DOMAIN=https://gitlab.com/
ENV DEBIAN_FRONTEND=noninteractive

# Download the latest draw.io release from GitHub
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN apt-get update \
  && apt-get install jq -y --no-install-recommends \
  && mkdir /usr/build \
  && curl -fsSL -o drawio.tar.gz --compressed "$(curl -fsSL --compressed "https://api.github.com/repos/jgraph/drawio/releases/latest" | jq -r ".tarball_url")" \
  && tar xzf drawio.tar.gz -C /usr/build --strip-components=1

# Replace all references to gitlab.com
WORKDIR /usr/build
RUN grep -rl 'https://gitlab.com/' | xargs sed -i "s|https://gitlab.com/|$GITLAB_DOMAIN|g"

# Continue to build like the official release 
WORKDIR /usr/build/etc/build/
RUN ant war

FROM tomcat:9.0 as TARGET
COPY --from=BUILD /usr/build/build/draw.war /usr/local/tomcat/webapps/
EXPOSE 8080
CMD ["catalina.sh", "run"]

Then build the Dockerfile with your custom URL:

docker build \
  --build-arg GITLAB_DOMAIN=https://gitlab.example.fr/ \
  -t drawio-custom-gitlab \
  .

Maybe not the nicest solution, but it should work. 😉

m4r10k commented 5 years ago

@fhemberger we did the first tries to do the same on Friday but weren't able to finish our testing. Therefore many thanks for your great summary - we will test it next week for our on-premises installation too. 🤗

fhemberger commented 5 years ago

@m4r10k Haven't tested it yet with our GitLab instance (maybe next week), just a quick idea I came up with. It should do the trick … looking forward for your feedback!

m4r10k commented 5 years ago

@fhemberger ... sure! Such ideas are great to exchange - you are some steps ahead. I just hacked this stuff quick and dirty into the draw.io Docker image directly for testing 🤣

The problem that we have is, that I know were to put the Client ID - but I am not sure about the Secret (from the Gitlab Applications...)

davidjgraph commented 5 years ago

Refreshing to see some constructive solutions in issues, but this one is a lot easier for us to deal with :). With 11.1.2 you can add a URL parameter "gitlab=yourURL" to set the path to your custom instance.

fhemberger commented 5 years ago

@davidjgraph Awesome, thanks!

m4r10k commented 5 years ago

@davidjgraph Many many thanks!

yetanothern commented 5 years ago

A stupid question: where should I add this parameter?

fhemberger commented 5 years ago

@stepkh http://draw.io?gitlab=https://mygitlab.example.com/

yetanothern commented 5 years ago

When I put url like http://my.drawio.local/?gitlab=http://my.gitlab.local I get "Save diagram to:" window, the same if I put drawio url without parameters.

yetanothern commented 5 years ago

This url works: http://my.drawio.local/?mode=gitlab&gitlab=http://my.gitlab.local. But when I authorize in gitlab, I get an error "Client authentication failed due to unkown client, no client authentication included, or unsupported authentication method". The same error was got by me when I manually changed 'gitlab.com' to 'my.gitlab.local' in previous version 11.1.1.

davidjgraph commented 5 years ago

OK, the main question is can anyone get this to work with the URL parameter? I don't have a local gitlab instance. If yes, authorizing in gitlab sounds off-topic for draw.io. If no, we need to debug.

agaget commented 5 years ago

Hi, Come back from holiday and I see there already is a solution. Thanks for your prompt reaction :) I encountered the same problem as @stepkh "client authentification blabla..." I'm not the administrator of our private instance so I will talk to my admin see if there is any right to unlock.

yetanothern commented 5 years ago

If it will help to debug, i glad to provide a couple test public containers with self-hosted draw.io and gitlab.

davidjgraph commented 5 years ago

Actually, I assumed with the above script that someone had this working, but probably not, reading it again. I should have explained the client ID.

At the top of GitLabClient.js there's a client ID. That's the ID for our application in Gitlab. It has custom redirect URLs for the authenication, so it won't work on your local URL.

I can tell you the setup using gitlab.com, I hope it's similiar on a local install. Under user settings there's an application option:

Screenshot 2019-08-06 at 08 57 52

The redirect URI needs to point to the gitlab.html file in the root of the draw.io installation and you need to select the following scopes:

Screenshot 2019-08-06 at 09 02 42

btw, if anyone can find a set of working scopes that doesn't include "API", please let us know.

When the app is saved, you need to swap out the clientID mentioned, but we'll add a URL parameter for that in a minute.

m4r10k commented 5 years ago

As I wrote in https://github.com/jgraph/drawio/issues/493#issuecomment-518007038 , there is a setting for the ClientID in GitLabClient.js . You can generate this ID by creating an Application in the GitLab Admin Settings under "Applications" - but I have currently no idea where to put the Secret that is also provided by GitLab during Application creation :-)

m4r10k commented 5 years ago

@davidjgraph ups, ninja'd :)

davidjgraph commented 5 years ago

Added as https://github.com/jgraph/drawio/commit/d6c66c439d53bc833bd9c9a92e11525ca9eb051e

URL parameter is "gitlab-id"

Note that if you can inject these global variables in the docker image:

window.DRAWIO_GITLAB_URL window.DRAWIO_GITLAB_ID

you don't have to use the URL parameters

agaget commented 5 years ago

In your example I think it shouldn't be MyGitlabPath but MyDrawIOPath. I'm using the online version so I've configured the application as you said using as redirectURI https://draw.io/gitlab.html image

When I entered this url with the client-ID given : https://www.draw.io/?gitlab=https://mygitlab.fr&gitlab-id=94b55953cd3865a78dd6903e2adbd6095f2ac5646ec58aa46d0a76d848d18fa1 I create a new diagram, wanted to store to gitlab, when I "Authorize"

The url sent is the following, the id doesn't seem to be used ? https://myGitlab.fr/oauth/authorize?client_id=5cdc018a32acddf6eba37592d9374945241e644b8368af847422d74c8709bc44&scope=api%20read_repository%20write_repository&redirect_uri=https%3A%2F%2Fwww.draw.io%2Fgitlab.html&response_type=token&state=123

davidjgraph commented 5 years ago

That won't work, because we haven't deployed this to the production version at draw.io yet...

agaget commented 5 years ago

That is a good reason :D

davidjgraph commented 5 years ago

Try the github pages version of this project for testing, https://jgraph.github.io/drawio/src/main/webapp/

agaget commented 5 years ago

Url for draw : https://jgraph.github.io/drawio//src/main/webapp/?gitlab=https://mygitlab.fr&gitlab-id=94b55953cd3865a78dd6903e2adbd6095f2ac5646ec58aa46d0a76d848d18fa1

https://mygitlab.fr/oauth/authorize?client_id=5cdc018a32acddf6eba37592d9374945241e644b8368af847422d74c8709bc44&scope=api%20read_repository%20write_repository&redirect_uri=https%3A%2F%2Fjgraph.github.io%2Fgitlab.html&response_type=token&state=123

Still don't work, still a different id, I imagine that the client_id should be similar to the gitlab-id I indicate in the url, is it ?

davidjgraph commented 5 years ago

Ah, sorry my bad, it's not built. https://jgraph.github.io/drawio/src/main/webapp/?dev=1 , you need dev=1 in the URL parameters

agaget commented 5 years ago

your url result as "Page could not be loaded"

fhemberger commented 5 years ago

@davidjgraph devhost.jgraph.com = ERR_NAME_NOT_RESOLVED

davidjgraph commented 5 years ago

yeah, I'm used to our dev mode just working (TM) locally. I've built and committed the minified files. Try again without the dev parameter.

agaget commented 5 years ago

When i used https://jgraph.github.io/drawio//src/main/webapp/?gitlab=https://myGitlab.fr&gitlab-id=94b55953cd3865a78dd6903e2adbd6095f2ac5646ec58aa46d0a76d848d18fa1 I'm redirected to this url : https://myGitlab.fr/oauth/authorize?client_id=94b55953cd3865a78dd6903e2adbd6095f2ac5646ec58aa46d0a76d848d18fa1&scope=api%20read_repository%20write_repository&redirect_uri=https%3A%2F%2Fjgraph.github.io%2Fgitlab.html&response_type=token&state=123 I have a message alarm : The redirect URI included is not valid. So i have added application using https://jgraph.github.io/gitlab.html redirect URI in gitlab an a new id, I have now this, so it seems client-id is ok ! image

When I click on "authorize" , it goes there https://jgraph.github.io/gitlab.html#access_token=7e048a4d09e72e3a2c50897bda01b5b287261c8f57f9f43a389e60223efc37ee&token_type=bearer&state=123 And it says 404 error

yetanothern commented 5 years ago

Yes, the same thing as @agaget got.

rickywu commented 5 years ago

@davidjgraph Can we use env variable to set URL and ID?

Also, can you support gitlab with subdirectory, such as http://mygitlab.net/gitlab I tried change source to read gitlab url and client id from json file, but not able to get folders in repository.

rickywu commented 5 years ago

Added as d6c66c4

URL parameter is "gitlab-id"

Note that if you can inject these global variables in the docker image:

window.DRAWIO_GITLAB_URL window.DRAWIO_GITLAB_ID

you don't have to use the URL parameters

I tried both window.DRAWIO_GITLAB_URL and DRAWIO_GITLAB_URL but not work environment: DRAWIO_GITLAB_URL: http://xxx:8989/gitlab DRAWIO_GITLAB_ID: a8ef2e21acff13f734127b0cbfe6a22e5302095630b2378a39f33239d9b459b7

davidjgraph commented 5 years ago

I've run up a gitlab instance, but don't get to the authorize screen. It's http, rather than https, does Gitlab not support auth over http?

davidjgraph commented 5 years ago

If there's an instance I can access, would help.

agaget commented 5 years ago

Mine it's not possible, but @stepkh has the same problem as me and proposed to share one earlier.

rickywu commented 5 years ago

@davidjgraph Try my gitlab http://arch.syfintech.cn:8989/gitlab

How could I inject docker image to setup URL and ID variables?

davidjgraph commented 5 years ago

Thanks, but I need an instance I can log into.

Docker image I don't know. https://github.com/fjudith/docker-draw.io is the recommended docker image, I think they process environment variables.

rickywu commented 5 years ago

@davidjgraph Try register an account for your self and set application with id

Also gitlab shoud have access to your draw.io when auth

rickywu commented 5 years ago

Thanks, but I need an instance I can log into.

Docker image I don't know. https://github.com/fjudith/docker-draw.io is the recommended docker image, I think they process environment variables.

I think window.EXPORT_URL is a browser parameter not for docker container enviroment variable

Probally we need support read from enviroment variable?

davidjgraph commented 5 years ago

@rickywu Thanks. Yeah, it doesn't seem to be as easy as changing the URL and client ID.

@agaget That 404 is due to the draw.io domain not being on the jgraph.github.io root.

Let's try again with https://gitlab-test-dot-drawdotio.appspot.com/ . It's the github version, but the redirect will be at https://gitlab-test-dot-drawdotio.appspot.com/gitlab.html

davidjgraph commented 5 years ago

We don't use environment variables in our deployment. You're welcome to submit a PR, but it must work on Google App Engine, where our production draw.io runs.

agaget commented 5 years ago

🎉 it works !! Just a strange details, in the selection menu of my repo it seems that it always select the last "projects" of the "groups" image For example if i click on ESS Modules /m-epics-PF704_Cpl_IPC it will select ESS Modules / PF704-dev (same for the other projects of the groups)

davidjgraph commented 5 years ago

@agaget can you open that as a new issue please? I'll close this one if the basics work.

agaget commented 5 years ago

@davidjgraph done #510

droub commented 4 years ago

I am facing the same kind of issues but it's hard to understand what setup is finally working. Could someone summarize the solution please ?

agaget commented 4 years ago

I can try.. As it seems that it's not in production yet I will use the https://gitlab-test-dot-drawdotio.appspot.com url. In your Gitlab instance : image

The redirect URI needs to point to the gitlab.html file in the root of the draw.io installation and you need to select the following scopes:

image (use https://gitlab-test-dot-drawdotio.appspot.com not draw.io)

Then it will give you an Application-id paste it.

Use https://gitlab-test-dot-drawdotio.appspot.com/?gitlab=https://MyGitLab.com&gitlab-id=Application_id_just_pasted Authorize the app

It should work.

rickywu commented 4 years ago

The latest version add PreConfig.js to let you config DRAWIO_GITLAB_URL and DRAWIO_GITLAB_ID before compile or before start docker container

window.DRAWIO_GITLAB_URL = 'blabla';
window.DRAWIO_GITLAB_ID = 'blabla';
droub commented 4 years ago

Thanks, I still get the : "The redirect URI included is not valid." after authorization.

My draw.io docker is not served on a root url i.e. https://myapps.mycomp.com/draw could that be a problem ? My redirection does point to https://myapps.myconm.com/draw/gitlab.html

Also, I can see a 404ed request to the gitlab: /oauth/undefined#search:1 Failed to load resource: the server responded with a status of 404 ()

I can't tell if my problem is on the draw.io side or on the gitlab side :(.

rickywu commented 4 years ago

You have to modify this line https://github.com/jgraph/drawio/blob/c823b4fe9d54ffc4657aaba7da9f18004c7befd9/src/main/webapp/js/diagramly/GitLabClient.js#L42

change window.location.origin to DRAWIO_GITLAB_URL and set DRAWIO_GITLAB_URL = http://host.com/draw

see https://github.com/jgraph/drawio/issues/515

davidjgraph commented 4 years ago

We'll change that to:

var href = window.location.href;
var dir = href.substring(0, href.lastIndexOf('/'));

in the next release.