search

kitspace / kitspace-v2

Work in progress on using Gitea as the Kitspace backend.
GNU Affero General Public License v3.0
13 stars 6 forks source link

Implement login #6

Closed kasbah closed 3 years ago

kasbah commented 4 years ago

We'll want to hide away most of gitea and allow logging in via our own frontend. Ideally we can also enable "login with github/facebook/google".

kasbah commented 4 years ago

To add some more details on this. The way to authentication works is:

  1. nginx routes GET requests for the apex domain, which ishttp://kitspace.test:3000 in development, internally to http://gitea:3000/__kitspace
  2. Gitea authenticates the user and attaches the session as a body to the GET request and sends that to the "frontend" which is our NextJS server
  3. NextJS reads the session of the body and adds it to the internal NextJS request data
  4. It's then accessible both server-side and client-side in React

Gitea runs on http://gitea.kitspace.test:3000 and internally on http://gitea:3000. The first one is used on the client-side and the second one on the server-side. You need to send the same POST requests that are sent when you use http://gitea.kitspace.test:3000/login but we want to use the NextJS "frontend" and have our own login form on http://kitspace.test:3000/login

AbdulrhmnGhanem commented 4 years ago

Both http://kitspace.test:3000/install and http://gitea.kitspace.test:3000/install cannot be reached the connection was reset. Although http://gitea.kitspace.test and http://kitspace.test display the default Nginx page. Running containers

CONTAINER ID        IMAGE                                  COMMAND                  CREATED             STATUS                      PORTS                                          NAMES
15a67ee6d38a        kitspace_frontend                      "docker-entrypoint.s…"   37 minutes ago      Up 2 minutes                0.0.0.0:1234->3000/tcp                         kitspace_frontend_1
2feef990120c        kitspace_gitea                         "/usr/bin/entrypoint…"   37 minutes ago      Up 2 minutes                0.0.0.0:2222->22/tcp, 0.0.0.0:3333->3000/tcp   kitspace_gitea_1
f9dfc6c6245c        postgres:9.6                           "docker-entrypoint.s…"   37 minutes ago      Up 2 minutes                5432/tcp                                       kitspace_postgres_1
32496abae89f        kitspace/nginx-certbot-plugin:latest   "/bin/bash --verbose…"   37 minutes ago      Up 2 minutes                0.0.0.0:3000->80/tcp                           kitspace_nginx_1
9552e7818db3        7d1f7cb2441f                           "/bin/sh -c 'NODE_EN…"   About an hour ago   Exited (1) 59 minutes ago                                                  busy_austin

The output from docker-compose up

Starting kitspace_nginx_1 ... 
Starting kitspace_postgres_1 ... 
Starting kitspace_nginx_1
Starting kitspace_nginx_1 ... done
Starting kitspace_gitea_1 ... 
Starting kitspace_gitea_1 ... done
Starting kitspace_frontend_1 ... 
Starting kitspace_frontend_1 ... done
Attaching to kitspace_postgres_1, kitspace_nginx_1, kitspace_gitea_1, kitspace_frontend_1
postgres_1  | 
postgres_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1  | 
postgres_1  | LOG:  database system was interrupted; last known up at 2020-07-14 12:26:27 UTC
nginx_1     | sed -e 's/@KITSPACE_DOMAIN/kitspace.test/g' -e 's/@KITSPACE_URL/http:\/\/kitspace.test:3000/g' /etc/nginx/conf.d/kitspace.template > /etc/nginx/conf.d/default.conf && /opt/command
postgres_1  | LOG:  database system was not properly shut down; automatic recovery in progress
nginx_1     | Not requesting certificates through certbot, set RUN_CERTBOT=true to run.
gitea_1     | Server listening on :: port 22.
postgres_1  | LOG:  invalid record length at 0/14EEE78: wanted 24, got 0
gitea_1     | Server listening on 0.0.0.0 port 22.
postgres_1  | LOG:  redo is not required
nginx_1     | Running certbot renew
gitea_1     | 2020/07/14 14:38:15 cmd/web.go:107:runWeb() [I] Starting Gitea on PID: 15
postgres_1  | LOG:  MultiXact member wraparound protections are now enabled
gitea_1     | 2020/07/14 14:38:15 ...dules/setting/git.go:91:newGit() [I] Git Version: 2.24.3, Wire Protocol Version 2 Enabled
nginx_1     | 2020/07/14 12:38:14 [emerg] 8#8: host not found in upstream "frontend" in /etc/nginx/conf.d/default.conf:27
nginx_1     | nginx: [emerg] host not found in upstream "frontend" in /etc/nginx/conf.d/default.conf:27
gitea_1     | 2020/07/14 14:38:15 routers/init.go:88:GlobalInit() [T] AppPath: /app/gitea/gitea
gitea_1     | 2020/07/14 14:38:15 routers/init.go:89:GlobalInit() [T] AppWorkPath: /app/gitea
gitea_1     | 2020/07/14 14:38:15 routers/init.go:90:GlobalInit() [T] Custom path: /data/gitea
postgres_1  | LOG:  database system is ready to accept connections
gitea_1     | 2020/07/14 14:38:15 routers/init.go:91:GlobalInit() [T] Log path: /data/gitea/log
postgres_1  | LOG:  autovacuum launcher started
gitea_1     | 2020/07/14 14:38:15 ...dules/setting/log.go:233:newLogService() [I] Gitea v built with GNU Make 4.2.1, go1.13.12 : bindata, sqlite, sqlite_unlock_notify
gitea_1     | 2020/07/14 14:38:15 ...dules/setting/log.go:276:newLogService() [I] Gitea Log Mode: Console(Console:info)
gitea_1     | 2020/07/14 14:38:15 ...les/setting/cache.go:70:newCacheService() [I] Cache Service Enabled
gitea_1     | 2020/07/14 14:38:15 ...les/setting/cache.go:81:newCacheService() [I] Last Commit Cache Service Enabled
gitea_1     | 2020/07/14 14:38:15 ...s/setting/session.go:63:newSessionService() [I] Session Service Enabled
gitea_1     | 2020/07/14 14:38:15 routers/init.go:128:GlobalInit() [I] SQLite3 Supported
gitea_1     | 2020/07/14 14:38:15 routers/init.go:47:checkRunMode() [I] Run Mode: Development
nginx_1     | Saving debug log to /var/log/letsencrypt/letsencrypt.log
nginx_1     | 
nginx_1     | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
nginx_1     | 
nginx_1     | No renewals were attempted.
nginx_1     | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
nginx_1     | 2020/07/14 12:38:15 [notice] 11#11: signal process started
nginx_1     | 2020/07/14 12:38:15 [error] 11#11: open() "/var/run/nginx.pid" failed (2: No such file or directory)
nginx_1     | nginx: [error] open() "/var/run/nginx.pid" failed (2: No such file or directory)
gitea_1     | 2020/07/14 14:38:16 cmd/web.go:164:runWeb() [I] Listen: http://0.0.0.0:3000
gitea_1     | 2020/07/14 14:38:16 ...s/graceful/server.go:55:NewServer() [I] Starting new server: tcp:0.0.0.0:3000 on PID: 15
frontend_1  | yarn install v1.16.0
frontend_1  | [1/4] Resolving packages...
frontend_1  | success Already up-to-date.
frontend_1  | Done in 0.68s.
frontend_1  | yarn run v1.16.0
frontend_1  | $ next build
frontend_1  | (node:50) ExperimentalWarning: The fs.promises API is experimental
frontend_1  | Creating an optimized production build...
frontend_1  | > Using external babel configuration
frontend_1  | > Location: "/app/.babelrc"
frontend_1  | 
frontend_1  | Compiled successfully.
frontend_1  | 
frontend_1  | Automatically optimizing pages...
frontend_1  | Warning: You have opted-out of Automatic Static Optimization due to `getInitialProps` in `pages/_app`. This does not opt-out pages with `getStaticProps`
frontend_1  | Read more: https://err.sh/next.js/opt-out-auto-static-optimization
frontend_1  | 
frontend_1  | 
frontend_1  | Page                                                           Size     First Load JS
frontend_1  | ┌ λ /                                                          1.17 kB         121 kB
frontend_1  | ├   /_app                                                      2.88 kB        76.9 kB
frontend_1  | ├ λ /1-click-bom                                               1.05 kB         114 kB
frontend_1  | ├ λ /404                                                       2.92 kB         116 kB
frontend_1  | ├ λ /bom-builder                                               2.96 kB         116 kB
frontend_1  | └ λ /projects/new                                              5.32 kB         125 kB
frontend_1  |     └ css/1a79ca7f41815bc658ec.css                             822 B
frontend_1  | + First Load JS shared by all                                  76.9 kB
frontend_1  |   ├ static/pages/_app.js                                       2.88 kB
frontend_1  |   ├ chunks/270908f2bf4b19e8b537024d3c357c0ed436ee35.207e29.js  4.01 kB
frontend_1  |   ├ chunks/commons.afec7b.js                                   22.3 kB
frontend_1  |   ├ chunks/framework.c61f0e.js                                 40.8 kB
frontend_1  |   ├ runtime/main.df0d1a.js                                     6.27 kB
frontend_1  |   ├ runtime/webpack.1c5199.js                                  746 B
frontend_1  |   └ css/822514b03fd56455f2e7.css                               39.8 kB
frontend_1  | 
frontend_1  | λ  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
frontend_1  | ○  (Static)  automatically rendered as static HTML (uses no initial props)
frontend_1  | ●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
frontend_1  | 
frontend_1  | Done in 16.88s.
frontend_1  | yarn run v1.16.0
frontend_1  | $ nodemon server.js --watch server.js --watch next.config.js
frontend_1  | [nodemon] 2.0.2
frontend_1  | [nodemon] to restart at any time, enter `rs`
frontend_1  | [nodemon] watching dir(s): server.js next.config.js
frontend_1  | [nodemon] watching extensions: js,mjs,json
frontend_1  | [nodemon] starting `node server.js`
frontend_1  | (node:195) ExperimentalWarning: The fs.promises API is experimental
frontend_1  | Defining routes from exportPathMap
frontend_1  | > Using external babel configuration
frontend_1  | > Location: "/app/.babelrc"
frontend_1  | event - compiled successfully

The Content of etc/hosts

127.0.0.1   kitspace.test
127.0.0.1   gitea.kitspace.test

127.0.0.1   localhost
# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
kasbah commented 4 years ago

I think you already have Nginx running on 127.0.0.1, i.e. on your OS (Ubuntu?) directly. If you are running Ubuntu or Debian then you should be able to do:

sudo apt remove nginx

(unless you need it for something else)

You might need to stop the service separately but I think apt remove will do it for you.

AbdulrhmnGhanem commented 4 years ago

Didn't fix it. Also, the build log for the gitea(golang) image had a fatal error.

step 10/23 : RUN if [ -n "${GITEA_VERSION}" ]; then git checkout "${GITEA_VERSION}"; fi  && make clean build
 ---> Running in 16a49cdc5ce1
fatal: not a git repository: /go/src/code.gitea.io/gitea/../.git/modules/gitea

The output from git submodule ffb9f26672cdf2e2ffdca62dbbcaa5d433a3a627 gitea (v1.7.0-dev-2145-gffb9f2667)

Partial log, the previous error is at the bottom.

Step 1/23 : FROM golang:1.13-alpine3.11 AS build-env
1.13-alpine3.11: Pulling from library/golang
cbdbe7a5bc2a: Pull complete
408f87550127: Pull complete
fe522b08c979: Pull complete
52f40d9edaaa: Pull complete
26f4f477de4c: Pull complete
Digest: sha256:f6b068dff49bfa0e88bd784152674365a2118a43a0888d4f78e812157f2303d1
Status: Downloaded newer image for golang:1.13-alpine3.11
 ---> d1021b5f942b
Step 2/23 : ARG GOPROXY
 ---> Running in d85ceca42e99
Removing intermediate container d85ceca42e99
 ---> 49d65e790209
Step 3/23 : ENV GOPROXY ${GOPROXY:-direct}
 ---> Running in 8ebf6842402b
Removing intermediate container 8ebf6842402b
 ---> 482918b75dc5
Step 4/23 : ARG GITEA_VERSION
 ---> Running in 7c2b8e5948c2
Removing intermediate container 7c2b8e5948c2
 ---> bf6275f4ab91
Step 5/23 : ARG TAGS="sqlite sqlite_unlock_notify"
 ---> Running in 04dc4029ed69
Removing intermediate container 04dc4029ed69
 ---> 9860b307a9a1
Step 6/23 : ENV TAGS "bindata $TAGS"
 ---> Running in 2d5b3b25a8f4
Removing intermediate container 2d5b3b25a8f4
 ---> 5b62fe7bbb4a
Step 7/23 : RUN apk --no-cache add build-base git nodejs npm
 ---> Running in dc2702137378
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/27) Installing libgcc (9.2.0-r4)
(2/27) Installing libstdc++ (9.2.0-r4)
(3/27) Installing binutils (2.33.1-r0)
(4/27) Installing libmagic (5.37-r1)
(5/27) Installing file (5.37-r1)
(6/27) Installing gmp (6.1.2-r1)
(7/27) Installing isl (0.18-r0)
(8/27) Installing libgomp (9.2.0-r4)
(9/27) Installing libatomic (9.2.0-r4)
(10/27) Installing mpfr4 (4.0.2-r1)
(11/27) Installing mpc1 (1.1.0-r1)
(12/27) Installing gcc (9.2.0-r4)
(13/27) Installing musl-dev (1.1.24-r2)
(14/27) Installing libc-dev (0.7.2-r0)
(15/27) Installing g++ (9.2.0-r4)
(16/27) Installing make (4.2.1-r2)
(17/27) Installing fortify-headers (1.1-r0)
(18/27) Installing build-base (0.5-r1)
(19/27) Installing nghttp2-libs (1.40.0-r1)
(20/27) Installing libcurl (7.67.0-r0)
(21/27) Installing expat (2.2.9-r1)
(22/27) Installing pcre2 (10.34-r1)
(23/27) Installing git (2.24.3-r0)
(24/27) Installing c-ares (1.15.0-r0)
(25/27) Installing libuv (1.34.0-r0)
(26/27) Installing nodejs (12.15.0-r1)
(27/27) Installing npm (12.15.0-r1)
Executing busybox-1.31.1-r9.trigger
OK: 248 MiB in 42 packages
Removing intermediate container dc2702137378
 ---> 0472c74eca31
Step 8/23 : COPY . ${GOPATH}/src/code.gitea.io/gitea
 ---> 6d33fbe99460
Step 9/23 : WORKDIR ${GOPATH}/src/code.gitea.io/gitea
 ---> Running in ed7f6d0dc9bf
Removing intermediate container ed7f6d0dc9bf
 ---> a0a63b0c8d83
Step 10/23 : RUN if [ -n "${GITEA_VERSION}" ]; then git checkout "${GITEA_VERSION}"; fi  && make clean build
 ---> Running in 16a49cdc5ce1
fatal: not a git repository: /go/src/code.gitea.io/gitea/../.git/modules/gitea
go clean -i ./...
rm -rf gitea dist modules/public/bindata.go modules/options/bindata.go modules/templates/bindata.go modules/public/bindata.go.hash modules/options/bindata.go.hash modules/templates/bindata.go.hash \
        integrations*.test \
        integrations/gitea-integration-pgsql/ integrations/gitea-integration-mysql/ integrations/gitea-integration-mysql8/ integrations/gitea-integration-sqlite/ \
        integrations/gitea-integration-mssql/ integrations/indexers-mysql/ integrations/indexers-mysql8/ integrations/indexers-pgsql integrations/indexers-sqlite \
        integrations/indexers-mssql integrations/mysql.ini integrations/mysql8.ini integrations/pgsql.ini integrations/mssql.ini
npm install --no-save

Edit: running lsof -i:3000 the output indicates it's actually listening to the docker container port

COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
docker-pr 16525 root    4u  IPv6            0t0  TCP *:3000 (LISTEN)
kasbah commented 4 years ago

Did you figure this out? (The gitea error "fatal: not a git repository: /go/src/code.gitea.io/gitea/../.git/modules/gitea" is normal I think, I always see it too but it works fine.)

AbdulrhmnGhanem commented 4 years ago

Sorrowfully, no. I was extremely busy, after the last comment I pruned all the containers and ran docker-compose again but it didn't fix the issue. I read a little about gjtea, I'm curious to know if you have other plans for it aside from the user management; it seems like overkill, we will use a tiny fraction of it. We shall keep the project a simple as possible especially as it's just getting started.

kasbah commented 4 years ago

It will make the architecture more complicated than the "JAM stack" one we currently have. I have considered it for a long time though and think it's worth it. Beyond user accounts it gives us the ability for people not familiar with Git to make projects by uploading files directly. I believe this will let us expand to a whole new user base without having to re-implement a git-hosting and user-management backend ourselves.

AbdulrhmnGhanem commented 4 years ago

why do we need git-hosting? I don't know whether the current version of the site supports versioning for projects or not. Even if we want to support versioning it will be way simpler than git; roughly it will be the content of version X of some board will be Y, the content of version X.1 of some board will be Y.2, and so forth. This can be done as a database table, we don't need fully-fledged version control.

Note: I'm pushing back to get more clarity and if possible reach better architecture, not to oppose you, and not because I can't build the project properly!

kasbah commented 4 years ago

Let's discuss in #15

AbdulrhmnGhanem commented 4 years ago

To make sure I get you clearly. We are going to hook our custom forms (sign in, sign up) to the Gitea endpoints (/user/login/, /user/forgot_password/, /user/sign_up/).

AbdulrhmnGhanem commented 4 years ago

The issue with forwarding the request to /user/login, etc is it returns HTML not JSON response. I tried with Postman. Or are we going to use the API endpoint /api/v1/admin/user/? User management requires being authorized as an admin which needs an access token attached to the request. Doing this from the frontend will expose the access token. How are going to work around this? Can we do this on the Nginx level?

kasbah commented 4 years ago

Adding your quote from matrix/element chat:

  • The initial plan was to forward the request from our site /login/ to Gitea's /user/login/. The issue is the response from Gitea is an HTML not JSON which will make further processing to the data tedious and will require parsing the document. I think this is a bad approach.
  • An alternative plan would be to use a reverse proxy to route Kitspace /login/ to Gitea's api/v1/admin/user/ but the admin endpoint requires being authorized as an admin. We can't put the access token or the header in the frontend so we have to add it through the reverse proxy.
  • For login we can create access tokens through api/v1/{user}/tokens/ yet we have to manage all the JWT thing on the frontend ourselves.
  • I haven't figured out how reset password is going to be handled.
  • some relevant links:
kasbah commented 4 years ago

Thanks @AbdulrhmnGhanem. Some thoughts:

Anything that requires admin is a no-go in my book. I don't think we should ever use the /admin endpoint for user related things.

You should make a WIP: pull-request with your branch by the way. It makes it easier to review the code in one place.

AbdulrhmnGhanem commented 4 years ago
  • Another option might be writing our own login endpoint in Gitea.

Can you elaborate on this?

kasbah commented 4 years ago

Dig into the Gitea source a bit and replicate what's done on /user/login but return JSON. Was just taking a look, maybe it's in this source file? https://github.com/go-gitea/gitea/blob/master/routers/user/auth.go

AbdulrhmnGhanem commented 4 years ago

Yeah, it seems the right file. Bad news it explicitly returns HTML. Are you familiar with Go? I never used it before, I'll see if I can find a resource to get me up rapidly.

  • Can we make do with the HTML response (I know it's not ideal, but it could still work).

I'm quite comfortable with the original python BeautifulSoup so I'll consider JSSoup.

kasbah commented 4 years ago

I tend to use learnxinyminutes.com for languages I need to hack something in but am not familiar with. https://learnxinyminutes.com/docs/go/

Alright, JSSoup looks interesting actually. I'm constantly on the search for a nice JS HTML lib that you can use server and client side. Cheerio is still the best I've found but I'm not happy with it.

AbdulrhmnGhanem commented 4 years ago

I'm afraid it needs node to work so it can't be used in the frontend. :disappointed_relieved:

kasbah commented 4 years ago

Right, well, if that's all you need for now and it makes it easier for you then I'm still fine with it.

AbdulrhmnGhanem commented 4 years ago

I found the parsing HTML approach will add a chain of 6 requests until the response gets to the client, compared to 2 requests using a custom Gitea endpoint. I didn't consider the proxy server as a stage in both cases. Furthermore, it'll make the JWT part a bit crazy. Let's see what I can do.

AbdulrhmnGhanem commented 3 years ago

closed by #18