Migrate to gitlab.com #46

Open Changaco opened 8 years ago

GitLab has been continuously improving, and it should now be possible to migrate Liberapay with only a small effort. The advantages are: it's open source, the gitlab.com instance provides free private repositories, GitLab is a remote company (we're a remote organization). The downsides are: need to migrate from Travis CI to GitLab CI, smaller user community than GitHub.

So, should we migrate?

The missing watch a repo feature is a blocker. Closing this issue for now, we'll reconsider later.

:(

GitLab issue for the "watch a repo" feature: https://gitlab.com/gitlab-org/gitlab-ce/issues/9013

It looks like watching a repo will be implemented soon, it's time to reconsider this.

Watching a repo has been implemented by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3986.

Someone told me in IRC that GitLab CI is great but that you have to set up your own worker, which is a bit of additional work compared to simply using Travis.

We have a full Gitlab / Gitlab CI installation at work, It's a bit painful to setup the first time, but not too much. I'm okay to help with that if you want (I'm managing my workplace's Gitlab since almost 2 years).

I don't think we want our own Gitlab instance unless using gitlab.com turns out to be problematic.

If you use the hosted version, I think you can use also their CI workers, (meaning you just need to write relevant configuration in .gitlab-ci.yml), it should be easy enough to set up. Maybe it's also possible to have an hybrid setup: Hosted Gitlab, self-hosted CI worker ?

Maybe it's also possible to have an hybrid setup: Hosted Gitlab, self-hosted CI worker ?

It's almost certainly possible, but we should try the shared CI first.

P.S. in English there are no spaces before punctuation marks. ;-)

P.S. in English there are no spaces before punctuation marks. ;-)

Yeah, that's a real problem for me... I'll try to be more careful, thanks!

Regarding the migration, a way to smooth up the whole thing could be:

create a repository mirror on Gitlab,
Configure the project to run tests on Gitlab CI as well as on Travis.
Import remaining data on Gitlab (issues, etc.)
Invert the mirroring: Github becames a mirror of the Gitlab repository
See if everything works fine
Remove Travis, Github mirroring, archive the project and Github (or redirect directly to gitlab, if possible).

However, from a personal perspective, I'm not sure if it's really worth it to migrate from GitHub to hosted Gitlab.com / CI. Both companies provide almost the same services, (apart from the private repositories policy, as you pointed earlier).

Migrating to self-hosted Gitlab would make sense to me, to avoid depending on anyone regarding the project hosting, but it will also take more time / resources.

Moving away from a closed source service to an open source one is a gain in itself. There are many people in the free software community who prefer Gitlab over GitHub because the former is open source. In fact not too long ago we had someone who gave up contributing to Liberapay because I told him he'd have to use GitHub.

Moving away from a closed source service to an open source one is a gain in itself.

You're right, however, we don't have any way to verify that the Gitlab.com instance really shares the same code than the open source one. In a way, a hosted service is a closed source service, even if the code is published somewhere else. Also, if the service is shut down or decide to close a project's account, the damage would be exactly the same.

That's why, in my mind, open source is important, but using hosted open source services does not really add much value in itself, compared to self-hosting an open source service. I agree that the former is still better than nothing though.

Self-hosting gives even more control of course, but there are economies of scale in hosting many projects on a shared instance. Simplicity and cost efficiency are big reasons why most people don't self-host.

Got an error I don't fully understand while trying to migrate to GitLabCI:

Cloning into '/builds/jorgesumle/liberapay.com'...
Checking out dec12ed1 as master...
Downloading www/assets/fonts/ubuntu-bold-italic-webfont.ttf (357 KB)
Error downloading object: www/assets/fonts/ubuntu-bold-italic-webfont.ttf (2c33de3): Smudge error: Error downloading www/assets/fonts/ubuntu-bold-italic-webfont.ttf (2c33de34d51065a5e56bae5e1c7c13cd128268b26569838a38db038e6ac9723d): batch request: exec: "ssh": executable file not found in $PATH

Errors logged to /builds/jorgesumle/liberapay.com/.git/lfs/objects/logs/20180401T052759.323459372.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: www/assets/fonts/ubuntu-bold-italic-webfont.ttf: smudge filter lfs failed

In addition to what https://github.com/liberapay/liberapay.com/issues/46#issuecomment-264203840 says, we shouldn't forget about Weblate.

@jorgesumle Looks like git-lfs failed because the ssh command wasn't available.

Yes, and I cannot do anything, as it clones the repository before executing the before_script part. See an attempt to do that. It seems like I would need to create a Docker image with SSH installed in addition to Python.

@jorgesumle Deleting the .lfsconfig file might "solve" the problem by removing the need for SSH.

No, it didn't work. It looks like the problem is the version of git-lfs used. GitLab explains:

Git LFS v1 original API is not supported since it was deprecated early in LFS development

And from I can see the files tracked by git-lfs have this version https://git-lfs.github.com/spec/v1.

@jorgesumle I don't think so, the GitLab doc you linked is about their implementation of the server side of Git-LFS.

@Changaco, you are right.

Solved the Git-LFS issue thanks to the article Migrating Repositories with Git LFS, but now I cannot get posgreSQL to work.

@jorgesumle Looks like the postgres daemon hasn't been launched.

GitLab has non-libre code from Google reCAPTCHA. The community seems to agree to remove it, see vote count.

As I suppose you all know, Microsoft acquired Github, thus this issue may become more relevant? Can't help with the migration, but wish you all the best to get it work :) I am waiting on the other side :)

Almost done, but I need some help making GitLab-CI work with posgresql, see #1143

@Changaco @jorgesumle Can you add me to the gitlab team member so I can at least clone the project? I'm quite familiar with gitlab since I use it since several year and could be some help here.

I cannot add members, because I have the "Maintainer" role.

@Changaco?

https://blog.github.com/2018-10-26-github-and-microsoft/

@MartinDelille I've added you on gitlab as requested. (Don't take this as a sign that I'm ready to migrate the 3 main Liberapay repositories to gitlab, because I'm not.)

@Changaco Yes: the article I posted this morning contains some good argument to stay on Github. Accepting Gitlab merge request would be good indeed to have more contributors!

Unfortunately, it seems that there is a problem with LFS.

Investigation started here: https://gitlab.com/gitlab-org/gitlab-ce/issues/40616#note_112865255

Fixed: https://gitlab.com/liberapay/liberapay.com

It's probably too late to point out, that GitLab is also problematic, since it's a for-profit organization which took $165M venture-capital. The investors want to see profits!

A good alternative i found is Gitea. It's developed by the community! There are also instances hosted by the community. For example Codeberg, a non-profit organization. Or maybe the FSFE allows you to use their instance.

But you need an external CI, like Drone CI.

Thanks a lot. It's a move from a fully closed-source and monopolistic platform. This is a huge step.

Indeed GitLab.com isn't perfect, if the closed-source features aren't used (couldn't quickly find a list) then we will stay independent and will be able to easily move to other intances (running the libre version) if GitLabs venture-capital makes them do bad things.

if the closed-source features aren't used (couldn't quickly find a list)

GitLab pricing - anything that isn't Core (Self-hosted) seem to be their commercial initiative.

If we decide to migrate to GitLab, the file .github/FUNDING.yml should be removed.

Are there still blockers for the migration?

Any progress on this? Is this still planned?

@jorgesumle no need to remove the FUNDING.yml, as it's mirrored on GitHub, those looking around on GitHub they will see all the donations options.

@Changaco GitLab has a program for opensource projects, we took advantage of this for Remmina.

You get an enterprise subscription for free. If you need more info let me known.

As a side note, I've migrated myself Remmina to GitLab a while back, and it's the best move we have done. We were afraid to loose the community effect, but on the contrary the number of contributions went up. On GitHub there many lurkers, while on GitLab are actually active contributors.

Now GitLab has even more features than GitHub, but most important, when you need support, there humans behind the organization and not bots and MS enterprise policies

If you need help let me know, it's really a good change and it's worth the effort

liberapay / liberapay.com

Migrate to gitlab.com #46