Proposal: Import/Migration of all repos in your data-dir should work with a single cli or web-command

[Cyber1000]

Edit :exclamation: : Changing the title, since there is just one point left, my proposal to imports of local repos can be read from starting here: https://github.com/go-gitea/gitea/issues/4474#issuecomment-583788178

---

• Gitea version (or commit ref): fe78154
• Operating system: ubuntu (gitea within docker)
• Database (use [x]):
  • [ ] PostgreSQL
  • [ X] MySQL
  • [ ] MSSQL
  • [ ] SQLite

Description

• In https://docs.gitea.io/en-us/backup-and-restore/ "gitea dump" is introduced. Wouldn't it be the same if I just save my docker-volume and make backup of mysql-db? I don't wan't to zip my whole repo-dir, it is backuped daily file per file.
• Is it fine to have a cleartext password in app.ini (which is also saved with gitea dump all the time). Wouldn't it be better to cache this in another file (inside docker or also outside if you don't use docker) to exclude this from daily backup.
• Is it somehow possible to place all your git-repos into /data/git/repositories/[username]/ and then do some kind of import? (Webgui or command line). If not is there an easy way to implement this? For example with changes to migration-task?

Thanks, Cyber1000

[Cyber1000]

I've done some research and came to this solution (base is a docker gitea configuration):

1. Backing up my volume with all the configs (except repositories), backing up my repositories and dumping the mysql-database should include everything. So I may go with this ...
2. I came to a solution. Having a script with following content:

!/bin/bash

echo "#File is created automatically, don't change things in here" > ./data/gitea/conf/app.ini cat ./data/gitea/conf/app.part.ini>>./data/gitea/conf/app.ini cat ./data/gitea/conf/app.pwd.ini>>./data/gitea/conf/app.ini docker-compose up -d

app.part.ini is everything except password app.pwd.ini is:

[database] PASSWD = xxxxxx

On merging, two database-sections are in the resulting file, but that doesn't matter. Solution could be better (I have to think to start this script instead of a simple docker-compose up, everytime I change something), but for my purposes it's sufficent.

3. This is somehow my mainpoint for now, since I have a handful of small repos. For now I've turned on IMPORT_LOCAL_PATHS and I'm migrating one by one directly from filesystem. But it's a little bit anoying. A think it would be nothing more than to traverse the directory of a user (or all directories in repositories?), check if the path is entered in database and if it is not -> internally execute the Migrate-Command and pass the name of the found folder, path and user as arguments. More or less, I don't have the time to go deeper, at least for now,

Thanks, Cyber1000

[Cyber1000]

Just a follow up, cause I did a cleanup to my configuration:

1. The approach I used is just fine, with backing up the mysql-db and my repo separately without gitea dump
2. I have my own template in ./data/etc/templates/app.ini I can put environment variables in there (more can be found here https://github.com/go-gitea/gitea/blob/70d2244e49e60e11877f850803d33ef1e3900fa6/docker/root/etc/s6/gitea/setup):

   [database]
   DB_TYPE  = mysql
   HOST     = database
   NAME     = gitea
   USER     = $DB_USER
   PASSWD   = $DB_PASSWD

I didn't make everything variable (like the database), since I just wanted to mask the password/user, and some other small things. My docker-compose.yaml (just relevant parts):

environment:
      - USER_UID=1000
      - USER_GID=1000
      - DB_USER=${DB_USER}
      - DB_PASSWD=${DB_PASSWD}
   volumes:
      - ./data/etc/templates/app.ini:/etc/templates/app.ini

• Make sure that the real app.ini isn't present when starting your docker-container, there is some magic within gitea, which takes the template from /etc/templates/app.ini, replaces all environment variables and saves in gitea/conf/app.ini
• I'm backing up just my template file and not the app.ini (since it can be recreated from my template) Passwords are stored in an .env file which isn't backuped either

[Cyber1000]

Changing title of this issue, since the other 2 points are well working.

An easy import of all local stored repos would be nice on first start (or by triggering it) either from webui or from cli (or both)