v0.20 mani sync bug

[mikeboiko]

I just installed the latest version of mani on arch, and I want to report a new bug I noticed.

I think the bug is related to the fact that I store my mani.yaml file within a mani repo. See https://github.com/alajmo/mani/issues/27.

For example, my mani.yaml contains:

  spotidry:
    path: ~/Documents/GitRepos/OpenSource/spotidry
    url: https://github.com/mikeboiko/spotidry

When I run mani -c ~/Documents/GitRepos/Linux/config/mani.yaml sync, my repo ends up in ~/Documents/GitRepos/Linux/config/spotidry

[alajmo]

Thanks for reporting the bug. I tried to re-create it but wasn't able to, here's my config (the mani.yaml file is located in ~/tmp/GitRepos/Linux/config/mani.yaml.:

projects:
  linux:
    path: ~/tmp/GitRepos/Linux/config
    url: https://github.com/alajmo/dashgrid.git

  spotidry:
    path: ~/tmp/GitRepos/OpenSource/spotidry
    url: https://github.com/mikeboiko/spotidry

~/tmp $ tree GitRepos -L 2

GitRepos
├── Linux
│   └── config
└── OpenSource
    └── spotidry

4 directories, 0 files

mani list projects -c ~/tmp/GitRepos/Linux/config/mani.yaml --headers=project,path,relpath
 Project    | Path                                         | Relpath
------------+----------------------------------------------+---------------------------
 linux      | /home/samir/tmp/GitRepos/Linux/config        | .
 spotidry   | /home/samir/tmp/GitRepos/OpenSource/spotidry | ../../OpenSource/spotidry

Could you paste your config.yaml here including where the mani.yaml is placed and also paste the output of mani version.

There's also 2 new env variables you could use by the way, also adds auto-completion which the -c flag doesn't.

 MANI_CONFIG
        Override config file path

 MANI_USER_CONFIG
        Override user config file path

[mikeboiko]

No problem, let's see if can get this figured out :)

My config file is located in /home/mike/Documents/GitRepos/Linux/config/mani.yaml and for the purposes of this test, I've modified it to:

projects:
  Linux:
    path: ~/Documents/GitRepos/Linux
    url: https://github.com/mikeboiko/Linux
    tags: [main]

  spotidry:
    path: ~/Documents/GitRepos/OpenSource/spotidry
    url: https://github.com/mikeboiko/spotidry
    clone: git clone https://github.com/mikeboiko/spotidry --branch dev
    tags: [main]
    env:
      branch: dev

specs:
  custom:
    output: table
    parallel: true
    omit_empty: true

tasks:
  git-status:
    desc: show working tree status
    spec: custom
    cmd: git status -s

I will give the new ENV vars a try. For now, I've just been using this alias:

╭─mike@xps ~/Documents/GitRepos
╰─➤  which mani
mani: aliased to mani -c ~/Documents/GitRepos/Linux/config/mani.yaml

When I have the spotidry repo pulled already, mani sync works:

╭─mike@xps ~/Documents/GitRepos
╰─➤  mani sync

 Project  | Synced
----------+--------
 Linux    | ✓
 spotidry | ✓

However, when I delete the repo locally and run mani sync, mani tries to pull it into Linux/config:

╭─mike@xps ~/Documents/GitRepos/OpenSource
╰─➤  ls
intouch2  tbl2mysql  Vira
╭─mike@xps ~/Documents/GitRepos/OpenSource
╰─➤  mani sync

Project [spotidry] ***********************************************************************************************************************************************************

fatal: destination path 'spotidry' already exists and is not an empty directory.
exit status 128

 Project  | Synced
----------+--------
 Linux    | ✓
 spotidry | ✕

╭─mike@xps ~/Documents/GitRepos/OpenSource
╰─➤  cd ../Linux
total 104K
drwxr-xr-x 3 mike mike 4.0K Apr 11 06:24 ansible
drwxr-xr-x 2 mike mike 4.0K Apr 27 09:23 bin
drwxr-xr-x 7 mike mike 4.0K May  5 14:56 config
drwxr-xr-x 2 mike mike 4.0K May  1 09:19 dotfiles
drwxr-xr-x 8 mike mike 4.0K May  5 14:53 .git
drwxr-xr-x 3 mike mike 4.0K Apr 11 06:24 git
-rw-r--r-- 1 mike mike 2.5K Jul 29  2020 .gitattributes
-rw-r--r-- 1 mike mike  145 Jul 29  2020 .gitignore
-rw-r--r-- 1 mike mike  262 Jul 29  2020 Pipfile
-rw-r--r-- 1 mike mike 9.1K Jul 29  2020 Pipfile.lock
drwxr-xr-x 4 mike mike 4.0K Jul 29  2020 python
-rw-r--r-- 1 mike mike  137 Jul 29  2020 requirements.txt
drwxr-xr-x 5 mike mike 4.0K Apr 27 21:18 scripts
lrwxrwxrwx 1 mike mike   16 Jul 29  2020 setup.cfg -> config/setup.cfg
-rw-r--r-- 1 mike mike  41K May  5 14:51 tags
╭─mike@xps ~/Documents/GitRepos/Linux  ‹master*›
╰─➤  git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   config/mani.yaml
        modified:   dotfiles/.flow.yml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        config/spotidry/

no changes added to commit (use "git add" and/or "git commit -a")
╭─mike@xps ~/Documents/GitRepos/Linux  ‹master*›
╰─➤  mani version
Version: 0.20.0
Commit: e4e36f17079c57ba7fdead6de1dc441233fffe09
Date: 20220504-09:06:53

Please let me know if you need any more details. Thanks!

[mikeboiko]

Furthermore, I got the same results when I used $MANI_CONFIG:

╭─mike@xps ~/Documents/GitRepos/Linux  ‹master*›
╰─➤  unalias mani
╭─mike@xps ~/Documents/GitRepos/Linux  ‹master*›
╰─➤  which mani
/usr/bin/mani
╭─mike@xps ~/Documents/GitRepos/Linux  ‹master*›
╰─➤  export MANI_CONFIG=~/Documents/GitRepos/Linux/config/mani.yaml
╭─mike@xps ~/Documents/GitRepos/Linux  ‹master*›
╰─➤  mani sync

Project [spotidry] ***********************************************************************************************************************************************************

Cloning into 'spotidry'...

 Project  | Synced
----------+--------
 Linux    | ✓
 spotidry | ✕

╭─mike@xps ~/Documents/GitRepos/Linux  ‹master*›
╰─➤  git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   config/mani.yaml
        modified:   dotfiles/.flow.yml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        config/spotidry/

no changes added to commit (use "git add" and/or "git commit -a")

[alajmo]

I think I see the issue now, it's because you override the clone command (when you override the clone command, it doesn't use the path and name attribute):

clone: git clone https://github.com/mikeboiko/spotidry --branch dev

The solution in this case is:

clone: git clone https://github.com/mikeboiko/spotidry ~/Documents/GitRepos/OpenSource/spotidry --branch dev

I might add in the future the ability to specify common git clone options such as branch, sparse-checkout, etc. so you would be able to do:

  spotidry:
    path: ~/Documents/GitRepos/OpenSource/spotidry
    url: https://github.com/mikeboiko/spotidry
    git:
      branch: dev
      sparse: true
    tags: [main]
    env:
      branch: dev

[mikeboiko]

Ahh, that makes sense. I like the git options plan!

[alajmo]

I've added the environment variables MANI_PROJECT_URL and MANI_PROJECT_PATH so the clone command can be simplified to:

projects:

  spotidry:
    path: ~/Documents/GitRepos/OpenSource/spotidry
    url: https://github.com/mikeboiko/spotidry
    clone: git clone $MANI_PROJECT_URL $MANI_PROJECT_PATH
    tags: [main]
    env:
      branch: dev

[mikeboiko]

I updated my clone command to include the new env vars and tested 0.21.1 and everything works great. Thank you!