`wstool update` returns `error: Entry 'XXX' not uptodate. Cannot merge.`

[130s]

Many times wstool update -t command fails for me, where in the current local branch in the failed package there's no change made.

I tried to come up with a reproduction case but I couldn't. So this ticket may be closed until someone provides scenario, but I just wanted to dump this error message here.

Error message example:

[ros-industrial-consortium/roadmapping] Updating /home/rosnoodle/cws_industrial/src/ros-industrial-consortium/roadmapping
Rebasing repository
error: Entry 'RoadmappingDocument.md' not uptodate. Cannot merge.
fatal: Could not reset index file to revision 'remotes/origin/master'.
WARNING [vcstools] Command failed: 'git reset --keep remotes/origin/master'
 run at: '/home/rosnoodle/cws_industrial/src/ros-industrial-consortium/roadmapping'
 errcode: 128:
error: Entry 'RoadmappingDocument.md' not uptodate. Cannot merge.

fatal: Could not reset index file to revision 'remotes/origin/master'.

[/vcstools]
[ros-industrial-consortium/hilgendorf] Updating /home/rosnoodle/cws_industrial/src/ros-industrial-consortium/hilgendorf
[ros-industrial-consortium/hilgendorf] Done.
[youbot/youbot_driver] Updating /home/rosnoodle/cws_industrial/src/youbot/youbot_driver
[youbot/youbot_driver] Done.
[ipa320/ipa_seminar] Updating /home/rosnoodle/cws_industrial/src/ipa320/ipa_seminar
[ipa320/ipa_seminar] Done.
[ipa320/brics_showcase_industry] Updating /home/rosnoodle/cws_industrial/src/ipa320/brics_showcase_industry
[ipa320/brics_showcase_industry] Done.
Exception caught during install: Error processing 'ros-industrial-consortium/roadmapping' : [ros-industrial-consortium/roadmapping] Update Failed of /home/rosnoodle/cws_industrial/src/ros-industrial-consortium/roadmapping

ERROR in config: Error processing 'ros-industrial-consortium/roadmapping' : [ros-industrial-consortium/roadmapping] Update Failed of /home/rosnoodle/cws_industrial/src/ros-industrial-consortium/roadmapping

And there's no change in the package:

$ cd src/ros-industrial-consortium/roadmapping/
$ git branch -vv
* master 400ad6b [origin/master: behind 2] Updated README.md link to the Roadmap document
$ git status
On branch master
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

nothing to commit, working directory clean

Actually, the similar error occurred for multiple packages today when I first ran wstool update with this .rosinstall file (for 5 or more packages I remember). As I just repeated running the same command, errors somehow got resolved but this last one remains. Now I manually ran git pull in the package directory, it ran just fine pulling upstream change. Then I ran wstool update again that resulted in success.

The entire .rosintall file just in case it helps:

######################## From here Industrial ########################
- git: { local-name: start-jsk/open_industrial_ros_controllers, uri: 'https://130s@github.com/start-jsk/open_industrial_ros_controllers.git', version: hydro-devel }
- git: { local-name: start-jsk/denso, uri: 'https://github.com/start-jsk/denso.git', version: hydro-devel }

- git: { local-name: ros-industrial/robotiq, uri: 'https://github.com/ros-industrial/robotiq.git' }
#- git: { local-name: gt-ros-pkg/ethercat-soem, uri: 'https://github.com/gt-ros-pkg/ethercat-soem.git', version: hydro-devel }  # Required by robotiq
- git: { local-name: ros-industrial/kuka_experimental, uri: 'https://github.com/ros-industrial/kuka_experimental.git' }
- git: { local-name: ros-industrial/abb_experimental, uri: 'https://github.com/ros-industrial/abb_experimental.git', version: indigo-devel }
- git: { local-name: ros-industrial/staubli, uri: 'https://github.com/ros-industrial/staubli.git' }
- git: { local-name: ros-industrial/staubli_experimental, uri: 'https://github.com/ros-industrial/staubli_experimental.git', version: indigo-devel }
- git: { local-name: ros-industrial/crcl, uri: 'https://github.com/ros-industrial/crcl.git', version: master }
- git: { local-name: ros-industrial/keyence_experimental, uri: 'https://github.com/ros-industrial/keyence_experimental.git', version: master }
- git: { local-name: ros-industrial/industrial_core, uri: 'https://github.com/ros-industrial/industrial_core.git', version: indigo-devel }
#- git: { local-name: ros-industrial/industrial_calibration, uri: 'https://github.com/ros-industrial/industrial_calibration.git', version: indigo-devel }
- git: { local-name: ros-industrial/universal_robot, uri: 'https://github.com/ros-industrial/universal_robot.git' }
- git: { local-name: ros-industrial/rep, uri: 'https://github.com/ros-industrial/rep.git', version: master }
- git: { local-name: ros-industrial/abb, uri: 'https://github.com/ros-industrial/abb.git' }
- git: { local-name: ros-industrial/fanuc, uri: 'https://github.com/ros-industrial/fanuc.git', version: indigo-devel }
- git: { local-name: ros-industrial/fanuc_experimental, uri: 'https://github.com/ros-industrial/fanuc_experimental.git' }
- git: { local-name: ros-industrial/ethercat-soem, uri: 'https://github.com/ros-industrial/ethercat-soem.git', version: indigo-devel }
- git: { local-name: ros-industrial/motoman, uri: 'https://github.com/ros-industrial/motoman.git', version: indigo-devel }
- git: { local-name: ros-industrial/industrial_training, uri: 'https://github.com/ros-industrial/industrial_training.git' }
- git: { local-name: ros-industrial/siemens_experimental, uri: 'https://github.com/ros-industrial/siemens_experimental.git', version: hydro-devel }
- git: { local-name: ros-industrial/ros_canopen, uri: 'https://github.com/ros-industrial/ros_canopen.git', version: indigo-devel }
- git: { local-name: ros-industrial/industrial_moveit, uri: 'https://github.com/ros-industrial/industrial_moveit.git', version: indigo-devel }

- git: { local-name: ros-industrial-consortium/descartes, uri: 'https://github.com/ros-industrial-consortium/descartes.git' }
- git: { local-name: ros-industrial-consortium/descartes_tutorials, uri: 'https://github.com/ros-industrial-consortium/descartes_tutorials.git' }
#- git: { local-name: ros-industrial-consortium/fermi, uri: 'https://github.com/ros-industrial-consortium/fermi.git' }
- git: { local-name: ros-industrial-consortium/roadmapping, uri: 'https://github.com/ros-industrial-consortium/roadmapping.git', version: master }
- git: { local-name: ros-industrial-consortium/hilgendorf, uri: 'https://github.com/ros-industrial-consortium/hilgendorf.git', version: hydro-devel }

- git: { local-name: youbot/youbot_driver, uri: 'https://github.com/youbot/youbot_driver.git' }

- git: { local-name: ipa320/ipa_seminar, uri: 'https://github.com/ipa320/ipa_seminar.git', version: indigo_dev }
- git: { local-name: ipa320/brics_showcase_industry, uri: 'https://github.com/ipa320/brics_showcase_industry.git', version: groovy_dev }
#- git: { local-name: ros-industrial-consortium/ipa_seminar, uri: 'https://github.com/ros-industrial-consortium/ipa_seminar.git' }
######################## Til here Industrial ########################

I noticed that one last package above (roadmapping) is the one that's listed at the latest in .rosinstall file among all other packages that initially emitted similar error.

Today isn't the first time I saw this error; I've been seeing this since long ago.

[tkruse]

Hi Kentaro,

that seems worrying indeed. Running wstool update again should not produce other behavior in general. The only thing I can imagine now is that if too many things happen in parallel, you get network issues, and errors are related to those.

Have you googled the error? Like this page: http://stackoverflow.com/questions/1248029/git-pull-error-entry-foo-not-uptodate-cannot-merge

Note that wstool does not git pull, but instead does git reset --keep remotes/origin/master (or another branch). So to reproduce with git only, use that command. git pull is considered a "dangerous" command as it can have undesired effects when used wrongly.

[skohlbr]

I just ran into this also. I had a workspace setup that was not updated for 3 months, so with a new call of wstool update, many repos had updates. I was greeted with errors for at least 10 repos as described above. I reran the update a second time with no change. I then ran wstool and it showed "C" as repo status for all repos that produced errors. I then re-ran wstool update a third time and suddenly it worked. Might be coincidence but from this single data point it looked like doing the plain wstool call magically made the update work afterwards.

[tkruse]

I believe this git bug is most likely the culprit: http://git.661346.n2.nabble.com/Possibly-spurious-not-uptodate-Cannot-merge-td1654935.html http://stackoverflow.com/questions/878554

It may help to post more logs here, and the git version used.

[a-price]

I'm seeing this problem as well. Running in a Docker container on a build server. (Package name has been replaced with XXX, file name with YYY.)

Output from wstool update:

[ar_track_alvar] Updating /root/catkin_ws/src/ar_track_alvar [ar_track_alvar] Done. [XXX] Updating /root/catkin_ws/src/XXX remote: Counting objects: 10, done. remote: Compressing objects: 100% (7/7), done. remote: Total 10 (delta 5), reused 0 (delta 0) Unpacking objects: 100% (10/10), done. From servername/XXX a77dc85..ecf8052 master -> origin/master WARNING [vcstools] Command failed: 'git reset --keep remotes/origin/master' run at: '/root/catkin_ws/src/XXX' errcode: 128: error: Entry 'YYY' not uptodate. Cannot merge. fatal: Could not reset index file to revision 'remotes/origin/master'. [/vcstools] [rasd_aim_robots] Updating /root/catkin_ws/src/rasd_aim_robots [rasd_aim_robots] Done. [rasd_aim_tooling] Updating /root/catkin_ws/src/rasd_aim_tooling [rasd_aim_tooling] Done. Exception caught during install: Error processing 'XXX' : [XXX] Update Failed of /root/catkin_ws/src/XXX

ERROR in config: Error processing 'XXX' : [XXX] Update Failed of /root/catkin_ws/src/XXX

Output from git --version:

git version 1.9.1

Output from git reset --keep remotes/origin/master:

error: Entry 'YYY' not uptodate. Cannot merge. fatal: Could not reset index file to revision 'remotes/origin/master'.

Output from git status:

On branch master Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded. (use "git pull" to update your local branch)

nothing to commit, working directory clean

In my experiments, repeatedly calling wstool update did nothing. However, cding into the package directory and calling git status seemed to help the situation, as on returning to catkin_ws/src, wstool update works as expected. I have tried this on three instances of the Docker container successfully, so there seems to be something repeatable going on.

The server is on an enterprise network, so I can't ship you the docker container with the pointers to the repo, but are there any tests that I can run on my end now that I can reproduce the error?

[tkruse]

If you can reproduce, we need to know 2 things:

1. What is the filesystem difference between the case it works and the case it does not.
2. Which of the low-level git commands from vcstools causes this.

It would be great if you could provided these, once you have the time. Say so if you need help getting this done.

[a-price]

Running diff -r on copies of the source directory before and after the git status shows:

Binary files src_after_status/XXX/.git/index and src_partial_up/XXX/.git/index differ

I'm not sure what to test next. Running cmp says the two index files differ at line 1, byte 16, but beyond that I'm not sure how to inspect the differences.

[a-price]

Ok, I tried running a diff on the directory before and after the wstool update. Here's the output:

root @ afd98e3c0e54:~/catkin_ws# diff -r src/ src_after_update/ Only in src_after_update/XXX/.git: FETCH_HEAD Only in src_after_update/XXX/.git/logs/refs/remotes/origin: master Only in src_after_update/XXX/.git/objects: 52 Only in src_after_update/XXX/.git/objects: 68 Only in src_after_update/XXX/.git/objects: 6b Only in src_after_update/XXX/.git/objects: 70 Only in src_after_update/XXX/.git/objects: 81 Only in src_after_update/XXX/.git/objects: ac Only in src_after_update/XXX/.git/objects: c7 Only in src_after_update/XXX/.git/objects: e5 Only in src_after_update/XXX/.git/objects: ec Only in src_after_update/XXX/.git/objects: fa Only in src_after_update/XXX/.git/refs/remotes/origin: master

[a-price]

For what it's worth, I've gotten around the issue by calling wstool status before calling wstool update. If I can help troubleshoot the root problem further please let me know, but I'll need some guidance on what to try.

[tkruse]

Hi, sorry, don't have much time to think about this. So from what we can see is that the fetching of origin/master is somehow key to the failure. Probably wstool status also involves git fetch, meaning it also gets origin/master. Can you reveal anything else that would be particular about the difference between your local master and origin/master? Is it a fast-forward, how many commits, how large? Are you on master when you run wstool update? Things like that. it might be possible for you to send a gitk screenshot, or an ascii-graph with sensitive information masked out.

The other thing I would be interested in is to run the git shell commands on the command line one by one, exactly as wstool does, and see whether that reproduces the problem. If so then either the commands have a bug, or git a bug. I am not sure if the --verbose option prints the commands out for you, or if you'd be willing to patch vcstools (our library used by wstool) to see the commands printed out.

[tompe17]

We are seeing this also. Especially for people that do update seldom.

[ayrton04]

We're also seeing this, and it is also solved by running wstool status before wstool update. Example output:

[example_package] Updating /path/to/workspace/src/example_package
WARNING [vcstools] Command failed: 'git reset --keep remotes/origin/master'
 run at: '/path/to/workspace/src/example_package'
 errcode: 128:
error: Entry 'example_package/CHANGELOG.rst' not uptodate. Cannot merge.

$ cd /path/to/workspace/src/example_package
$ git status
On branch master
Your branch is behind 'origin/master' by 10 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

nothing to commit, working directory clean

[MichaelGrupp]

@tkruse I also ran into this problem in https://github.com/googlecartographer/cartographer_ros/pull/1041 and tested a bit. The crucial part of the workaround with wstool status seems to be that git status "refreshes the index" (https://git-scm.com/docs/git-status#_background_refresh).

If you're running wstool status && wstool update, you're essentially running sth like this:

git status
git fetch
git reset --keep remotes/origin/master

If you explicitly disable index refreshing in git status as explained in the manpage, it crashes again:

git --no-optional-locks status
git fetch
git reset --keep remotes/origin/master  # dies with "...not uptodate. Cannot merge"

Only refreshing the index without status can be done by:

git update-index --refresh
git fetch
git reset --keep remotes/origin/master 

and this fixes the issue. I guess git reset is just missing an implicit refresh as opposed to git pull (which involves merge, and thus should have the fix for the old Git bug you mentioned before). I'm not sure if this is a bug in git reset, but I opened a PR in vcstools to include the fix: https://github.com/vcstools/vcstools/pull/144

---

Probably wstool status also involves git fetch, meaning it also gets origin/master.

git fetch doesn't affect this and is not called by wstool status