"go get -u" failure: generated files are tracked by git

[quantonganh]

                  git pull --ff-only
                  From https://github.com/therecipe/qt
                     dde655c..8fdb852  master     -> origin/master
                  error: Your local changes to the following files would be overwritten by merge:
                        bluetooth/bluetooth.go
                        charts/charts.go
                        core/core.go
                        datavisualization/datavisualization.go
                        dbus/dbus.go
                        designer/designer.go
                        gui/gui.go
                        help/help.go
                        location/location.go
                        multimedia/multimedia.go
                        network/network.go
                        nfc/nfc.go
                        positioning/positioning.go
                        printsupport/printsupport.go
                        purchasing/purchasing.go
                        qml/qml.go
...

Should these files be ignored if qtsetup replace them?

[therecipe]

Usually generated files should be ignored, but services like godoc need the Go code in the repo to function properly. So I'm not sure.

[quantonganh]

What happens if we switch to use git directly and run a git reset before that?

[jeanlucmongrain]

Usually generated files should be ignored, but services like godoc need the Go code in the repo to function properly. So I'm not sure.

@therecipe what/which step exactly overwrite those files?

it's weird, because I never noticed that myself on my won sandbox

[therecipe]

what/which step exactly overwrite those files?

It seems like these files are overridden during qtsetup generate. This can happen, if another Qt version is used (5.7.1 instead of 5.7.0 for example). Or if the HomeBrew Qt version is used on mac, MSYS2 on windows or PkgConfig on linux instead of the official Qt packages.

But because only *.go files are altered, I suspect that qtsetup was executed with QT_STUB=true.

[akamensky]

@quantonganh @therecipe

One solution that I am using is I added this project as git submodule to my whole project. Then in .gitmodules I specify ignore = dirty for this specific library.

This does work as a charm and let's me to pin the project to any commit I like (and update when needed). That's what git submodules were made for.

You are welcome.

Steps to do so:

1. go get github.com/therecipe/qt
2. (in your project directory) git submodule https://www.github.com/therecipe/qt.git src/github.com/therecipe/qt
3. Edit newly create .gitmodules to add "ignore = dirty"
4. git add src/github.com/therecipe/qt
5. git add .gitmodules
6. git commit ... && git push ...

Be aware it will lock it to current commit. use git submodules update ... command to update to latest commit.

[quantonganh]

This can happen, if another Qt version is used (5.7.1 instead of 5.7.0 for example). Or if the HomeBrew Qt version is used on mac, MSYS2 on windows or PkgConfig on linux instead of the official Qt packages.

No, we are using the official Qt package.

But because only *.go files are altered, I suspect that qtsetup was executed with QT_STUB=true.

Yes, as the VM has only 2GB ram.

[quantonganh]

I would like to inform that it also happened on a full installation:

On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   bluetooth/bluetooth.cpp
    modified:   bluetooth/bluetooth.go
    modified:   bluetooth/bluetooth.h
    modified:   charts/charts.go
    modified:   core/core.cpp
    modified:   core/core.go
    modified:   core/core.h

even somtimes core/core.go was deleted:

On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    deleted:    bluetooth/bluetooth.cpp
    modified:   bluetooth/bluetooth.go
    deleted:    bluetooth/bluetooth.h
    deleted:    charts/charts.cpp
    modified:   charts/charts.go
    deleted:    charts/charts.h
    deleted:    core/core.cpp
    deleted:    core/core.go
    deleted:    core/core.h

and it caused our build failed:

Error: no buildable Go source files in /var/lib/jenkins/go/src/github.com/therecipe/qt/core

Why qtsetup generate deleted core/core.go?

[therecipe]

@akamensky Thanks for the info :)

@quantonganh

Why qtsetup generate deleted core/core.go?

It's usually easier to just delete the files first instead of overriding them. (I think there were problems, especially on windows with overriding)

Also this makes sure that there are no artifacts from previous generations. For example: The QT_STUB mode only generates *.go files, so I would need to manually remove the *.h, *.cpp, and cgo_* files first, instead of just deleting the whole folder. Just to be able to override the old *.go file.

But I got an idea about the changed files, could it be that only the permission of the files changed and not the content?

About the missing files, I'm not sure. It could be that git simply needs some time to process the changes. Or are these files really missing after the setup?

[therecipe]

Could be that the generated code is inconsistent at the moment. I'm investigating.

[therecipe]

The code generation should be consistent again with https://github.com/therecipe/qt/commit/fdef10e5ac52b888704ee5c92d2f4adc720a2a95

[quantonganh]

could it be that only the permission of the files changed and not the content?

No, for e.g, bluetooth/bluetooth.go:

diff --git a/bluetooth/bluetooth.go b/bluetooth/bluetooth.go
index 4e2253e..4619fcc 100755
--- a/bluetooth/bluetooth.go
+++ b/bluetooth/bluetooth.go
@@ -2,26 +2,13 @@

 package bluetooth

-//#include <stdint.h>
-//#include <stdlib.h>
-//#include "bluetooth.h"
 import "C"
 import (
-       "fmt"
-       "github.com/therecipe/qt"
        "github.com/therecipe/qt/core"
        "github.com/therecipe/qt/network"
-       "runtime"
        "unsafe"
 )

-func cGoUnpackString(s C.struct_QtBluetooth_PackedString) string {
-       if len := int(s.len); len == -1 {
-               return C.GoString(s.data)
-       }
-       return C.GoStringN(s.data, C.int(s.len))
-}
-

It could be that git simply needs some time to process the changes. Or are these files really missing after the setup?

They are really missing after the setup.

The code generation should be consistent again with fdef10e

I'm sorry. What do you mean by "consistent"? Does this fix the missing files problem?

[akamensky]

@quantonganh

If any files are deleted after running qtsetup, this means your qtsetup did not load those frameworks. This happens when the qt_dir/qt_version are not set.

You should have export QT_VERSION= with your QT version, as well as export QT_DIR= with the path to QT installation. I understand it suppose to detect those, but to be safe you need to export them.

After you export those you should do go get github.com/therecipe/qt/cmd/... to make sure your qtsetup is up to date. And then (with those same variables set) run qtsetup.

To confirm where it looks for the frameworks you can do $GOPATH/bin/qtsetup check desktop. The output should contain:

INFO[0000] QT_VERSION: 5.7.1
INFO[0000] QT_VERSION_MAJOR: 5.7
INFO[0000] QT_DIR: /Users/username/Qt5.7.1
INFO[0000] QT_STUB: false
INFO[0000] QT_HOMEBREW:
INFO[0000] IsHomeBrewQtDir: false
INFO[0000] UseHomebrew: false
INFO[0000] QT_DARWIN_DIR: /Users/username/Qt5.7.1/5.7/clang_64

If relevant QT_VERSION and QT_DIR are not matching with your installed Qt, that means you did not set environment variables properly.

[quantonganh]

@akamensky It was run along with the environment variables, not export then run separately. There is no reason to missing.

Here're the output when running qtsetup:

                  time="2017-01-03T10:15:48Z" level=info msg="running setup/check desktop"
                  time="2017-01-03T10:15:48Z" level=info msg="VERSION: 0.0.4"
                  time="2017-01-03T10:15:48Z" level=info msg="GOOS: linux"
                  time="2017-01-03T10:15:48Z" level=info msg="GOARCH: amd64"
                  time="2017-01-03T10:15:48Z" level=info msg="GOVERSION: go1.7.3"
                  time="2017-01-03T10:15:48Z" level=info msg="GOROOT: /usr/local/go"
                  time="2017-01-03T10:15:48Z" level=info msg="GOPATH: /var/lib/jenkins/go"
                  time="2017-01-03T10:15:48Z" level=info msg="GOBIN: /var/lib/jenkins/go/bin"
                  time="2017-01-03T10:15:48Z" level=info msg="HASH: 8fdb852dd5e830c8a71eb36f976586042922a9b0"
                  time="2017-01-03T10:15:48Z" level=info msg="QT_VERSION: 5.7.0"
                  time="2017-01-03T10:15:48Z" level=info msg="QT_DIR: /opt/Qt5.7.0"
                  time="2017-01-03T10:15:48Z" level=info msg="QT_STUB: true"
                  time="2017-01-03T10:15:48Z" level=info msg="Distro: ubuntu"
                  time="2017-01-03T10:15:48Z" level=info msg="QT_PKG_CONFIG: "

[therecipe]

@quantonganh

They are really missing after the setup.

Ah yes, this is expected because you used the STUB mode. Which doesn't generates .h, .cpp or cgo files at all. (Because they are not needed) And the .go files are changed, because they are missing all the cgo stuff when build with QT_STUB

I'm sorry. What do you mean by "consistent"? Does this fix the missing files problem?

A few commits (5 max) prior to fdef10e the generated code could differ after qtsetup generate (even when used "normally" without QT_STUB being set), because the functions could be generated in a different order. And no, this is not the cause of the missing files. (It is QT_STUB explained above)

@akamensky

If any files are deleted after running qtsetup, this means your qtsetup did not load those frameworks. This happens when the qt_dir/qt_version are not set.

Yes, this or if you useQT_STUB=true

[jeanlucmongrain]

@quantonganh I strongly suggest to simply git reset --hard before run anything