Git auto push does not actually push changes to the repository on Windows

[wittenator]

Bug classification

I tried to setup a pipeline for pdf creation from latex files in gitlab. Part of that endeavour is the git push support from betterbibtex. I set it up, added the git config in the repo and exported the collection with the right flag. Whenever I add a new item to the collection, the .bib file gets updated. Sadly the actual push to the remote git repo does not happen. I tried both saving the ssh key and the credentials for gitlab, because I suspected that that may be a reason for a silent failure. But it didn't help. Did anyone encounter similar problems with this feature?

Exporting references

• FYA494LP (BibTeX)

Non-export problems with BBT

• EX8RDSJ5

[retorquere]

Can you confirm your repo has a .git/config file that has push = true?

[wittenator]

Yes, there is an entry: [zotero "betterbibtex"] push = true

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6717.issue-972 ("more logging for #972").

[wittenator]

T4D4R2MB That's pretty interesting. I had a a look at the logs and apparently it doesn't even start the procedure. Just to be sure that I didn't fuck up the setup: Do I have to do anything else besides exporting the collection to an existing git repo and setting the config flag?

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6720.issue-972 ("more debug logging for #972").

[retorquere]

It does start the procedure in the sense that it tries to probe if it's been set up, but it silently doesn't finish the probe -- how it does that I don't know. Can you try again with 6720?

There's nothing to configure outside these two:

• Set up collection for autoexport
• Configure with git config zotero.betterbibtex.push true

in any order, and that should be it. It works on my end. The new log should be able to tell me more.

[wittenator]

WMC7QTRI Then it can't be that. Hmm, that's getting really interesting!

[retorquere]

I have found at least part of the problem, but circleCI has an outage which prevents a release of a new debug release for further testing. As soon as that resolves we can proceed.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6726.issue-972 ("only test for parent dir existence, not bib file").

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6725.issue-972 ("only test for parent dir existence, not bib file").

[retorquere]

And finally, it's back. Can you get me a new log from 6725?

[wittenator]

UB8H79FL By mistake I destroyed my last OS and had to do a fresh install from scratch. Interestingly enough, even after the fresh install this problem persists.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6744.issue-972 ("even more logging").

[retorquere]

That's a pretty costly mistake.

It looks like it's just suddenly stopping mid-detect. I log whether it could find the config yes/no and it's simply not getting there. You do have extended debugging on right? Can you try again with 6744?

[wittenator]

HLIUN39U For every new build I restart the whole application with debugging enabled and try to add a file to the exported collection or force an export with the corresponding menu option. Is there an option to enhance the output logs even more?

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6747.issue-972 ("whut. isFile fails?").

[retorquere]

It says it can't open /home/tim/Projects/the_status_quo_of_energy_sector_blockchain/text/.git/config. Can you try again with 6747?

[retorquere]

(there's no way to expand the logs more. I was just surprised to see the logging stop mid-detect, but I think I may have a handle on that particular problem now)

[wittenator]

8N58NXTT Ahh, I see. Betterbibtex seems to assume that any export is always saved to the root of a git repo (I exported to a folder in a git repo until now - that could be a feature request.) Now it also starts logging, but still doesn't push it.

[retorquere]

I do assume the bib file is in the repo root. It's a performance issue -- I do not want to walk all parent directories to the root of the disk just to find out if I'm in a git repo.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6750.issue-972 ("even more logging").

[retorquere]

You know the spiel

[wittenator]

UMCYBV5D Hmm, I think that a note in the docs could be really helpful. Should I open an enhancement ticket ?

[retorquere]

You can edit the page yourself here: https://github.com/retorquere/zotero-better-bibtex/edit/master/docs/push-and-pull.md

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6758.issue-972 ("ambiguous true").

[retorquere]

as always

[retorquere]

Do you have a windows machine you could test on BTW? I may have a solution for files not in the root of the repo but there are cross-platform issues I need to assure I've tackled.

[retorquere]

(and I don't have a windows machine)

[wittenator]

I am going to send the output later today. And I have a Windows machine and a VM to test on (Win7 and Win10 respectively). Did you also stumble upon a way to get the git root folder via git rev-parse --show-toplevel and use it with command substitution? BTW: Is there any documentation on how to build the .xpi from the repo? Then I could also help in finding the bug myself.

[retorquere]

If you clone the repo, and you have node and yarn installed, this should do it:

git checkout 972
yarn install
yarn run build

the xpi will end up in the xpi directory. There's also a second xpi there (debug-bridge), but you don't want to install that unless you want to run automated tests, which require a lot more setup. Having the debug bridge installed is a security risk (to your library, not to your computer in general), you really don't want it unless you're running tests on a system you trust.

The latest code in the 972 branch has code to do pushes of files deeper in the repo. Tests on that are running and it should be announced here in a few minutes by @blip-bloop.

I've looked at git rev-parse --show-toplevel but finding the root isn't the problem (rev-parse is going to do essentially the same, walking the parent dirs until it finds .git in them) but to find the relative path between the repo root and the bibfile. The javascript environment BBT is running in doesn't have an API to get a relative path from two absolute paths; I'm doing string comparisons now but that's a little iffy cross-platform, and I can't test on Windows.

[retorquere]

(missed one: what do you mean by command substitution?)

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.139.6780.issue-972 ("deeper git pushes").

[retorquere]

The stuff we're looking at lives here.

[retorquere]

(the build process outputs some lint warnings and an error from the docs parser, these can be ignored. If the build fails for other errors, those cannot be ignored)

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.141.6801.issue-972 ("new build").

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.141.6804.issue-972 ("compacter").

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.0.141.6806.issue-972 ("better varnames").

[retorquere]

Any feedback? I think I have it now, even for bib files that don't sit at the repo root, but confirmation would be much appreciated.

[retorquere]

Assuming fixed

[wernersa]

This is still an issue for Windows.

Getting this in the log:

{better-bibtex +16} git.init: git not found: <Error: Executable not found: git.exe# SubprocessWin.pathSearch<@resource://gre/modules/subprocess/subprocess_win.jsm:162:17 TaskImpl_run@resource://gre/modules/Task.jsm:319:42 TaskImpl@resource://gre/modules/Task.jsm:277:3 createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14 pathSearch@resource://gre/modules/Subprocess.jsm:157:16 init@chrome://zotero-better-bibtex/content/common.js:53709:30 init@chrome://zotero-better-bibtex/content/common.js:53934:15 init@chrome://zotero-better-bibtex/content/common.js:54703:15 >

Checked system environment "PATH" which includes: C:\Program Files\Git\cmd

Also, the conditional inclusion of .exe should not be neccesary, as the Subprocess.pathSearch() function parses the PATHEXT environment, which includes the extensions.

[retorquere]

Is PATHEXT always set on Windows? I don't have a Windows machine.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.1.45.3385 ("pathsearch does extensions?")

Install in Zotero by downloading test build 5.1.45.3385, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

Can you try 3385?

[wernersa]

Seems like it https://en.wikipedia.org/wiki/Environment_variable#Windows

Running the "git" command in CMD shouldnt be possible if not.

Still fails to find git when trying 3385: (3)(+0000016): {better-bibtex +16} git.init: git not found: <Error: Executable not found: git# SubprocessWin.pathSearch<@resource://gre/modules/subprocess/subprocess_win.jsm:162:17 TaskImpl_run@resource://gre/modules/Task.jsm:319:42 TaskImpl@resource://gre/modules/Task.jsm:277:3 createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14 pathSearch@resource://gre/modules/Subprocess.jsm:157:16 init@chrome://zotero-better-bibtex/content/common.js:55012:30 init@chrome://zotero-better-bibtex/content/common.js:55237:15 init@chrome://zotero-better-bibtex/content/common.js:56006:15 >

[retorquere]

I have no idea what CMD does here to be honest. I didn't know about pathext, I thought .exe was just something hard-coded into it.

So if pathsearch searches PATH, and git.exe is in your PATH, why isn't pathsearch finding it?

[wernersa]

Also tried running as administrator, doesn't make a difference.

Noticed that my environment %PATH% variable was named "Path", tried changing this to "PATH" as windows reads the environment case-insensitive. Still no change.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.1.45.3388 ("EDTF date export")

Install in Zotero by downloading test build 5.1.45.3388, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

That's strange because looking into the sources of pathSearch, the change from Path to PATH would have been the most likely cause of the error -- pathSearch looks only at PATH. I've added a workaround for that in 3388, can you see if that behaves better? Maybe there's more places where pathSearch release on case-sensitivity of environment variable names.

Otherwise I'd have to write a pathSearch replacement myself. Not a huge problem as it's fairly simple, but I'd prefer to use existing platform code where I can.

[wernersa]

Great! It works perfectly 👍 !

Output from log:

(3)(+0000018): {better-bibtex +18} pathSearch: looking for git in C:\Program Files (x86)\Zotero;C:\Python37\Scripts\;C:\Python37\;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\Broadcom\Broadcom 802.11 Network Adapter;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\ProgramData\chocolatey\bin;C:\Program Files\Git\cmd;C:\Program Files\nodejs\;C:\Program Files\PuTTY\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;

(3)(+0000140): {better-bibtex +140} pathSearch: git found at C:\Program Files\Git\cmd\git.EXE

Output from git log:

> git log
commit ada6eb995dd721295xx415x3039193f016e51019 (HEAD -> master, origin/master, origin/HEAD)
Author: Werner Sævland <werner.saevland@gmail.com>
Date:   Sun Jan 27 16:33:00 2019 +0100

    BibLaTeX updated by Better BibTeX for Zotero

Great job Emiliano 🥇 !

The only downside with Zotero.Utilities.Internal.exec() is that it opens 5 different cmd windows sequentially in the foreground, which could get a bit annoying when exporting on update. It would be great if it could be possible to execute all commands in the same process, of if the process launched was done silently (in the background). I'm not sure if either is possible though.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.1.45.3395 ("startHidden")

Install in Zotero by downloading test build 5.1.45.3395, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".