Left over user-local authtoken.secret from previous install can cause GUI and CLI failure

[edouardswiac]

Hello, I tried to install the zerotier mac app and it worked but the .app wont start (Authorization token invalid or ZeroTier One service not running.) and there is also no zt0 network interface. I used to be on Yosemite before and things went wrong after the upgrade.

Not sure where the issue is. I had the problem with the cli disappearing too (solved promptly thanks to a fix posted on that other issue) but I have not seen anything like my pb on the issues here.

Thanks.

[adamierymenko]

Hmm… is this a fresh install on El Capitan?

Can you open activity monitor and see of zerotier-one is running? Or open a terminal and type “ps aux | grep zerotier”.

On Oct 19, 2015, at 12:15 AM, Edouard Swiac notifications@github.com wrote:

Hello, I tried to install the zerotier mac app and it worked but the .app wont start (Authorization token invalid or ZeroTier One service not running.) and there is also no zt0 network interface. I used to be on Yosemite before and things went wrong after the upgrade.

Not sure where the issue is. I had the problem with the cli disappearing too (solved promptly thanks to a fix posted on that other issue) but I have not seen anything like my pb on the issues here.

Thanks.

— Reply to this email directly or view it on GitHub https://github.com/zerotier/ZeroTierOne/issues/241.

[edouardswiac]

It's not a fresh install, I updated my system.

Looks like the service is running: root 843 0.0 0.0 2466088 664 ?? Ss 12:07AM 0:00.23 zerotier-one

The cli returns the following:

eswiac@mbp ~ $ zerotier-cli info
401 info

[edouardswiac]

Is there a debug mode or something similar that I could do to debug it myself? The problem is zt0 not being setup, shouldn't it fail during install?

[edouardswiac]

Here is a list of all my interfaces. Maybe there is a conflict?

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
    options=3<RXCSUM,TXCSUM>
    inet6 ::1 prefixlen 128
    inet 127.0.0.1 netmask 0xff000000
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
    nd6 options=1<PERFORMNUD>
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    ether 20:c9:d0:85:8e:ff
    inet6 fe80::22c9:d0ff:fe85:8eff%en0 prefixlen 64 scopeid 0x4
    inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255
    nd6 options=1<PERFORMNUD>
    media: autoselect
    status: active
en1: flags=963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX> mtu 1500
    options=60<TSO4,TSO6>
    ether 32:00:19:4a:10:80
    media: autoselect <full-duplex>
    status: inactive
en2: flags=963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX> mtu 1500
    options=60<TSO4,TSO6>
    ether 32:00:19:4a:10:81
    media: autoselect <full-duplex>
    status: inactive
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
    ether 02:c9:d0:85:8e:ff
    media: autoselect
    status: inactive
awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484
    ether 82:1b:61:ff:6b:bc
    inet6 fe80::801b:61ff:feff:6bbc%awdl0 prefixlen 64 scopeid 0x8
    nd6 options=1<PERFORMNUD>
    media: autoselect
    status: active
tap0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1446
    ether 8a:d6:49:01:f4:aa
    inet 9.0.0.1 netmask 0xff000000 broadcast 9.255.255.255
    inet6 fe80::88d6:49ff:fe01:f4aa%tap0 prefixlen 64 scopeid 0x9
    inet6 2aa1::1 prefixlen 8
    nd6 options=1<PERFORMNUD>
    media: autoselect
    status: active
    open (pid 56)
bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=63<RXCSUM,TXCSUM,TSO4,TSO6>
    ether 22:c9:d0:58:17:00
    Configuration:
        id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
        maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
        root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
        ipfilter disabled flags 0x2
    member: en1 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 5 priority 0 path cost 0
    member: en2 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 6 priority 0 path cost 0
    nd6 options=1<PERFORMNUD>
    media: <unknown type>
    status: inactive

(I installed tap0 myself for something unrelated to zt)

[adamierymenko]

Ahh... I see a tap0. Are you running some other virtual networking thing like OpenVPN?

Do this:

cd "/Library/Application Support/ZeroTier/One"
sudo kextload -verbose tap.kext

What does it say? Then:

kextstat | grep zerotier

I see:

  139    0 0xffffff7f82da4000 0x7000     0x7000     com.zerotier.tap (1.0) EFD27D47-368B-3F9E-830F-434D2CF62ACE <7 5 4 1>

It could be that there is some sort of conflict between the two 'tap' drivers. The OS X driver is in need of a refresh to address some other issues, so if this is a problem I'd like to try to address it then.

[edouardswiac]

I installed openvpn after facing out the issue (zt0 not being created). tap.kext is loaded successfully (or already loaded), and I see the same output as you 130 0 0xffffff7f80a21000 0x7000 0x7000 com.zerotier.tap (1.0) EFD27D47-368B-3F9E-830F-434D2CF62ACE <7 5 4 1>

[edouardswiac]

I uninstalled tap (tap0 is gone). I reinstalled zerotier for mac, then immediatly ran kextstat | grep zerotier but it returned nothing. I then succesfully loaded the kext with kextload and grep'd it (141 0 0xffffff7f82ef2000 0x7000 0x7000 com.zerotier.tap (1.0) EFD27D47-368B-3F9E-830F-434D2CF62ACE <7 5 4 1>). I don't know if loading the kext is supposed to make to create zt0, but it's not there if my ifconfig. hope that helps!

[colesperks]

I am having the same issue, but I am not getting (142 0 0xffffff7f82a7c000 0x7000 0x7000 com.zerotier.tap (1.0) <7 5 4 1>) back out. Any suggestions?

[adamierymenko]

Loading the kext does not create zt0, but it does create files called /dev/zt# (0 to 31) which are then used to create these devices.

It does sound as if there is some sort of problem/conflict with another tap driver. We'll be doing some testing and trying to address in the next version.

[pjurg]

I have the same issue on a laptop I upgraded to El Capitan. kextstat gives the same output as above. However, there is no tap device visible when I use ifconfig. I might have installed a VPN client a while ago. I am not sure.

[adamierymenko]

Looks like some work on the driver is definitely needed. As soon as we wrap up our capacity testing on clustering that will be the next priority.

[adamierymenko]

I actually might make the kext optional and use utun (built-in tun) by default. The only limitation here would be that utun would not allow Ethernet bridging. So if you want that we could maybe have the kext as a downloadable and it would use it if it's installed. The advantage would be no kext, which would let us avoid any of these conflict issues and also possibly make a version for the Mac app store.

[adamierymenko]

To summarize for anyone watching: "works for me" on El Capitan but apparently some are having issues. Will investigate!

[pjurg]

It workd for me on Mac where I installed zerotier before upgrading to El Capitan and it does not work on another Mac where I first upgraded to El Capitan and then tried to install zerotier. Are you sure that El Capitan's System Integrity Protection does not prohibit enabling the interface? Ik would think not as it should stil be possible to create network interfaces, but it would explain the behaviour I encounter.

[adamierymenko]

For those who are having issues, here is a new build of the tap driver: https://www.zerotier.com/misc/PRERELEASE/tap.kext-new.tar.gz

To try it out first stop/unload the service:

sudo launchctl unload "/Library/LaunchDaemons/com.zerotier.one.plist"

Next make sure the kext is not still loaded (shouldn't be):

kextstat | grep zerotier

This should show nothing. If that's good, do this:

cd "/Library/Application Support/ZeroTier/One" sudo mv tap.kext tap.kext.orig sudo tar -xzf /path/to/downloaded/tap.kext-new.tar.gz sudo chown -R 0 tap.kext sudo chgrp -R 0 tap.kext

Now restart the service:

sudo launchctl load "/Library/LaunchDaemons/com.zerotier.one.plist"

How does this work vs. the original? This one was built with updates from the original tuntaposx package (on which it's based) and using El Capitan's version of Xcode.

[adamierymenko]

@glimberg says maybe the kext must be in /System/Library/Extensions on El Capitan -- if so we will update the installer and code for that

[adamierymenko]

Maybe if you have it in the old place and then upgrade to El Capitan it's okay... hmm... have to investigate.

[adamierymenko]

Apparently not, but who knows... let's see if that updated build fixes anything. :8ball:

[pjurg]

You can not write in /System/Library/Extensions in El Capitan. If it should be anywhere else, then in /Library/Extensions. I'll try the new kext later.

[pjurg]

I tried the new tap.kext, but it does not help. I also disabled SIP (https://support.apple.com/en-us/HT204012) and did a fresh install of zerotier, but the result is still that it does not work.

BTW: I noticed that after the fresh install tap.kext did not load automatically, so I loaded it from the command line.

[pjurg]

Sorry, wrong link, Should be: https://developer.apple.com/library/prerelease/mac/documentation/Security/Conceptual/System_Integrity_Protection_Guide/FileSystemProtections/FileSystemProtections.html#//apple_ref/doc/uid/TP40016462-CH2-SW1

[adamierymenko]

@pjurg I'm installing a fresh El Capitan in Parallels to try installing it there. Will look at this today since we are prepping for 1.1.0 release. Also thinking seriously about implementing utun to dispense with the kext if not too hard and making the kext a downloadable for people who want to try things like bridging or non-IP protocols.

[edouardswiac]

It actually works for me (without trying that new kext) but I have to sudo so that looks like a permission issues with the kext location in El Capitan?

eswiac@mbp agent (master) $ zerotier-cli info
401 info
eswiac@mbp agent (master) $ sudo zerotier-cli info
200 info a790e5f109 ONLINE 1.0.5

[adamierymenko]

Just did a fresh install of ZeroTier One on fresh El Capitan and it worked fine. The only issue was the missing zerotier-cli, which is known and will be fixed in an upcoming release. It needs to be symlinked to /usr/local/bin, not /usr/bin. The kext loaded fine and the network device is fine.

[adamierymenko]

Fix El Capitan zerotier-cli with:

sudo mkdir -p /usr/local/bin
sudo ln -sf "/Library/Application Support/ZeroTier/One/zerotier-one" /usr/local/bin/zerotier-cli

Should be fine after that.

[edouardswiac]

A friend of mine recently installed zerotier for the first time ever on El Capitan and it worked as well. I think the issue is when you install zt on an OSX version that predates El Capitan, and upgrade to El Capitan afterwards.

On Thu, Nov 12, 2015 at 3:18 PM, Adam Ierymenko notifications@github.com wrote:

Fix El Capitan zerotier-cli with:

sudo mkdir -p /usr/local/bin sudo ln -sf "/Library/Application Support/ZeroTier/One/zerotier-one" /usr/local/bin/zerotier-cli

Should be fine after that.

— Reply to this email directly or view it on GitHub https://github.com/zerotier/ZeroTierOne/issues/241#issuecomment-156268069 .

[adamierymenko]

I just did that on my own system and it worked fine. I can't reproduce any kext issues. Maybe I'm horribly confused about what problem is exactly being reported here.

[adamierymenko]

I'm wondering if the confusion is over "zt0 not created" -- no interfaces will be created until/unless you join a network.

I can't reproduce any kext issues, and I'm starting to think that's not actually an issue here. As I re-read the comments they revolve around the UI having issues after upgrade and the cli disappearing. The former I can't reproduce, and the latter is a known issue (links need to be in /usr/local/bin).

[adamierymenko]

Certificate verification is also fine on El Capitan:

kextutil -tn /Library/Application\ Support/ZeroTier/One/tap.kext
/Library/Application Support/ZeroTier/One/tap.kext appears to be loadable (including linkage for on-disk libraries).

[pjurg]

@adamierymenko Maybe we are looking in the wrong direction. After you said it worked fine for you, I tried to join a network using zerotier-cli and it seems to work fine now. The gui however does not (see screenshot). Previously I assumed that the non-working gui implied zerotier did not work, but I was wrong. It is just the gui that has a problem.

When installing ZeroTier I consistently see the following error in the console: CoreServicesUIAgent Error -60005 creating authorization

[adamierymenko]

Possibly relevant: https://developer.thalmic.com/forums/topic/585/

I think this is some kind of app signing and/or permission elevation issue related to El Capitan. Might not be manifesting on my system or test systems due to security settings.

Under security and privacy what is your settings here?

[pjurg]

I have it set to "Anywhere", but it seems to make no difference.

[heri16]

I am having the same problem on Yosemite when I am compiling my own zerotier from tarball (to fix the non-infinite UPnP issue). I think the problem is due to missing codesign certs.

[heri16]

This is what I did:

1. Create a Self-signed Code-signing Certificate (Free) via Mac OS X's KeyChain Certificate Assistance:
   • http://bd808.com/blog/2013/10/21/creating-a-self-signed-code-certificate-for-xcode/
   • https://developer.apple.com/library/mac/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html -> To use the Certificate Assistant to create a self-signed signing identity
   • If unsure, use "Self-signed Code" as certificate's CommonName.

2. Edit make-mac.mk at root folder of zerotier sources:

   • Edit the lines below accordingly to the name of your self-signed certificate.

   # Disable codesign since open source users will not have ZeroTier's certs
   CODESIGN=codesign
   PRODUCTSIGN=productsign
   CODESIGN_APP_CERT="Self-signed Code"
   CODESIGN_INSTALLER_CERT="Self-signed Installer"

3. Execute: make

4. Create a Self-signed Installer Certificate (Free) via OpenSSL:

   • http://security.stackexchange.com/questions/17909/how-to-create-an-apple-installer-package-signing-certificate
   • Create apple.conf with the following content:

   [ req ]
   distinguished_name = req_name
   prompt = no
   [ req_name ]
   CN = Self-signed Installer
   [ extensions ]
   basicConstraints=critical,CA:false
   keyUsage=critical,digitalSignature
   extendedKeyUsage=critical,1.2.840.113635.100.4.13
   1.2.840.113635.100.6.1.14=critical,DER:0500

   • Generate the key:

   openssl genrsa -out apple.key  2048

   • Create the self-signed certificate:

   openssl req -x509 -new -config apple.conf -nodes \
   -key apple.key -extensions extensions -sha256 -out apple.crt -days 365

   • Wrap the key and certificate into PKCS#12:

   openssl pkcs12 -export -inkey apple.key -in apple.crt -out apple.p12

   • Import it into keychain with open apple.p12. Select "Always trust".
5. make mac-dist-pkg

[heri16]

Alright, I can confirm that after latest security updates by Apple, Zerotier GUI does not work on Yosemite too. I used the official release downloaded from Zerotier.com

[rjocoleman]

I also have had the Authorization token invalid or ZeroTier One service not running issue on El Capitan. I had tried uninstalling and reinstalling several times without any change.

cli was outputting 401 JOIN when trying to Join networks.

How I resolved it (with a shotgun):

• I deleted my ZeroTier application support directory rm -rf ~/Library/Application\ Support/ZeroTier/ (CLI started outputting zerotier-cli: missing authentication token and authtoken.secret not found (or readable) in /Library/Application Support/ZeroTier/One)
• I reinstalled ZeroTier over the top of the existing install.
• Ran the GUI, granted admin privs to do whatever it does on install
• CLI and GUI now work as expected.

I didn't touch the new tap drivers. I had previously symlinked to fix CLI.

[heri16]

Alright, I verified that zerotier-cli (non-sudo) and ZeroTier.app UI Wrapper on Mac Yosemite cannot access authtoken.secret that leads to the same error message that has been screenshot by @pjurg and @edouardswiac. I accessed the port directly via Safari browser and everything works fine. Great, now we can narrow down the problem. Likely sandboxing and filesystem metadata on Mac os x.

[heri16]

@rjocoleman Your solution solved my problem. Seems like "~/Library/Application\ Support/ZeroTier/One/" is an undocumented folder which is different from ""/Library/Application\ Support/ZeroTier/One/". Documentation and uninstall.sh needs updating...

[pjurg]

@rjocoleman @heri16 Yes, this fixed it! I also appeared to have a directory "~/Library/Application\ Support/ZeroTier/One/" with two files in it: authtoken.secret ui.ini I deleted those files. Now the gui works and automagically the authtoken.secret from "/Library/Application\ Support/ZeroTier/One/" was copied to "~/Library/Application\ Support/ZeroTier/One/". This means that I can start the gui with the correct file. It also means that I had zerotier installed before. I thought I did not do that as I did a fresh install of Yosemite a while ago.

[adamierymenko]

So wait... you had it installed a while ago, uninstalled it, then tried again and it didn't work because it had a left-over old auth token in your home folder's ~/Library?

If that's the case it's an easy fix and the real bug is "app must check and rewrite local authtoken.secret if it is not valid."

[rjocoleman]

you had it installed a while ago, uninstalled it, then tried again and it didn't work because it had a left-over old auth token in your home folder's ~/Library?

If that's the case it's an easy fix and the real bug is "app must check and rewrite local authtoken.secret if it is not valid."

In my case yes. The error message was exactly correct. Uninstall didn't remove the old auth token. CLI gave me a descriptive but unhelpful 401. I initially conflated the GUIs ZeroTier One service not running portion of Authorization token invalid or ZeroTier One service not running to be the tap issue.

[pjurg]

It was the same in my case.

[adamierymenko]

Sounds like that's the real issue then. Will update to fix this. So there's no real El Capitan compatibility issue per se -- we certainly haven't been able to reproduce one.

[adamierymenko]

This should be fixed in edge, which is getting released very very soon (as soon as world is updated... right now it still talks to testnet). New version of Mac UI wrapper checks the file time on authtoken.secret and requests auth to update it again if the one on the system is newer than yours. That should solve this problem. Not an issue on Windows since Windows uses Windows app auth every time and just uses the one on the system (global).

Closing for now but will re-open if anything persists after 1.1.0 release.

[heri16]

Can't wait for the new release. Zerotier One is getting better day by day.

[kyleparisi]

I stumbled on this thread with a fresh install of El Capitan. I set https://github.com/zerotier/ZeroTierOne/issues/241#issuecomment-156584533 to Anywhere and the cli still complained. I then opened the App UI. This asked for my password and then it was able to function.

[raymond-liao]

I faced the same issue, what I meet and did:

1. In my network when I ping another node I get ping: sendto: Host is down, but I don't know how to fix it, so;
2. Uninstalled ZeroTier from my mac(M1) by sudo "/Library/Application Support/ZeroTier/One/uninstall.sh" and get result below:

   
   Stopping any running ZeroTier One service...
   Removing ZeroTier One files...
   Uninstall complete.

Your identity and secret authentication token have been preserved in: /Library/Application Support/ZeroTier/One

You can delete this folder and its contents if you do not intend to re-use them.

4. As the suggetion I think I need fully uninstall the ZeroTier, so I did delete `/Library/Application Support/ZeroTier`；
5. Re-install the latest version from [ZeroTier](https://www.zerotier.com/download/);
6. The problem happend: the GUI auto quit after opened few seconds, the command `zerotier-cli info` result `401 info {}`;
7.  I spend few hours to find and try some solutions from Google, someone says it should be the `authtoken` miss match between `/Library/Application Support/ZeroTier/One/authtoken.secret` with user home;
8. I re-uninstalled the ZeroTier and deleted the user authtoken, and then reinstall it get to work.

Hah, it's looks strange why the `authtoken.secret` did change.
Wish to help someone else!

[here is the ZeroTier authtoken-location](https://docs.zerotier.com/config/#authtoken-location)