Mac OS X El Capitan: RoboPaint not connecting

[oskay]

It appears that RoboPaint is unable to access the serial port under MacOS X 10.11.

Tested on RoboPaint 0.9.5, 0.9.6.

Here are some links that may touch on related issues:

https://forum.pjrc.com/threads/29626-Mac-OS-X-El-Capitan-Serial-port-problems-need-more-info-to-raise-issue-with-Apple

https://github.com/voodootikigod/node-serialport/issues/552

Likely NOT related: Device drivers in El Capitan are now required to be signed. ( See http://www.machamradio.com/blog/2015/7/16-unsigned-device-drivers-in-os-x-v1011-el-capitan ) I do not think that this is related, because (1) there is no driver there and (2) we are able to get partial communication (at least) from python via pyserial:

https://github.com/evil-mad/wcb-ink/issues/42

Obviously, I'll be happy to run any tests that might be helpful. :)

[oskay]

Note from Apple: There is a "new usb stack" in 10.11.

https://developer.apple.com/library/prerelease/mac/releasenotes/Darwin/RN_USB/Articles/10_11Changes.html

[oskay]

Some progress made at evil-mad/wcb-ink#42

[oskay]

It looks like there is a fix now available at node-serialport.

[techninja]

Yup! But... it's not in a release yet > https://github.com/voodootikigod/node-serialport/compare/v2.0.0...master . I'd vote to not get rambunctious till they roll at least v2.0.1 or so.

Because the NG Paper.JS version is so much better than the current master codebase, I'm working my butt off to make my checklist here (#228), or at least an abbreviated version of it with just feature parity, but it's taking a bit of time. Also I'm working on trying to make ends meet with three other part time jobs. Good news is I'll be checking off at least two items by tonight.

[techninja]

They just rolled 2.0.1, and I've now built for all systems! (Only took 3.5 hours, 3 computers and one virtual system.)

This is ready for latest 0.34.0 Electron (and tested working). If we're crazy I could possibly release a real 0.9.5b with mavericks support and latest CNCServer, but there's only a few items left on the NG release (which we may want to start calling RP 2.0), that will open the door to powerful modularity and performance.

[techninja]

@oskay We have confirmation this is working on the 2.0.0-Beta.1?

[oskay]

Erm... not yet. Sorry-- been testing offline. I'll see if I can wrangle a robot here for a minute. :)

[oskay]

Working perfectly as of v2.0.0 beta 1. :)

[oskay]

One report of it not working correctly on El Cap: http://forum.evilmadscientist.com/discussion/comment/1702

(I have not been able to reproduce the issue.)

[jleverenz]

Hi! New to this project and WCB, but via some local hacking on the RB tip, I suspect one related issue is this change: https://github.com/techninja/cncserver/commit/fb12c65af8db7d339088a92f9b5031ba81f1e47a

Might be a cncserver bug. In my El Cap env, serialpost.list is returning an empty string for pnpId. The latest logic therefore checks pnpid (fails), but then never check manufacturer. So no port match, no connection.

I have some changes and tests I can submit, but wanted to get some feedback first. Not sure what was going on with the linux side to drive the logic change.

[oskay]

Hello and welcome! Yes, the root cause is in the realm of cncserver (which does handle all of the hardware interfacing). I'm not sure about that change. @techninja ?

[techninja]

A LOT changed recently in node-serialport in the move to 2.x. There were changes breaking all three systems in different versions, it was nuts. The Linux support went off the rails a bit and I couldn't pin anything down so I ended up changing this. Their unix base is shared between darwin and linux, so I likely should have tested this there.

I'll be testing on el-cap and verifying CNCServer is in good shape and not easily confused, may need to upgrade node-serialport yet again :P

[techninja]

FWIW, (and some very good news) I've just installed El Cap on a machine I just successfully tested before the upgrade, and it doesn't work (specifically, CNCServer doesn't see the EBB), so I should have a clear path to fix and verify! Good stuff.

[oskay]

Sweet! (I wonder if the EggBot/Wcb-ink software is borked there as well....)

[techninja]

Likely. Well, maybe. The methods used to detect which ports to use and getting their info seems very odd and roundabout in node-serialport. I think I see the port showing up as /dev/tty.usbmodem411... I'll be messing with my code a bit and seeing if I can get node-serialport to list it. They appear to have "fixed" some of these issues in 2.0.6, but have failed to publish this to NPM... Hrmmmm

[techninja]

serialport 2.0.5 on El cap listing the EBB as the following:

{
    comName: '/dev/cu.usbmodem411',
    manufacturer: 'SchmalzHaus',
    serialNumber: '',
    pnpId: '',
    locationId: '0x04100000',
    vendorId: '0x04d8',
    productId: '0xfd92'
}

With Ubuntu Listing it as the following:

{
    comName: '/dev/ttyACM0',
    manufacturer: 'SchmalzHaus',
    serialNumber: 'SchmalzHaus_EiBotBoard',
    pnpId: 'usb-SchmalzHaus_EiBotBoard-if00',
    vendorId: '0x04d8',
    productId: '0xfd92'
}

Maybe we should just match by Product ID and Vendor ID and call it done?

[oskay]

I would recommend matching the productId and manufacturer-- the VID/PID are generic microchip.

[jleverenz]

I should have clarified, but I get the same response for the listing as you've posted, @techninja. But because pnpid is an empty string, the cncserver logic never tests against manufacturer. If it did, it would correctly find the EBB. I tested this by simplifying the test to just the "All other OS detect" section.

[techninja]

Here's Windows:

{
    comName: 'COM3',
    manufacturer: 'SchmalzHaus LLC',
    serialNumber: '',
    pnpId: 'USB\\VID_04D8&PID_FD92\\6&10988163&0&8',
    locationId: '',
    vendorId: '',
    productId: ''
}

Kinda looks like the other data isn't being parsed correctly? Hrmm.

I'm going to vote we move to OS specific detection, with priority on Win/OSX. This will hopefully make it a lot easier to debug, and very accurate on the systems it allows for it. Will push something I've tested on four OS's soon.

[techninja]

Tested the commit above fully on all the OS' with flying colors, going to build this into the new build and hopefully make a customer quite happy... Less the fact that it's still weird and beta level software :P

[oskay]

You will at least make me quite happy. :)

[EmbeddedMan]

And me too. I can't believe that each OS reports such different information about the EBB. Maybe I messed up the descriptors somehow and some OSes cover for it better than others.

*Brian

On Sun, Jan 3, 2016 at 10:04 PM, Windell Oskay notifications@github.com wrote:

You will at least make me quite happy. :)

— Reply to this email directly or view it on GitHub https://github.com/evil-mad/robopaint/issues/235#issuecomment-168576235.

[techninja]

Don't be hard on yourself @EmbeddedMan, I'm fully betting this just some wackiness in Node-Serialport. Those guys need all the help they can get. They got super popular, changed a bunch of things up, have tons of people expecting everything to work across a bunch of different operating systems, and last I checked are still in need of a full time maintainer. We all do the best we can.

[jleverenz]

Thanks for the fix! Just tried it out on the wcb, and robopaint master now seems to be working from a clean checkout.

Slightly unrelated question: is the developer mailing list still a good place to discuss general project questions? Would like to understand more about the 0.9.x/1.0 vs 2.0 effort.

https://groups.google.com/forum/#!forum/wcb-developer

[techninja]

@jleverenz That group I'd say is great for general discussion, though all the meat and real movement happens here in the issue queue. I work full time on other things and come over as often as I can to make things happen, though as far as the 2.0 changes go, you can catch up on some of what's been happening by looking at the open/closed issues sorted by updates over the last 6 months or so.

If you're looking for a good discussion on how things are going to be setup, feel free and look at the paperJS branch.. a lot of new things have been documented.. though some haven't. If you're interested in helping our or build new things we'd LOVE the input and help. I'd totally be willing to setup a Google Hangout video chat and I can walk you through the project and talk about goals and all that jazz. We love our community and want to empower it as much as possible :)

[jleverenz]

Sounds good, thanks. I'll check out more of the project, and come here with questions. Thanks for working on this, my kids like watching it paint!

[tobiasboyd]

I am not having any luck with the RoboPaint 2.0.0-beta.1-2 on 10.11. When the application launches, it hangs at "Port found, connecting..." and I'm seeing these errors in the console:

1/7/16 9:18:03.000 PM kernel[0]: 016777.463375 PRT3@fd300000: AppleUSBHostPort::resetGated: retrying enumeration in 100 ms
1/7/16 9:18:12.745 PM com.apple.xpc.launchd[1]: (com.apple.quicklook[6746]) Endpoint has been activated through legacy launch(3) APIs. Please switch to XPC or bootstrap_check_in(): com.apple.quicklook
1/7/16 9:18:13.078 PM launchservicesd[99]: SecTaskLoadEntitlements failed error=22
1/7/16 9:18:13.081 PM launchservicesd[99]: SecTaskLoadEntitlements failed error=22
1/7/16 9:18:13.150 PM appleeventsd[67]: SecTaskLoadEntitlements failed error=22
1/7/16 9:18:13.238 PM launchservicesd[99]: SecTaskLoadEntitlements failed error=22
1/7/16 9:18:13.240 PM launchservicesd[99]: SecTaskLoadEntitlements failed error=22
1/7/16 9:18:15.291 PM lsd[313]: LaunchServices: Could not store lsd-identifiers file at /private/var/db/lsd/com.apple.lsdschemes.plist
1/7/16 9:18:15.298 PM lsd[313]: LaunchServices: Could not store lsd-identifiers file at /private/var/db/lsd/com.apple.lsdschemes.plist
1/7/16 9:18:18.129 PM launchservicesd[99]: SecTaskLoadEntitlements failed error=22
1/7/16 9:18:18.129 PM launchservicesd[99]: SecTaskLoadEntitlements failed error=22
1/7/16 9:19:00.037 PM lsd[313]: LaunchServices: Could not store lsd-identifiers file at /private/var/db/lsd/com.apple.lsdschemes.plist
1/7/16 9:19:00.124 PM lsd[313]: LaunchServices: Could not store lsd-identifiers file at /private/var/db/lsd/com.apple.lsdschemes.plist

[oskay]

@techninja These errors are with the January 3 version of the beta. Have you encountered any of these errors before?

It looks like the primary error there is a security/permissions issue (SecTaskLoadEntitlements). (Possibly related to code signing?)

[oskay]

On my computer (MacOS 10.11.2): RoboPaint appears to launch and (to the extent that I have tested) operate correctly. Looking at the console, I see every one of the warnings/errors listed above except for the first one, AppleUSBHostPort::resetGated: retrying enumeration in 100 ms.

[tobiasboyd]

Not sure if the hw matters, I am on 10.11.2 on a 2010 Mac Pro (which has both a builtin usb 2 controller on the mobo and a usb 3 PCIe card - I have been plugging the WCB into the usb 2 ports).

[techninja]

HMMMMmmmm. Very interesting. This is a great detail and I'll look into what others are saying about it, which should hopefully give us insight in how to fix this.

[techninja]

The problem does appear to have something to do with USB 3, and is not limited to just serial devices. A Google search for "AppleUSBHostPort::resetGated" brings up a number of interesting results that makes it clear that this is an OS level issue, a bit beyond EMSL hardware and this software. :confused:

I'll check a little more after work, but till then I encourage anyone watching the issue to work their Google-Fu as well.

[oskay]

@techninja Is there a way to log/view the USB port data on the computer where this issue is happening? It could plausibly provide some insight.

[oskay]

Good news from @tobiasboyd:

I tried switching to a usb 3 port out of curiosity, same error. But then I launched the Jan 3 2.0.0-beta of RoboPaint, and it works! It generated a bunch of familiar errors in the console, but it worked! https://gist.github.com/tobiasboyd/b85116ab6954e3d4cc29

Not sure if this means we're out of the woods, but it's certainly a great sign. :)

[oskay]

I think that I'd like to recommend that we publish (open) the 2.0.0 beta at this point. Anecdotal reports suggest that... we need more reports to really understand if this works for everyone. Objections?

[techninja]

I've got none! Will build from latest, and hopefully you can sign the app?

[oskay]

Sounds good; there are still more tests that we (especially, I) need to perform-- but so long as we have the beta label, this is a positive step forward. :)

[techninja]

Beautiful. I agree there's a lot of rough spots in this Beta release, but I've got them in my sights and have a plan. Here's hoping we can all be on the same boat with them and make it work. Latest Mac Release has been uploaded to the draft, will be pushing the other two momentarily.

[RI0]

Has a general release been made to correct this? I need to point a teacher to a URL she can download and install. She would like to use the Watercolor Bot in class and it no longer connects to her machine.
She has OS X El Capitan 10.11.3 on a MacBook Air, no development tools.

[oskay]

@RI0 The release available for download at: https://github.com/evil-mad/robopaint/releases/tag/v2.0.0-Beta.1-2016.01.13 is fully functional and does not require development tools. It has the "beta" label and a few dialog boxes here and there that could use better wording, but it is certainly "good enough" to use.

(Not sure if it's what you want to use, but the Inkscape driver is also updated for Mac, available here: https://github.com/evil-mad/wcb-ink/releases/tag/v1.3.2)

[techninja]

I believe this is managed with the latest soon to be 2 releases. Closing this out, can be re-opened if there's issues with the new releases.