Deployment directions for RobotPy 2015

[virtuald]

This is mostly a placeholder to collect thoughts on how it should be done. I don't have a RoboRIO yet so it's hard to know what will work best here.

My current thought is that it would be nice if pyfrc (or similar) was the primary supported mechanism, and it uploaded your code/etc. Presuming we have SSH access to the bot or similar, it could make sure the robot has the latest version of wpilib/hal/etc installed also?

Alternatively, we could develop an eclipse plugin -- though I'd rather have a deployment mechanism that I could just write a script to run.

[PeterJohnson]

Yes, there is SSH access to the bot, and that's the method used by the eclipse plugins to upload Java and C++ programs. Currently, the default login is "admin" with no password. The default configuration uses mDNS with the name "roborio-team#.local", e.g. "roborio-294.local".

After upload, the eclipse plugins for Java and C++ execute the new user code by kicking off a shell script which kills and restarts the user program and then does a tail -f on the UserProgram.log file (which captures stdout/stderr output). We could do that, or alternatively do the instant-reload thing ala django or other web frameworks that detects changed files and auto-restarts. The current Java plugin does NOT check for Java being installed (the user program simply fails to start if you've not previously deployed Java).

Packaging wise, the robot uses opkg (which uses .ipk files). Ideally our install script will simply upload 2 or 3 .ipk files and then run "opkg install" to install them. I already have an .ipk file built for python3, although I need to update it for the latest libraries (they switched from readline5 to readline6 recently).

[virtuald]

I think that our install script should upload the python interpreter as an .ipk. However, I think that managing the python packages on the robot should be done via pip. If we have some general way to upload a tarball and run 'pip install' on it, that would be an easy avenue for users to get stuff onto their robots easily enough.

[virtuald]

I wonder if we should run the robot code inside of something like screen? One thing that was nice (and also not nice) in previous years was the ability for multiple users to connect to NetConsole and be able to view the output.

Another alternative is to write all output to a file and let people tail it... but I worry about excessive logging by inexperienced programmers either filling the disk or wearing out the flash.

[PeterJohnson]

Well, the current Java and C++ default approach does the "write all output to a file and tail it" approach, but I concur this is not the best approach. I don't think it'd be that hard to create a multicasting server - e.g. provides a TCP port (multiple) computers can attach to... or it could be done with a doorbell-style UDP port. Screen isn't all that great for hooking into IDE console windows because it's not line-based (although it does open up cool ideas like using ANSI commands for color and screen position, but if you're getting that fancy, why not just use SmartDashboard?)

[virtuald]

Ha, we could just implement something that sends on the same port NetConsole used to write to.

[PeterJohnson]

Well, we can't broadcast (as everything is dynamic IPs now), but we could implement a doorbell-style uni/multicast with the same ports.

[virtuald]

I don't see why broadcast wouldn't work, just detect your current network settings and you're good to go.

[virtuald]

It's worth noting that NetConsole has been introduced in Java/C++ somewhere... but I'm not quite sure how it works.

[PeterJohnson]

It's pretty much exactly what you suggested above. There is a netconsole-host binary on the RoboRIO that redirects stdout/stderr of its fork'ed child to broadcast on port 6666. This is run by the /home/lvuser/robotCommand shell script, which for Java reads "netconsole-host java -jar FRCUserProgram.jar" (with appropriate absolute paths). We should do something very similar to this when deploying Python programs (e.g. deploy an appropriate robotCommand script along with the user's code).

The new "riolog" plugin in Eclipse simply listens on port 6666.

[virtuald]

I didn't see any checkins for that, they must be maintaining it elsewhere. @bradamiller it would be pretty sweet if it held the most recent parts of the netconsole log in memory and there was an ability to retrieve that.

[PeterJohnson]

Are you talking about the robot side (netconsole-host) or the Eclipse side? The robot side appears to be part of the NI suite of software for the robot, which is not open source. Note that you can send info directly to the driver station as well (e.g. exception backtraces are sent this way), and I believe those may be separately recorded--I should look and see if the new DS does that. Note that storing it on the robot side is only a good thing if you do it very occasionally (like when the robot is disabled), otherwise flash wear can become an issue. And you need to write it to disk because the robots are turned off after the end of the match. So mostly you're better off sending it to the DS and/or Dashboard and saving it there.

[virtuald]

Ah, I thought they created something new to do that.

[computer-whisperer]

Something I had been thinking of recently was a sort of decentralized, networked logging system. You would have the system run on as many devices as you want, and they would all share logs over the network, and store them. In light of Peters comment you could set it up so that it only stored a subset of the logs at given locations, or none at all.

I had originally planned to build this during build season for my team, but would it be useful here? Another perk is that you could connect your dev computer to either the bot or the DS after a match, and automatically sync all logs to the dev computer.

[virtuald]

That's a bit separate from what this github issue was originally about -- though it is a useful capability. :)

However, that sort of thing is done all over the place (sometimes well, sometimes not). There's even a DARPA program that was concentrating on doing that sort of thing. Typically one uses syslog to do that sort of stuff. I would recommend investigating what other solutions exist out there before creating your own solution.

I suppose most of the existing solutions are centralized, and not decentralized.

Oh, and @dangyogi's ERL does this sort of thing too. We were discussing making it a separable component so it could be used with RobotPy and ERL. Though, I think his solution uses a VM for the database server, which is a bit annoying.

[virtuald]

@PeterJohnson it's not NI's binary, they created their own version of Netconsole: https://github.com/frc1418/wpilibtools

[virtuald]

I'm starting work on parts of this. Shouldn't take too long.

[virtuald]

The implementation is functional and the documentation is done as of aa95f97