Make the upgrade path easier

virtuald commented 5 years ago

As we get more and more packages, knowing to upgrade them all is more problematic. Need a simple way that ideally 'just works' and doesn't cause users to miss updates.

First thought was to inspect the local ipk/pip cache and download new versions of those, then on install do the same.

One thought is that pyfrc should probably check all robotpy installed versions when deploying to the robot? Make sure local matches remote, etc.

Or, what about creating a dependency file of some sort, and operating on that? Then users can check it in, and we could sync it both locally and on the remote.

auscompgeek commented 5 years ago

A dependency file sounds like a good idea. Heck, download-pip already supports passing -r requirements.txt. (Does that work for install-pip?)

virtuald commented 5 years ago

We could deprecate using the installer directly. If we add an extension point to the install, maybe the workflow is something like

User has to put a robotpy.json file next to their robot.py (or maybe as code in robot.py?)
When running locally, local pip repo is checked against this list
On deploy, remote dependencies are checked against this list (not at runtime!)
To install on the robot, you do py robot.py dep-download and py robot.py dep-install
Hm, local dependencies sync wouldn't be able to use robot.py because it wouldn't be able to execute due to missing dependency

MrRSquared commented 5 years ago

I like the .json idea, because it preserves the customization and minimal install options. It also allows the use of new experimental libraries. However, I do wonder about the possibility of prolonging, or breaking deploys at events. Is it possible to add an install/upgrade option where users select packages on install or upgrade of robotpy on the rio?

Alternatively, I wonder about an installer similar to the frc vscode installer. Users could select libraries on install through a lightweight GUI with the ability to install/upgrade using pre-downloaded packages.

auscompgeek commented 5 years ago

I'm not too sold on the JSON choice, honestly. Not very user-friendly; can't put comments.

MrRSquared commented 5 years ago

I was just working in vscode on a new computer, and realized that the FRC java/ c++ languages use a json for their external libraries. I wonder if using their install/update mechanism is an easy way to accomplish the task.

auscompgeek commented 5 years ago

A system like GradleRIO's vendor JSON is probably rather heavyweight for what we currently have, although it would make it easier for vendors if they ever wanted to create their own packages for RobotPy.

virtuald commented 5 years ago

@MrRSquared publishing the dependencies isn't the problem we're trying to solve here -- we have a place where the dependencies live. The problem is that it's difficult to install them. The way GradleRIO deals with this is downloads them to your local computer at build time and then copies them during the deploy. That's slightly more difficult for us, because there's no 'build time', nor really a 'build file'.

MrRSquared commented 5 years ago

That makes sense. Forgive my ignorance, I did not actually know the third party wpilib libraries used gradleRio. I just knew the acquisition and updating of them is easy and standardized (however, now that I really think about it, they still need to be updated individually). I understand that with this library there is not really a build time. I definitely do not want to add work for you all, it is so helpful that you are all actively developing these libraries.

virtuald commented 5 years ago

@auscompgeek what would you suggest instead of JSON then? ini files?

auscompgeek commented 5 years ago

INI is pretty poorly-defined (different versions of Python have incompatible configparser formats). TOML is pretty good. YAML isn't too terrible either, but I don't think we would ever need the complexity YAML provides.

virtuald commented 5 years ago

Related bug that occurred because a team's versions weren't quite right: https://github.com/robotpy/robotpy-wpilib-utilities/issues/142

JDuskey commented 5 years ago

Hey, I'd love to help out with this if I can. From a user's standpoint it'd be great if a new robotpy release version was created everytime a package is updated. Same with installing to the programming computer itself. Could new versions of pyfrc be created with requirements.txt asking for the most up to date version of the packages? That way the --upgrade would also resolve the dependencies as well. Let me know if there is anything I can do to help you guys out at all, you've all been a huge help.

virtuald commented 5 years ago

@JDuskey would love some help, the amount of time I've had this offseason hasn't been very much because of work commitments.

I guess the problem with 'robotpy release version' is it's not particularly meaningful, as there are a lot of robotpy packages at this point.

What I really want is to force the user to declare what their dependencies are, and on sim/deploy it checks (a) the local environment and (b) the robot environment to make sure that they match. If you have an idea on how to tackle that problem, that would be quite useful. It doesn't seem particularly difficult, just needs to be done.

... thinking about it, I guess we could just repurpose requirements.txt for this? Maybe require it to be a specific name like 'robotpy-requirements.txt'? Would need to figure out the best way to specify cscore -- as it typically can't be installed locally, only on the roborio.

A secondary thing that could come out of this is the mechanism could be used check for updates via pip, which would solve your 'new robotpy release version' problem?

As an aside, I'd like to stop distributing robotpy 'builds' and have users use the installer instead. It's more flexible and less likely to get out of date.

robotpy / robotpy-installer

Make the upgrade path easier #27