Issues with dependency management in projects

[richiksc]

How FTC handles SDK publishing is very odd for Java/Android convention. They give you a full Android project with a subdirectory for your code. This causes problems for updating the SDK when a new version is released. You can use git to pull from ftc_app/master but there are many merge conflicts and it is extremely frustrating to resolve them.

By convention, FTC should publish a .jar/.aar file to a Maven repository that we can include in our projects. If we want to make another layer of abstraction as another library (like we are doing) we can simply include that as a dependency.

Thanks, Richik MechaTech FTC 10617

[magneticflux-]

@richiksc I've devoted a lot of thought to this, and the main problem encountered is how to deploy code to the robot. If the app were a dependency only, nothing would run when app is deployed to the phone. There is currently no way to include a fully-featured app as a dependency in Android, and no real point to it. The alternative (and probably better) solution is to dynamically load a jar file delivered to the app, but that runs in to even more problems: namely, apps on the Google Play store can not load arbitrary code and execute it. Ultimately, there are many problems with the immediate alternatives. I personally favor #337 as a step forward. Additionally, tangential concerns to yours were addressed in #165.

[cmacfarl]

Jeremy's diff can be merged into a personal fork of ftc_app and management of your fork should not be too big of an issue moving forward.

A somewhat simpler solution for integrating software stored external to ftc_app is to add a couple lines to TeamCode/build.gradle to point it at your external software. e.g.

apply plugin: 'com.android.application'

// Custom definitions may go here
android {
    sourceSets {
        main.java.srcDirs += '<path to your local source>'
    }
}

The path may be relative. So if you keep your local source in a directory beside ftc_app you can do

main.java.srcDirs += '../../team-xyz-source/<some further path maybe>'

If you do this in a fork and you just fetch/merge from upstream whenever FIRST releases a new SDK it's relatively painless.

[cmacfarl]

While the Maven idea is interesting a rather large percentage of the SDK customer base downloads the software as a zip file. Anything that adds a dependency to an external source would have to be looked at in the context of all of the ways that teams acquire and manage the SDK software.

[richiksc]

@cmacfarl we would not remove zip download, we would just add the option

[cmacfarl]

But how would that work in practice?

You've added a maven declaration to the gradle repositories block. Android Studio now needs to connect to maven the first time it builds after downloading the zip file. While this may be fine for most users, it may not be fine for the user that downloads the zip file, puts it on a thumb drive, walks into a classroom with restrictive, or no, internet access, and tries to build the software.

Remember, the people who read these emails and are following ftctechnh/ftc_app are typically the power users. And yet FIRST wants very much to reach those who don't have the resources and/or experience of its most advanced teams. Sometimes things seem odd until you put that into perspective.

On Tue, Sep 12, 2017 at 1:20 PM, Richik SC notifications@github.com wrote:

@cmacfarl https://github.com/cmacfarl we would not remove zip download, we would just add the option

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ftctechnh/ftc_app/issues/385#issuecomment-328972885, or mute the thread https://github.com/notifications/unsubscribe-auth/ABXn9-qRBc2jrmd3w_EyJJVoARd9KTcmks5shuejgaJpZM4PVJaj .

[richiksc]

The the git repository would not use maven. The SDK itself would be published to maven that teams could use by adding it to build.gradle. But there are limitations. A full-featured Android app can not be a jar dependency.

On Tue, Sep 12, 2017, 3:35 PM Craig MacFarlane notifications@github.com wrote:

But how would that work in practice?

You've added a maven declaration to the gradle repositories block. Android Studio now needs to connect to maven the first time it builds after downloading the zip file. While this may be fine for most users, it may not be fine for the user that downloads the zip file, puts it on a thumb drive, walks into a classroom with restrictive, or no, internet access, and tries to build the software.

Remember, the people who read these emails and are following ftctechnh/ftc_app are typically the power users. And yet FIRST wants very much to reach those who don't have the resources and/or experience of its most advanced teams. Sometimes things seem odd until you put that into perspective.

On Tue, Sep 12, 2017 at 1:20 PM, Richik SC notifications@github.com wrote:

@cmacfarl https://github.com/cmacfarl we would not remove zip download, we would just add the option

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/ftctechnh/ftc_app/issues/385#issuecomment-328972885 , or mute the thread < https://github.com/notifications/unsubscribe-auth/ABXn9-qRBc2jrmd3w_EyJJVoARd9KTcmks5shuejgaJpZM4PVJaj

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ftctechnh/ftc_app/issues/385#issuecomment-328976408, or mute the thread https://github.com/notifications/unsubscribe-auth/AIhooBgJTVX3ebK-WH-yWcmjeEFK23gAks5shur1gaJpZM4PVJaj .

[magneticflux-]

@richiksc I believe that there are ways to set up a Maven repository using GitHub Pages. If you're interested and will use it, I could host each version of the dependencies after fixing the version numbers to be semantic. Each release, I could reformat the version and dependency information and since it's hosted with GitHub, I wouldn't have to run my own server and teams wouldn't have to depend on me. Here's a post describing how to do it: link. I've been toying with the idea for a while, but it doesn't seem to complicated and I'm open to collaboration on it.

[richiksc]

@magneticflux- you just said there was not a way to do it because it is a full featured Android app

[magneticflux-]

@richiksc Ah, sorry, I should have clarified. The dependencies (RobotCore, Vuforia, etc.) that are in the libs/ directory may be easily exported to a Maven and replaced with versioned coordinates. The files inside the FtcRobotController submodule can not be easily extracted because they are tied to layouts, manifests, and resources in the app.

[cmacfarl]

Note that pretty much everything that is being talked about here can be fully prototyped in a local fork of ftc_app without any special permissions. e.g. The gradle is all there, the libraries are all there, and I think what you are trying to do is divorce things from TeamCode. So, go do it. Publicize what you are doing in the official forums. Get people pulling from your fork because what you've done is so compelling that they can't live without it. And then go make the case to FIRST that this is a better solution because of x, y, and z, and oh, I have 500 teams using this. Seems like a reasonable approach to things.

In the meantime sourceSets statements, and an occasion fetch/merge has worked pretty painlessly for my teams. :-)

p.s. Note that nobody should be working or modifying software directly in the /FtcRobotController directory. The design model, if you want to modify an example, is to copy and rename it into TeamCode, ergo, no merge conflicts. If you are modifying system files (layouts, resources, etc) well then you really shouldn't be doing that for any reason other than fun and experimentation, but you should also not expect clean, seamless, merges as you've, so to speak, voided the warranty.

[richiksc]

Does the FtcRobotController directory contain any classes essential to programming OpModes? Or is it just the app code and samples? If the latter, we can try it out and prototype a way to load external files from the app

On Tue, Sep 12, 2017, 5:54 PM Mitchell Skaggs notifications@github.com wrote:

@richiksc https://github.com/richiksc Ah, sorry, I should have clarified. The dependencies (RobotCore, Vuforia, etc.) that are in the libs/ directory may be easily exported to a Maven and replaced with versioned coordinates. The files inside the FtcRobotController submodule can not be easily extracted because they are tied to layouts, manifests, and resources in the app.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ftctechnh/ftc_app/issues/385#issuecomment-329006819, or mute the thread https://github.com/notifications/unsubscribe-auth/AIhooJ0ECuPjMO2uF9CI7zSOpgC41Nddks5shwuQgaJpZM4PVJaj .

[magneticflux-]

@richiksc It contains only the classes for loading OpModes, you shouldn't use anything in there under normal circumstances. The main hurdle with loading external "plugins" at runtime is the deployment method. ADB works over USB and wifi, and it works well enough. Plugin deployment would require writing a robust upload/download Gradle script that works for Android phones, and I'm not even sure that's feasible.

[cmacfarl]

The /FtcRobotController directory contains the activity that forms the foundation of the app. You can not build and deploy the RC without it.

And right, you can not build an OpMode in Android Studio independently of FtcRobotControllerActivity, deploy it to the phone, and then treat it like a plugin that gets sucked into the RC app. OnBot Java comes pretty close to that, in that there's now a compiler built into the RC and you can build OpModes on the phone itself, but it's a rather large leap to go from that to AS building and deploying independent OpModes that are treated as plugins to an APK that is already sitting on the phone.

But this isn't really what the subject of the original issue was. The complaint was quote, "there are many merge conflicts and it is extremely frustrating to resolve them." I think a few pointers and ideas have been expressed on how to solve this particular problem. Is it fair to close this issue and move further discussion of software management to the forums?