x/mobile: build tool and configuration across android and iOS

[crawshaw]

Over the holiday break I got Go building .apk files: github.com/crawshaw/apk

This could theoretically be embedded directly in the go tool, so go build produces a .apk when GOOS=android. However there is another issue: it should be possible to build an app without an AndroidManfiest.xml, with a shared configuration across iOS.

This could be a tool, golang.org/x/mobile/cmd/gombile. My original design doc described such a thing, but I have been avoiding it until we got closer to a working iOS build. Now we are getting closer, it's worth thinking about.

Several of the gomobile properties are straightforward: it should look as much like the Go tool as possible, it should produce a .apk for android, and a .app for iOS. It should know how to use ios-deploy if you have it installed. It should build the platform-specific configuration files for you.

First question: what (if any) of the configuration parameters in AndroidManfiest.xml need to be exposed through the build system? https://go-review.googlesource.com/#/c/2162/ brings up the fullscreen attribute. What else?

Second question: what should the configuration look like? It should be easily determinable at build time, and involve the least amount of invention. One very minimal possibility:

package main

import ( _ "golang.org/x/mobile/config/fullscreen" )

[hyangah]

That's very nice!

So far, we've seen the cases that need proper permission settings (e.g. android.permission.INTERNET to access network, android.permission.WRITE_EXTERNALSTORAGE to read/write data, etc). -- Through the explicit import from main sounds fine. But I wish we could infer the minimum set of required permission during build time. For example, since it imports net package eventually, this needs permission to access network.

Screen orientation if fullscreen deserves a separate configuration import.

What about application/activity attributes such as icon/theme?

Some apps may require a specific set of screen sizes or hardware features (camera, motion sensors) so that only users with the right sets of devices can see the app in Play Store. But that's probably not a high priority.

http://developer.android.com/guide/topics/manifest/manifest-intro.html

[gordonklaus]

I don't think a configuration parameter for fullscreen mode is necessary. One should simply be able to call app.Fullscreen(true) (or whatever) in app.Callbacks.Start. I guess this similarly true for screen orientation?

I just a did a comparison of setting the fullscreen attribute in AndroidManifest.xml vs setting the fullscreen flag in app.Callbacks.Start and there is only the slightest difference in timing, it's hardly noticeable, I think it is negligible.

Regarding the proposed configuration syntax: it'll be a bit wordy if every parameter needs golang.org/x/mobile/config as a prefix. Also, it might not work so well for non-boolean parameters, should any arise. How about //go:mobile foo=bar comments instead?

[rakyll]

Some more:

• Intents and intent filter definitions may be required.
• Device orientation configuration and how the app is reacting to it. (We also need to emit events when device rotates.)
• Localization and i18n is also another issue which is solved by res/values- on Android. Do we need some sort of resource management that plays well both with Android and iOS?

[dskinner]

And more

• Version Code/Name
• Custom Permissions
• Services
• Providers
• Supported Screens
• Metadata Nodes

and so on. All of these things can have a place in a native game.

It'd likely be simpler to ask what you don't want to support. Having so many variations makes _ "imports" look clunky and doesn't account for variations or even different manifests for product flavors (demo with IAP/ads vs paid).

I have a similar tool I put together for producing shared lib: https://github.com/dskinner/mobilize and sympathize with the effort here, nice work on binary_xml.go. It would be nice if go build did the right thing when GOOS=android as I currently have to conflate code gen and building just to keep workflow simple.

But, this feels like a perilous path leading down a dead end trying to support too many features of AndroidManifest and I'd likely never use it (or find it useful) in any currently conceivable fashion beyond prototyping.

I know the discussion is pure go games written for Android/iOS, but realistically, games are likely to include features such as push notifications, in-app purchases, ads, etc. and much of this functionality on android is brought in with third party libs and configured in the manifest and/or resource files.

[bryanturley]

When I experimented with https://github.com/bryanturley/goapk I generated the info for the xml and other files from odd pragma comments and things already known.

//android: package com.example.basic
//android: target android-16
func main() {
        app.Run(app.Callbacks{
                Draw: renderFrame,
        })
}

It also would read them from init() funcs but only from the main package. If you did it with init() funcs you could have a "manifest_android.go" that would not interfere with non-android systems. That would be my preferred way of doing it, if it didn't get complex (which seems likely...).

I don't think you can generalize the android build system out, it is too wonky. Probably be easier to just have one for each non-sane (insane?) platform.

I also considered doing something like cgo

/* 
<insert AndroidManifest.xml here>
// or perhaps cgo style directives
#android Manifest ./path/to/Manifest.xml
*/
import "Apk"  // triggers a .apk build like import "C" triggers cgo build

I didn't get much further after becoming frustrated with the android build system once again... ;)

[slimsag]

Having the go tool recognize //android: key value lines in the main package sounds like a decent idea.

[bryanturley]

I had a third idea this morning. Find a way to trigger an apk build (easy part).

The apk build starts a tool or a subsection of the go tool that works similar to "go test". It scans every package that will be included for a func named and defined "func ApkManifest(m *apk.Manifest)" and calls them with a single apk.Manifest. These funcs could be precompiled in the existing .a files without any changes to the current tools. Each package adds it's requirements to this manifest object which in turn generates the xml file. No need to write a complicated "pragment parser" and all of the ApkManifest() would be dropped from the final binary by the linker since only the "go test" like program uses them. The apk package would just be a normal go package. From what I can see this is the most robust solution, requiring the least amount of work while maintaining simplicity.

example ApkManifest for stdlib net package

// inside a file only built on android.
func ApkManifest(m *apk.Manifest) {
   m.Target(16) // target "android-16"
   m.ICanHazNetworks() // all funcs required to have lolcat based names.
}

Might want to tell godoc to ignore ApkManifest with the proper definition, or not. A similar thing could be created for ios and possibly other systems that like to wrap programs up in bundles.

[bryanturley]

I just now realized that would treat entire packages like interfaces. Almost like

am, ok := (import "something").(ApkManifester)

I think my brain has become fully Go saturated... (that is not a language feature request)

[dskinner]

@bryanturley actually, for addressing just a handful of cases, I'm partial to your suggestion of //android: package com.example.basic as just a quick way to type out and get going. Embedding xml in a docstring and for the sake of a .go extension doesn't seem worthwhile.

Having an apk.Manifest sounds enticing initially but I have a hard time seeing how well that would pan out over time.

I'd guess things like android: package could simply be mobile: package barring scenarios where one launches an app on one store and later attempts another store to find the package name is taken. Things like android: target likely may not be relevant for a pure go app and would simply default to the minimum target, of which the current case is android-9.

Overall, I think I'd prefer to see mobile: fullscreen and mobile: package com.foo.app (or even an implied package name based on import path) over actual imports to enable various characteristics. Then again, if after reviewing all the possibilities, the only thing up for discussion is enabling fullscreen with no discussion for supporting other unstated intentions, then I guess it doesn't matter too much.

But, the mobile: ... comment style grows with fewer pains I think.

[slimsag]

Magic comments and importing packages are the only two options that can be analyzed easily without running the program. I think @bryanturley's latter suggestion is perhaps a bit overzealous

[bryanturley]

The zealot is back ;) I wrote a prototype for my third idea, had to know if actually worked. https://github.com/bryanturley/pkgface sample test program that reads itself in test/ It is not finished but the idea is sound and works with vanilla go1.4 example inputs: https://github.com/bryanturley/pkgface/tree/master/test (those two .go files) example output: https://github.com/bryanturley/pkgface/blob/master/test/output.txt It is a little slow at 0.3 seconds but I am sure it could be faster easily. It will need some tricks to do main packages, probably the same ones go test is doing. (I assume it renames the actual main in the temp directory it works in).

Only took 6ish hours to write the prototype (which isn't finished). You can do anything in these functions, you can't ask for more from a build system. https://github.com/bryanturley/pkgface/blob/master/fake0/gyros/gyros.go#L9

Remember in reality those fake packages would have code other than the target funcs in them.

[keithkml]

Just curious - what is the motivation behind this request? What's wrong with having an AndroidManifest.xml like every other Android app does?

[crawshaw]

For the sake of inventing as little as possible, my current plan for the gomobile tool is:

If an AndroidManifest.xml is present, use it. If not, create one, inferring what we can from the program statically.

We can build a common configuration system later if there is strong evidence to suggest it is better than having separate android/iOS configurations.

[bryanturley]

@keithkml Not having extraneous config files during build is part of the Go ideology. With some systems that is not possible but we can still emulate it. It does make sense to have an override for the configs since Go and Android/IOS/whatever will probably not release in sync and supporting every little corner case will most likely be difficult. Also I keep forgetting some people want to embed Go into their Java apps.

[bryanturley]

@crawshaw So in my head it sounds like you are talking about gomobile having a predefined import to permission/need map (I am easily wrong here). I would prefer the packages inform gomobile. Why infer when you can directly know?

If it where per package strings instead of funcs, that could be read without having to build another program. People seemed to not want go generate doing that despite go test already doing it.

// unicode "wrapped present" (bundle) rune in there const MobileIOS🎁 = xml/json/simpler data format here Maybe does not have to be exported...

I think the func version would be easiest to write and maintain though, no extra parsers need to be written just one bundle struct per target.

Just a suggestion.

[rakyll]

Not having extraneous config files during build is part of the Go ideology.

An Android app is not a binary, it's an archive of resources. An AndroidManifest.xml is not a Makefile. Go ideology you mention above is mostly in the scope of building a binary. From that point on, it's not that opinionated. (I don't think the apk/app builder should ever go in go build, but should be a separate tool on the other hand.)

I don't think at the current stage, it's any worthy to think about how we will represent this stuff in vanilla Go. Reusing AndroidManifest or generating one sounds fair to me.

[gordonklaus]

An Android app is not a binary, it's an archive of resources. An AndroidManifest.xml is not a Makefile. Go ideology you mention above is mostly in the scope of building a binary. From that point on, it's not that opinionated. I don't think apk/app builder should ever go in go build, but should be a separate tool.

I agree that go build shouldn't handle the complete packaging of apps, but I think your argumentation could use some refinement. A binary is an archive of (often mostly code) resources; and an Android app could just as well have been packaged as a binary. The difference is irrelevant. What is relevant is what type of resource is packaged by what tool. go build should focus on packaging Go code resources (as it currently does). gomobile should focus on packaging a complete app.

I don't think at the current stage, it's any worthy to think about how we will represent this stuff in vanilla Go. Reusing AndroidManifest or generating one sounds fair to me.

I can understand waiting for a clearer picture to emerge before trying to abstract away app configuration. But down the road it would be nice to have a common mechanism, if it is practical.

[bryanturley]

@rakyll the sentences following the one you quoted are in agreement with you. I should have written them more clearly. I do have some experience here. I wrote goapk, it works for the very basics. I would write more of it but @crawshaw's apk package and gomobile tool have already gone more in depth than I would have (a good thing). I wrote pkgface because I thought packages as interfaces was funny but wasn't 100% sure how to actually do it, or if it would be feasible in a tool like this (it is). I have always seen the packager and go build as separate, but more in the way go install and go build are separate. goapk also installed, ran and uninstalled the apk with the current adb device, not sure if gomobile should do that though. I apologize if anyone feels I was attacking their ideas.

[crawshaw]

Done. The gomobile tool exists.

Documentation: https://golang.org/x/mobile/cmd/gomobile Design doc: https://golang.org/s/gomobile