proposal: runtime: drop support for linux/armv5E and linux/armv6

[minux]

Since the introduction of Linux/ARM support, the minimum required hardware to run Go with Linux was ARMv5E, e.g.ARM926.

At that time, this choice makes sense because ARM9 systems are still around, and we even had an ARMv5E builder for some time.

However, ARMv5E lacks some important atomic instruction support. LDREX/STREX is introduced in ARMv6 (ARM11), and LDREXB/STREXB, LDREXH/STREXH, LDREXD/STREXD are introduced in ARMv6K. Without these instructions, we resort to emulating the required 64-bit atomic operation using 57 spinlocks selected by address (which uses mod and is a slow operation on ARM) [see https://tip.golang.org/src/runtime/internal/atomic/atomic_arm.go]

I don't know how many projects out there that still uses ARMv5E and Go, but I don't expect many. I propose to increase the minimum architectural requirement to ARMv6 (and possibly ARMv6K), so that at least we can use LDREX/STREX, which should help a lot with the new atomic-heavy runtime.

To summarize the benefits,

1. if we raise requirement to ARMv6, then we can intrinsicify 32-bit atomics.
2. if we raise requirement to ARMv6K, then in addition to 1, we no long can remove the emulated 64-bit atomics from runtime by using the native LDREXD/STREXD instructions, but also intrinsicify them in the compiler like other ports. We can also use YIELD instruction to address #16663.

If you still use Go on ARMv5E systems, please help by listing the processor model used. Thanks.

PS: other supported OSes do require ARMv6K as only Linux provides required 64-bit cas kernel helper for sync/atomic. Another way to go is to modify the runtime/internal/atomic package to:

1. use LDREXD/STREXD on non-linux,
2. use kernel cas64 on linux. But this solution will complicate maintaining the code and as we don't have a real ARMv5E builder, I think the support is certainly going to bitrot.

PS2: The core used by Raspberry Pi 1, ARM1176JZF-S, is ARMv6K, so this proposal won't affect Raspberry Pi 1, the most popular ARMv6 systems.

[minux]

/cc @cherrymui @davecheney @josharian

[josharian]

@hasty

[davecheney]

I'm fine with this.

On Tue, Sep 13, 2016 at 10:31 AM, Josh Bleecher Snyder < notifications@github.com> wrote:

@hasty https://github.com/hasty

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/17082#issuecomment-246537987, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAcA_pdNkUCP-Uhku-pBXNnA4Z7NqSgks5qpe7ogaJpZM4J7K4x .

[josharian]

SGTM

[bradfitz]

SGTM, especially if it means I can kill the not-really-ARM5 linux-arm-arm5 builders.

[minux]

To be precise, do you support only removing armv5 support or also requiring armv6k (RPI 1 won't be affected)?

[minux]

Killing the linux-arm5 builder also requires us to drop GOARM=5 support, which is also used (overloaded) to mean FPU-less ARM CPUs (which does not necessarily mean ARMv5. Yes, this is very confusing.)

[bradfitz]

I know close to zero about the dozens of flavors of ARM but if all Raspberry Pis still work I find it hard to object to requiring ARMv6K if that makes things easier.

[hasty]

Thanks, @josharian No objections here; we've moved on to ARMv7A.

[davecheney]

I understood your request to be "remove GOARM=5", and I agree with that.

On Tue, Sep 13, 2016 at 12:22 PM, Minux Ma notifications@github.com wrote:

Killing the linux-arm5 builder also requires us to drop GOARM=5 support, which is also used (overloaded) to mean FPU-less ARM CPUs (which does not necessarily mean ARMv5. Yes, this is very confusing.)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/17082#issuecomment-246554244, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAcA9xGsaStiiuyeGF20NGifk9ZzWm6ks5qpgjKgaJpZM4J7K4x .

[peteArnt]

The Atmel SAM9G25 SOC looks like it may be a victim of this feature request. It implements the ARM architecture version 5TEJ instruction set. Products like the Arietta G25 from ACME systems (which I use) will become incompatible with future versions of the GO ARM runtime if what I'm reading here is correct.

While I generally agree with what is proposed here for GO 1.8, is there no way for a developer to re-introduce a V5-compatible runtime into his/her own GO development environment?

If the architecture requirement gets bumped to ARMv6 or 6K, how long will that be good for?

-Pete

[minux]

The main goal for this proposal is to bump the minimum ARM requirement to ARMv6K so that we can use native 8-bit/32-bit/64-bit LDREX/STREX instructions and inline them in the program. (As explained in the proposal, we currently use emulated 64-bit atomic operations in the runtime.)

Once we drop the support for v5E and v6 (without K), it will be pretty hard to regain the support because it's not a master of re-introduce necessary runtime functions.

One difficulty for maintaining ARMv5E and ARMv6 support is that, it's really hard to find systems with enough memory to host a Go builder (we need at least 512MB, preferably 1GB)

Dropping support for ARMv5E mainly affect various ARM926 systems, and dropping support for ARMv6 mainly affects various ARM1136 and ARM1156 based SoCs (one notable example iMX3x series, used in earlier Kindle eReaders.)

There are only two ARM11-series processors that implements ARMv6K: ARM1176 and ARM11MPCore. The other two ARM1136 and ARM1156 doesn't implement the required 8/64-bit LDREX/STREX instruction.

We could consider just dropping ARMv5E support, and then see if we can reduce the use of 64-bit atomic operations in the runtime. @aclements, are 64-bit atomic operations critical for runtime performance on 32-bit systems? If not, I think we can just drop ARMv5E support for Go 1.8 in order to continue support ARM11 processors.

[aclements]

@aclements, are 64-bit atomic operations critical for runtime performance on 32-bit systems?

It's hard to guess, but I don't think so. I took a quick look through all of the 64-bit atomic operations and we're pretty careful to amortize the cost of almost all of them. It might worth turning gcController.dedicatedMarkWorkersNeeded and gcController.fractionalMarkWorkersNeeded into int32s, since those can get hit on every scheduling decision during GC, but those are also perfectly fine as int32s (I think I just made them int64s because we don't have atomic ops on int32).

[adg]

We should probably do some more research to find people who might be using these chips. @minux, do you want to mail golang-nuts? Failing any pushback, this sounds like it's a go.

[rsc]

Minux, did you email golang-nuts?

[rsc]

I see no email to golang-nuts. I am happy to send one. I may ask about OS X 10.8 at the same time.

@minux, is there an easy command to run on Linux systems to find out whether they'd be affected?

Thanks.

[bradfitz]

Ping @minux. Do you want this to happen for Go 1.8? If so, time is running out.

[minux]

I think we can revisit this in the next cycle because the tree is frozen so there won't be any change that would benefit from this.

The test, at least on glibc linux/arm system, is this: LD_SHOW_AUXV=1 ls | grep AT_PLATFORM (the ls command can be replaced with any dynamic linked program)

And if the output says: AT_PLATFORM: v6l or v7l, then the system won't be affected. If its v5l, then it will be affected.

[bradfitz]

I think deleting code & updating docs is okay for the freeze if you want to proceed for Go 1.8.

[minux]

The real benefit of removing armv5e support is intrinsicify 32-bit atomics.

removing the support itself won't delete much code though. (we still need the softfloat implementation for mips.)

[minux]

I just verified that rpi1 supports the yield instruction in the ARM mode, therefore, if we raise the minimum arm architecture requirement to armv6k, then we can address #16663 as well.

[bradfitz]

Fixing #16663 is probably Go 1.9 material.

But I'd be happy with Go 1.8 breaking armv5 enough what we could remove the fake builders, add some docs to the Go 1.8 release notes, and simplify our lives going forward in Go 1.9.

[minux]

OK, how about this: In Go 1.8 release note, we will announce the intention to drop (at least) ARMv5E support in the next release.

I did a survey of how other project deprecates support for systems, it seems some notable projects (e.g. gcc) do announcements in the immediate preceding release first.

[davecheney]

SGTM

On Wed, 9 Nov 2016, 17:50 Minux Ma notifications@github.com wrote:

OK, how about this: In Go 1.8 release note, we will announce the intention to drop (at least) ARMv5E support in the next release.

I did a survey of how other project deprecates support for systems, it seems some notable projects (e.g. gcc) do announcements in the immediate preceding release first.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/17082#issuecomment-259345313, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAcA_ijMYvn_elwtvxYEe0SGC5k9DOkks5q8W0-gaJpZM4J7K4x .

[rsc]

How about we just drop it for Go 1.8?

[rsc]

Specifically, how about we make Go 1.8 require ARMv6K (what Raspberry Pi 1 has)?

[davecheney]

LGTM.

On Tue, 15 Nov 2016, 08:16 Russ Cox notifications@github.com wrote:

Specifically, how about we make Go 1.8 require ARMv6K (what Raspberry Pi 1 has)?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/17082#issuecomment-260465108, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAcA3Ts-z6Rv-7C9WMgE_SylBbpdhl-ks5q-M-pgaJpZM4J7K4x .

[cherrymui]

Is this comment still true? (https://go.googlesource.com/go/+/4ca3a8f7a807bba55e9db82b9aa8c43b1a186b8e/src/cmd/dist/util.go#530)

    if goos == "freebsd" || goos == "openbsd" {
        // FreeBSD has broken VFP support.
        // OpenBSD currently only supports softfloat.
        return "5"
    }

[rsc]

@cherrymui, I believe that is a different "ARM 5". We're not talking about dropping software floats. See Minux's comments above, in particular:

Killing the linux-arm5 builder also requires us to drop GOARM=5 support, which is also used (overloaded) to mean FPU-less ARM CPUs (which does not necessarily mean ARMv5. Yes, this is very confusing.)

The messaging here needs to be very clear.

[cherrymui]

Thanks. Then, are we going to do anything in the runtime or build tools to stop GOARM=5 program from building/running?

[rsc]

My understanding is that this bug is about retiring some very old arm hardware but not to the point of requiring VFP in hardware and so not about retiring GOARM=5.

[minux]

I still suggest we announce our plan to drop support for armv5 and armv6 for 1.9 and continue to support them for 1.8 to give existing users some time to upgrade.

The other reason is that there won't be much benefit of dropping support during the code freeze.

[minux]

Regarding FreeBSD VFP support, last time I checked they don't implement VFP support routines to correctly handle denormals (some VFP implementations rely on software feedback to support denormals.)

We could try GOARM=6 on Raspberry Pi 1 with latest FreeBSD and see if the math package tests pass. If it could, we can remove the forced GOARM=5 for FreeBSD.

[davecheney]

The problem was the vfp detection in cmd/dist would crash the kernel on freebsd/arm systems (this was close to two years ago, the situation may have changed now), setting GOARM=5 avoided the crash during bootstrap.

On Fri, Nov 18, 2016 at 10:10 AM, Minux Ma notifications@github.com wrote:

Regarding FreeBSD VFP support, last time I checked they don't implement VFP support routines to correctly handle denormals (some VFP implementations rely on software feedback to support denormals.)

We could try GOARM=6 on Raspberry Pi 1 with latest FreeBSD and see if the math package tests pass. If it could, we can remove the forced GOARM=5 for FreeBSD.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/17082#issuecomment-261399554, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAcA5dLeuXYcvnM_ez7LYCotQfvV_tFks5q_N74gaJpZM4J7K4x .

[gopherbot]

CL https://golang.org/cl/33455 mentions this issue.

[gopherbot]

CL https://golang.org/cl/33686 mentions this issue.

[minux]

We've got two requests for continuing ARMv5 support so far.

https://groups.google.com/forum/#!topic/golang-dev/CO15hgslZ0o https://groups.google.com/d/msg/golang-dev/lWL6L9-DejU/4ybgle4bCgAJ

There will always be ARMv5 hardware in service, like there are still Windows 2000 in service. Yet we've dropped support for the latter. we need to set a clear criteria for retaining ARMv5 support. My proposal is that we need to have a real linux/armv5e builder to retain the support. With a little swap and a lowered GOGC, even 256MB RAM could complete all.bash (using tip to bootstrap), so the builder requirement is not that hard to meet for an ARMv5 machine anymore.

[jtolio]

As I mentioned in the thread but saving for posterity here, Space Monkey is more than happy to provide all of the internet connected 256MB Feroceon ARMv5TE builders with swap you need.

Apologies for not seeing this proposal sooner!

[OrangeTux]

The posts in the 2 threads linked in a previous posts suggests that main reason to drop ARMv5 support is the lack of proper ARMv5 builders. @jtolds already offered help addressing this issue. Is their any chance that his help would make this proposal being rejected?

I don't know how many projects out there that still uses ARMv5E and Go, but I don't expect many.

We use Go in production on hardware based on the AT91SAMG25 chip. Beside some in house Go apps we run Telegraf. We're also planning to use Mender (a tool to update to do embedded firmware updates, it's client is written in Go). We hope to see this proposal being rejected.

But even if this proposal will be rejected this issue raises the question: for how long will Go support the hardware it currently supports? Does Go have a policy around this topic?

[kacf]

Mender chose Go because of its strong embedded support. It seems premature to drop support for ARMv5 if Go is serious about the IoT space, even if ARMv5 is not the most common.

[ALTree]

@OrangeTux

for how long will Go support the hardware it currently supports? Does Go have a policy around this topic?

See Go Release Cycle - release maintenance:

Minor releases to address non-security problems for Go 1.x stop once Go 1.x+1 is released.

So, in theory, if go1.8 is the last version that supports architecture X, you should expect bug fixes for that architecture to stop being pushed as soon as go1.9 is out (go1.10 for security issues).

[fuzzycow]

Apologies for not seeing this sooner, and with full respect and acceptance of Go dev team members decision.

Go was used for hobby and school robotics projects on Lego EV3 platform, which uses ARM926EJ-S, and has 64MB RAM (https://en.wikipedia.org/wiki/Lego_Mindstorms_EV3), on top of ev3dev (http://www.ev3dev.org/)

There are a number of projects on github, in various states of (non)maintenance which use Go to develop for EV3 (https://github.com/search?utf8=%E2%9C%93&q=ev3+language%3Ago) I believe in most cases the compilation was done on another (linux x86_64) system, for GOARM=5 GOARCH=arm target.

[minux]

I don't think we need to support ARMv5 forever, and there will always be ARMv5 hardware out there. I've heard that ARM, the company, has already stopped licensing its ARM9 cores, so at least there won't be any new ARM9 chips.

We can place the proposal on hold, but in the long term, I'd still like to remove ARMv5/ARMv6 support.

[philhofer]

I agree with @minux.

There are concrete disadvantages to ongoing support for ARMv5, beyond simply the maintenance burden. For instance, the runtime enjoys intrinsic atomics on many architectures, but not arm, precisely because Go supports ARM architectures without the modern SMP extensions. (The same is true for a couple other low-level details, like byte-swapping, which are made easier with v7 and newer v6 cpus.)

For as long as gc eschews compiler target flags (I'm agnostic on that point, FWIW), there will be a greater disadvantage to supporting older platforms than there would be for, say, gcc, because the compiler will always have to target the lowest common denominator of hardware capabilities. That's the price we're all paying for toolchain simplicity.

Just my two cents.

[bradfitz]

Maybe more trouble than it's worth, but:

What about copy/pasting GOARCH=arm into two halves, GOARCH=arm (which an evolve to assume modern stuff) and GOARCH=arm5 (which can receive minimal ongoing love, and get simplified in the process, assigning GOARM=5)

I feel like that's been discussed in the past. Is it time to revisit it?

[minux]

I'd expand on the reason why the current ARM backend only targets the lowest common denominator.

It all has to do with the real meaning of $GOARM. As far as the compiler and docs (https://golang.org/doc/install/source) are concerned, GOARM value only select the FPU status (5=softfloat, 6=VFPv1, 7=VFPv3), not ARM architecture versions as most people would expect.

Therefore, the compiler is forced to generate the same code (really the same code, GOARM=5 is supported by cmd/internal/obj inserting a call to softfloat emulation routine before a sequence of floating point instructions) for all three values of GOARM.

We made a mistake when defining GOARM: VFP status and ARM architecture version really are two separate property. For example, ARMv5E chips could also have FPU (at least in theory) and there are ARMv6 chips without VFP.

To continue supporting ARMv5/6 and also bring optimizations enjoyed by other architectures to ARMv7, we need to define a clear way to specify ARM architecture versions, independent of the VFP status, for the compiler.

[philhofer]

If arm were split into two back-ends, I'd argue that aarch32 should be its own architecture, since it's what the ARMv8 chips implement for 32-bit mode, and it's likely to be what most of the new 32-bit designs implement going forward. (Critically, it promises vfpv4 and NEON, which would mean that any of those instructions could safely be generated.) One hopes that it will have better longevity than prior versions, since it's likely to be the last 32-bit arm architecture with the 'A' suffix.

On Thu, Jan 5, 2017 at 5:08 PM, Minux Ma notifications@github.com wrote:

I'd expand on the reason why the current ARM backend only targets the lowest common denominator.

It all has to do with the real meaning of $GOARM. As far as the compiler and docs (https://golang.org/doc/install/source) are concerned, GOARM value only select the FPU status (5=softfloat, 6=VFPv1, 7=VFPv3), not ARM architecture versions as most people would expect.

Therefore, the compiler is forced to generate the same code (really the same code, GOARM=5 is supported by cmd/internal/obj inserting a call to softfloat emulation routine before a sequence of floating point instructions) for all three values of GOARM.

We made a mistake when defining GOARM: VFP status and ARM architecture version really are two separate property. For example, ARMv5E chips could also have FPU (at least in theory) and there are ARMv6 chips without VFP.

To continue supporting ARMv5/6 and also bring optimizations enjoyed by other architectures to ARMv7, we need to define a clear way to specify ARM architecture versions, independent of the VFP status, for the compiler.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/17082#issuecomment-270804305, or mute the thread https://github.com/notifications/unsubscribe-auth/ACzf5uO_t76DBLrm3Ybwdsl5bResOFfUks5rPZQWgaJpZM4J7K4x .

[janflyborg]

Sorry for responding so late, but the company I work for designs and manufactures IP-cameras for surveillance purposes. Some of those are still ARM5 based and it would be a really sad day for us if Go wouldn't be supported on them anymore. Having Go running on them opens up the possibility for us to have the same code base for all of our products. This is a big win for us.

[janflyborg]

And we are of course also willing to donate hardware for building if that is a problem.

[minux]

Please note that what's proposed here is for future Go versions to drop support for ARMv5/v6. ARMv5/v6 users can of course continue to use Go 1.8.

I don't think it's that big a deal because I doubt users of ARMv5 use the most recent version of gcc or linux kernel. But why do you insist on using the latest Go version? Are you really using the latest Go version for your ARMv5/v6 products? As Go 1.8 will receive security updates until at least Feb 2018. Are you still planning to produce and maintain ARMv5/6 hardware in Feb 2018?

(I'm speaking from practical product management prospective: usually, when a product is released, version for all used software will be frozen unless critical security vulnerability warrants a upgrade.)