Closed abarisani closed 4 years ago
ALTree
commented
4 years ago From Requirements for a new port:
A developer must be named (and agree) to maintain the builder, the machine trying each git revision and providing data for https://build.golang.org
How is the builder story?
randall77
commented
4 years ago I'm confused. You say you needed to modify the compiler. But when I look at the diff between f-secure-foundry/tamago-go and go tip, there don't appear to be any changes to cmd/compile. I wouldn't expect any changes to the compiler would be needed. Do you mean runtime or linker instead?
eliasnaur
commented
4 years ago Related to https://github.com/golang/go/issues/35956. I'd be very interested in the amd64 port of GOOS=none/noos/tamago, even if it only supports virtual machines.
abarisani
commented
4 years ago I'm confused. You say you needed to modify the compiler. But when I look at the diff between f-secure-foundry/tamago-go and go tip, there don't appear to be any changes to cmd/compile. I wouldn't expect any changes to the compiler would be needed. Do you mean runtime or linker instead?
You are correct, my bad. We don't modify cmd/compile at all. We modify mainly the runtime, stdlib packages (mostly for stubs and build flags) and the dist and link commands to add the additional GOOS.
abarisani
commented
4 years ago Related to #35956. I'd be very interested in the amd64 port of GOOS=none/noos/tamago, even if it only supports virtual machines.
We have an internal working amd64 port which runs under qemu, but it's not our focus and quite invasive. For this reason we propose only an ARM port as our target is ARM based embedded systems.
Having said that this might pave the future, if accepted, for future similar efforts on other architectures.
abarisani
commented
4 years ago From Requirements for a new port:
A developer must be named (and agree) to maintain the builder, the machine trying each git revision and providing data for https://build.golang.org
How is the builder story?
Thanks, I'll look into it.
eliasnaur
commented
4 years ago Related to #35956. I'd be very interested in the amd64 port of GOOS=none/noos/tamago, even if it only supports virtual machines.
We have an internal working amd64 port which runs under qemu, but it's not our focus and quite invasive. For this reason we propose only an ARM port as our target is ARM based embedded systems.
Interesting. Are you able to share the implementation? I'd also like to know what makes the amd64 port more invasive than the arm port. Thanks.
abarisani
commented
4 years ago Related to #35956. I'd be very interested in the amd64 port of GOOS=none/noos/tamago, even if it only supports virtual machines.
We have an internal working amd64 port which runs under qemu, but it's not our focus and quite invasive. For this reason we propose only an ARM port as our target is ARM based embedded systems.
Interesting. Are you able to share the implementation? I'd also like to know what makes the amd64 port more invasive than the arm port. Thanks.
Unfortunately we are not confident in sharing this implementation, our focus is just the ARM version for now. The amd64 port is more invasive due to the way the MMU needs to be managed, additionally multi-core support requires to add a thread scheduler in the runtime which is something we don't require for single-core ARM environments.
So given the complexity and lack of interest on our side we used this prototype only to initially validate the concept and then move over to ARM.
There are other efforts that implemented an amd64 port but they have been implemented in a way which I think it makes it too invasive for upstream adoption, see here and here.
As our goal is to run embedded firmware without C we don't focus on hypervisor support (as the hypervisor is in C anyway) and running without one on amd64 is very challenging given the amount of drivers required imho.
Not saying it's not doable ;), just not our focus.
abarisani
commented
4 years ago From Requirements for a new port:
A developer must be named (and agree) to maintain the builder, the machine trying each git revision and providing data for https://build.golang.org
How is the builder story?
Thanks, I'll look into it.
I checked docs. Ee would have no issues in providing a builder for this and maintain it.
abarisani
commented
4 years ago As mentioned in my initial issue message, I am quite sure that "tamago" needs to be changed for adoption as it's catchy ;) but too specific.
Therefore in the spirit of https://github.com/golang/go/issues/35956 I think we should settle on GOOS=none.
I can ensure that the proposed changes only kick in with GOOS=none, GOARCH=arm and GOARM=7 which is what this code currently targets.
FiloSottile
commented
4 years ago I am not qualified to comment on the runtime and linker changes, but from the security point of view, this would enable fantastic applications for embedded deployments. Easy to write, fully memory-safe firmware on consumer hardware would not only replace a lot of HSMs, but make hardware security available to many more projects. For example, I'd make use of it immediately in age, and I'm sure a lot of people would have more creative uses for it.
Secure elements and trusted hardware provide security properties that are simply impossible to replicate in software, like defending low-entropy passcodes from brute force.
I am truly looking forward to this being a mature and robust target, which is why I'd love to see it supported in the main tree.
rsc
commented
4 years ago This proposal seems like an exact duplicate of #35956. See my comments starting at https://github.com/golang/go/issues/35956#issuecomment-561806440. Is this situation different somehow (other than ARM vs x86)?
beoran
commented
4 years ago I understand the Go team itself cannot support Go on bare metal. However, there are many people who do want to support this. This suggests to me that the GOOS and GOARCH should be made pluggable somehow.
abarisani
commented
4 years ago This proposal seems like an exact duplicate of #35956. See my comments starting at #35956 (comment). Is this situation different somehow (other than ARM vs x86)?
Yes it is, please see my earlier comment. We think that arm bare metal support is considerably less complicated, the principle is the same but the implementation is far easier.
We also target embedded systems as we see much more value in running Go there than on amd64 based hypervisors, so also the vision is slightly different.
abarisani
commented
4 years ago I understand the Go team itself cannot support Go on bare metal. However, there are many people who do want to support this. This suggests to me that the GOOS and GOARCH should be made pluggable somehow.
We would be happy to maintain this just like other folks did for other ports (for example js/wasm). In fact the effort required to take this in is less, or comparable, to js/wasm in terms of LOC.
beoran
commented
4 years ago Yes, exactly, this is an archtecture that has 3rd party support. It would be easier if third party architectures could be used with the main go compiler without having to be integrated in the main repository. Maybe like this, add a GO3PARTY environment variable which should point to a go module with the same layout as the main Go repository that the compiler will execute tools in when either GOARCH or GOOS are unknown.
FiloSottile
commented
4 years ago Another difference from #35956 is that @abarisani indeed demoed an HTTPS server running on the bare platform, already crossing the threshold into being useful for applications.
abarisani
commented
4 years ago Yes, exactly, this is an archtecture that has 3rd party support. It would be easier if third party architectures could be used with the main go compiler without having to be integrated in the main repository. Maybe like this, add a GO3PARTY environment variable which should point to a go module with the same layout as the main Go repository that the compiler will execute tools in when either GOARCH or GOOS are unknown.
I honestly think this would add complications and open up to more "dirty hacks" if there is easy access in modifying the runtime.
Given the simplicity of our implementation a GO3PARTY might open up more problems than what it needs to solve in my opinion.
rsc
commented
4 years ago The main concerns I listed in https://github.com/golang/go/issues/35956#issuecomment-561806440 are:
I read your linked reply and don't see any differences between ARM and x86 on these. If anything, there are almost certainly more low-cost oddly-behaved ARM devices than x86 devices, making the chip errata problem worse. (We spent so long debugging bad ARM chips just running builders for the linux/arm port.)
This still seems like something that should be done out-of-tree, as a fork of the golang/go repo. Am I missing something?
abarisani
commented
4 years ago SoC erratas are not a concern in our changes against Go, our runtime changes only touch very few ARM standard specifications. The low level parts of the runtime changes are only these. So this is truly not a problem in my opinion.
We don't modify os or net other than minor stubbing or selection identical to what other architectures like plan9, js/wasm or nacl are doing. At the same time there are useful implementation, our example application has fully working Ethernet over USB by simply pulling gvisor networking in. So in a nutshell there is no new huge amount of code in the changes, not much to maintain yet at the same time we can achieve full Go functionality.
I'd encourage to please take a look at the diff and at my presentation and slides and the example application.
The (small) complexity is separated from the changes required to upstream, as all drivers are in a separate package which is here, most drivers are very simple while some others have little more code, such as the USB one but still very compact and clean in my opinion. But yet again this code is not some we are requesting to be upstreamed.
So in a nutshell chip erratas, if present, would be taken care in drivers that are outside the requested changes for the Go distribution.
Useful implementations are possible (I am developing a pure Go smartcard right now with this).
Drivers are already out-of-tree in our proposal and the changes to the Go distribution are minimal, isolated and clean to address all complexity/maintainability concerns within.
abarisani
commented
4 years ago To further add to my comments, this is a concrete example of what we accomplished using the proposed changes:
Demo video: https://twitter.com/AndreaBarisani/status/1247177053482487809 Repository: https://github.com/f-secure-foundry/GoKey
rsc
commented
4 years ago This still seems beyond the scope of the core Go team. It still seems to me that tamago should git clone the go repo, make changes, and maintain a fork. That would not be much different from tinygo. Git was made for exactly this kind of thing!
abarisani
commented
4 years ago I respect your opinion, but to me it seems inconsistent with support for other architectures that are within go already. Then why not proposing the same approach for wasm ? What's the difference? Why is one within the scope of the core Go team and the other not ? What is the criteria?
ianlancetaylor
commented
4 years ago That's a fair question. I don't know that we have specific criteria.
I looked at your patch. A lot of the file changes seem to be build tag additions. And a discouraging amount of the changes seem to be copying the NaCl code that we just got rid of.
Here is the diffstat of the patch:
Personally I think that we should start by trying to make this patch smaller. Let's try to think of some approach that reduces the number of build tag edits that are required. Perhaps we can introduce some sort of mapping by which build tags imply other build tags, as android implies linux, and then use that to avoid having to edit so many files.
More generally, we want to minimize the maintenance burden on the Go maintainers. It's an essential step that you are volunteering to maintain the port. But the burden remains. We don't want to break ports as we go, which is why changes like https://golang.org/cl/171823 wind up changing 30 files.
So lets start minimizing the new code required by any new port. I suggest that we take steps in that direction and see how far we can get.
cherrymui
commented
4 years ago Personally, I think the difference is that for Wasm, you can run it on any machine that has a modern browser; on the other hand, this requires special hardware, which makes maintaining, testing, debugging much harder.
beoran
commented
4 years ago When I look at that diff, it seems to me that platform support is spread out all over the Go runtime and compiler. In other words, it could be factored better, into a package, or a go module for each architecture and each platform. That could lessen the burden both for the Go developers and for those who want to maintain a platform themselves.
abarisani
commented
4 years ago That's a fair question. I don't know that we have specific criteria.
I looked at your patch. A lot of the file changes seem to be build tag additions. And a discouraging amount of the changes seem to be copying the NaCl code that we just got rid of.
Here is the diffstat of the patch:
.github/CODE_OF_CONDUCT.md | 3 .github/ISSUE_TEMPLATE | 38 .github/PULL_REQUEST_TEMPLATE | 25 .github/SUPPORT.md | 14 b/.github/workflows/build.yml | 16 b/src/cmd/dist/build.go | 2 b/src/cmd/dist/test.go | 4 b/src/cmd/dist/util.go | 4 b/src/cmd/go/go_test.go | 4 b/src/cmd/go/internal/base/signal_unix.go | 2 b/src/cmd/go/internal/imports/build.go | 1 b/src/cmd/go/internal/work/build_test.go | 2 b/src/cmd/go/internal/work/exec.go | 6 b/src/cmd/go/internal/work/gccgo.go | 4 b/src/cmd/go/internal/work/init.go | 4 b/src/cmd/internal/obj/x86/asm6.go | 6 b/src/cmd/internal/obj/x86/obj6.go | 3 b/src/cmd/internal/objabi/head.go | 5 b/src/cmd/link/internal/amd64/asm.go | 6 b/src/cmd/link/internal/amd64/obj.go | 1 b/src/cmd/link/internal/arm/asm.go | 3 b/src/cmd/link/internal/arm/obj.go | 3 b/src/cmd/link/internal/ld/config.go | 8 b/src/cmd/link/internal/ld/elf.go | 9 b/src/cmd/link/internal/ld/link.go | 6 b/src/cmd/link/internal/ld/sym.go | 3 b/src/crypto/rand/rand_tamago.go | 29 b/src/crypto/x509/root_tamago.go | 10 b/src/crypto/x509/root_unix.go | 2 b/src/go/build/syslist.go | 2 b/src/internal/poll/errno_unix.go | 2 b/src/internal/poll/fcntl_tamago.go | 14 b/src/internal/poll/fd_fsync_posix.go | 2 b/src/internal/poll/fd_poll_runtime.go | 2 b/src/internal/poll/fd_posix.go | 2 b/src/internal/poll/fd_unix.go | 2 b/src/internal/poll/hook_unix.go | 2 b/src/internal/poll/sys_cloexec.go | 2 b/src/internal/syscall/unix/nonblocking_tamago.go | 9 b/src/mime/type_unix.go | 2 b/src/net/error_posix.go | 2 b/src/net/error_tamago.go | 9 b/src/net/file_stub.go | 2 b/src/net/hook_unix.go | 2 b/src/net/http/roundtrip.go | 2 b/src/net/interface_stub.go | 2 b/src/net/internal/socktest/switch_unix.go | 2 b/src/net/internal/socktest/sys_unix.go | 2 b/src/net/iprawsock_posix.go | 2 b/src/net/ipsock_posix.go | 2 b/src/net/lookup_fake.go | 2 b/src/net/main_noconf_test.go | 2 b/src/net/net_fake.go | 2 b/src/net/port_unix.go | 2 b/src/net/rawconn_stub_test.go | 2 b/src/net/sendfile_stub.go | 2 b/src/net/sock_stub.go | 2 b/src/net/sockaddr_posix.go | 2 b/src/net/sockopt_stub.go | 2 b/src/net/sockoptip_stub.go | 2 b/src/net/tcpsock_posix.go | 2 b/src/net/tcpsockopt_stub.go | 2 b/src/net/udpsock_posix.go | 2 b/src/net/unixsock_posix.go | 2 b/src/os/dir_unix.go | 2 b/src/os/error_posix.go | 2 b/src/os/error_unix_test.go | 2 b/src/os/exec/lp_tamago.go | 23 b/src/os/exec_posix.go | 2 b/src/os/exec_unix.go | 2 b/src/os/executable_procfs.go | 2 b/src/os/file_posix.go | 2 b/src/os/file_unix.go | 2 b/src/os/path_unix.go | 2 b/src/os/pipe_tamago.go | 9 b/src/os/signal/signal_tamago.go | 7 b/src/os/stat_tamago.go | 54 + b/src/os/stat_unix.go | 2 b/src/os/sys_tamago.go | 15 b/src/os/user/lookup_stubs.go | 2 b/src/os/user/lookup_tamago.go | 29 b/src/os/wait_unimp.go | 2 b/src/path/filepath/path_unix.go | 2 b/src/runtime/cgo/gcc_setenv.c | 2 b/src/runtime/cgo/setenv.go | 2 b/src/runtime/env_posix.go | 6 b/src/runtime/internal/sys/zgoos_aix.go | 1 b/src/runtime/internal/sys/zgoos_android.go | 1 b/src/runtime/internal/sys/zgoos_darwin.go | 1 b/src/runtime/internal/sys/zgoos_dragonfly.go | 1 b/src/runtime/internal/sys/zgoos_freebsd.go | 1 b/src/runtime/internal/sys/zgoos_hurd.go | 1 b/src/runtime/internal/sys/zgoos_illumos.go | 1 b/src/runtime/internal/sys/zgoos_js.go | 1 b/src/runtime/internal/sys/zgoos_linux.go | 1 b/src/runtime/internal/sys/zgoos_netbsd.go | 1 b/src/runtime/internal/sys/zgoos_openbsd.go | 1 b/src/runtime/internal/sys/zgoos_plan9.go | 1 b/src/runtime/internal/sys/zgoos_solaris.go | 1 b/src/runtime/internal/sys/zgoos_tamago.go | 25 b/src/runtime/internal/sys/zgoos_windows.go | 1 b/src/runtime/internal/sys/zgoos_zos.go | 1 b/src/runtime/lock_tamago.go | 150 +++ b/src/runtime/malloc.go | 13 b/src/runtime/mem_tamago.go | 205 ++++ b/src/runtime/mmap.go | 1 b/src/runtime/netpoll.go | 2 b/src/runtime/netpoll_fake.go | 2 b/src/runtime/os_tamago_arm.go | 418 +++++++++ b/src/runtime/proc.go | 20 b/src/runtime/rt0_tamago_arm.s | 19 b/src/runtime/runtime1.go | 4 b/src/runtime/signal_tamago.go | 13 b/src/runtime/sigqueue.go | 1 b/src/runtime/stubs2.go | 1 b/src/runtime/stubs_nontamago.go | 9 b/src/runtime/sys_tamago_arm.s | 133 +++ b/src/runtime/timestub2.go | 1 b/src/syscall/asm_tamago_arm.s | 14 b/src/syscall/dirent.go | 2 b/src/syscall/env_unix.go | 2 b/src/syscall/fd_tamago.go | 253 +++++ b/src/syscall/fs_tamago.go | 853 +++++++++++++++++++ b/src/syscall/net_tamago.go | 973 ++++++++++++++++++++++ b/src/syscall/syscall_tamago.go | 321 +++++++ b/src/syscall/tables_tamago.go | 494 +++++++++++ b/src/syscall/time_tamago_arm.s | 11 b/src/syscall/timestruct.go | 2 b/src/syscall/zsyscall_tamago_arm.go | 25 b/src/syscall/zsysnum_tamago_arm.go | 7 b/src/time/zoneinfo_read.go | 2 b/src/time/zoneinfo_read_tamago.go | 21 b/src/time/zoneinfo_tamago.go | 13 133 files changed, 4349 insertions(+), 168 deletions(-)Personally I think that we should start by trying to make this patch smaller. Let's try to think of some approach that reduces the number of build tag edits that are required. Perhaps we can introduce some sort of mapping by which build tags imply other build tags, as
androidimplieslinux, and then use that to avoid having to edit so many files.More generally, we want to minimize the maintenance burden on the Go maintainers. It's an essential step that you are volunteering to maintain the port. But the burden remains. We don't want to break ports as we go, which is why changes like https://golang.org/cl/171823 wind up changing 30 files.
So lets start minimizing the new code required by any new port. I suggest that we take steps in that direction and see how far we can get.
I am honestly not quite sure how this can be made more minimal, we spent a lot of effort in making it as clean and minimal as possible. The tag additions are inevitable and a direct consequence on how go manages new architectures. If you think there is a better way, by all means, I am all ears.
The only think we pull from NaCl is the in-memory filesystem and simulated netowrk, which is something we can get rid of as it has little value for us and it will be replaced with a real fs driver anyway. Everything else pulled from NaCl is extremely small and only stubs. Please point me to other things that you think are discouraging and I'll be more than happy to try and get rid of them.
Do you think there is value in getting rid of these first? Regarding tags I am open to understand how to avoid it because I frankly just followed how other architectures are supported anyway.
abarisani
commented
4 years ago Personally, I think the difference is that for Wasm, you can run it on any machine that has a modern browser; on the other hand, this requires special hardware, which makes maintaining, testing, debugging much harder.
This can be executed under qemu, making debugging (using gdb) trivial.
abarisani
commented
4 years ago When I look at that diff, it seems to me that platform support is spread out all over the Go runtime and compiler. In other words, it could be factored better, into a package, or a go module for each architecture and each platform. That could lessen the burden both for the Go developers and for those who want to maintain a platform themselves.
Indeed, this would provide a generic benefit however (much needed) and not specific to this effort. We too thought it was cumbersome to having to touch so many files to add an architecture, however in certain ways I understand why it ended up this way and why it was necessary.
ianlancetaylor
commented
4 years ago @abarisani I agree that as the tree stands today your patch probably can't be made more minimal. That in itself discourages us from accepting your patch. I'm suggesting that we explore mechanisms, unrelated to your patch, that will permit your patch to become smaller.
ianlancetaylor
commented
4 years ago @abarisani We've run into quite a few problems with QEMU over the years, each of them extremely painful to debug. We can't rely on it for debugging a port that is intended to run on real hardware, or, indeed, anywhere other than using QEMU itself.
abarisani
commented
4 years ago @abarisani I agree that as the tree stands today your patch probably can't be made more minimal. That in itself discourages us from accepting your patch. I'm suggesting that we explore mechanisms, unrelated to your patch, that will permit your patch to become smaller.
That would be fine by me and happy to help, if this is a trigger to bring a general improvement I can only be happy about it.
abarisani
commented
4 years ago @abarisani We've run into quite a few problems with QEMU over the years, each of them extremely painful to debug. We can't rely on it for debugging a port that is intended to run on real hardware, or, indeed, anywhere other than using QEMU itself.
We would be happy to sponsor a certain number of boards to any core developer that wishes to be able to debug this. On the other hand we are also happy to be the maintainers for this architecture and ensure that it remains up to date (as we are doing already) and doesn't break anything. The integration within the standard distribution and runtime is so minimal that we think it makes it very easy even without a real target.
beoran
commented
4 years ago Should we make a new issue on refactoring the platform and OS support in runtime and compiler so it is easier to make ports if Go with minimal changes?
dmitshur
commented
4 years ago Yes, I think it would be better to use a separate issue for investigating and discussing that topic, since it would apply to many ports.
ianlancetaylor
commented
4 years ago I agree on having a separate issue.
@abarisani Thanks for the generous offer to sponsor development boards. I do want to note that @rsc raised a significant concern that that does not address, which is that adding this port will leave the Go runtime team responsible for fixing problem that arise on different ARM boards using different ARM processors. I understand that you are willing to help, but the runtime team will still bear the primary responsibility for understanding bugs that arise when people use this new platform.
beoran
commented
4 years ago There, I did impetuously charge forward and made that new issue. Hope you don't mind my clumsy writing, I hope everyone will contribute to it. Perhaps we can put this issue on hold while we discuss #38364?
rsc
commented
4 years ago Based on the discussion above, this seems like a likely decline. (We've noted multiple times that the amount of change plus the burden on the core team for maintenance are too large here.)
abarisani
commented
4 years ago @rsc I am puzzled, except for your comments every other single one is either supportive of this patch or acknowledges that the patch is as minimal as it gets and cannot get more minimal for reasons which are outside the patch itself, but rather generic issues (and for this reason a separate issue has been filed).
I really don't share the assessment "based on the discussion above" as you put it. Maybe I am biased, but I really feel this conclusion doesn't reflect at all the discussion that has been taking place.
Would it help if I can show how minimal the code to maintain is outside the network and fd ones copied from NaCl (which I can remove from the patch, I don't think they are necessary in the end) and outside the tedious one liner GOOS additions (which really require no maintenance) ? I think we are talking about no more than 500 lines of new code in the end...
networkimprov
commented
4 years ago Andrea, do you know of any tamago users or folks working with the hardware it supports inside Google?
abarisani
commented
4 years ago Andrea, do you know of any tamago users or folks working with the hardware it supports inside Google?
I am not sure I understand the relevance of this question.
embeddedgo
commented
4 years ago Hello @abarisani
I am in a similar situation to you. I'm maintainer of linux/thumb an noos/thumb ports: https://github.com/embeddedgo
The noos/thumb port allows to program for ARMv7-M architecture. The ARMv7-M microcontrollers are the most popular 32-bit MCUs and at the same time they are much more standardized that any other embedded systems including these based on ARMv7-A.
The problem is that the number of Go programmers interested in bare-metal programming even on such mainline MCUs is small. Enforce the Go team to track not so very popular port in the main repository doesn't make much sense. You can do it yourself at the side which doesn't stop you to try to introduce some more widely useful changes you want to the reference compiler.
That's my opinion.
I'm open to collaborate with you in the field of bare-metal programming. Currently I'm starting working on nnos/riscv64 port targeting Kendryte k210 AI capable chip. It's a new architecture to me. It can be a good starting point to some bare-metal collaboration :)
-- Michal
creker
commented
4 years ago The problem is that the number of Go programmers interested in bare-metal programming even on such mainline MCUs is small.
That's where the official port will be a big help. I think the interest in ability to write secure and fast programs in a safer and/or simpler to write languages like Go or Rust is quite high. People want to get away from C and even C++. Rust requires you to go with no_std which is a high price to pay. Go on the other hand is proven to work with pretty much all std library intact. That's a big win and would open up a new market for Go.
embeddedgo
commented
4 years ago People want to get away from C and even C++.
I'm not sure about that. What I usually hear from such people is: "Yes, I will try, but if there will be a way to disable GC".
You can count more on people from Micropython world but it's still too small. You can count the most on the current Go programmers that for various reasons need to program on the bare-metal hardware but there are only few of them.
creker
commented
4 years ago @embeddedgo usually people talk about disabling GC not based on actual performance metrics of specific GC implementation but just because they've been taught that way. GC - bad, C++ - good. I don't see any problem running Go GC in embedded applications where there's not much going on to fear that it will eat up too much of the resources. The heap is tiny. And it looks like @abarisani had the same experience. GC is not a problem in practice.
embeddedgo
commented
4 years ago I have the same experience as you.
Go GC is very well suited for embedded systems because of the way it avoids fragmentation. You can even write interrupt handlers in Go with the warranty that GC will not disturb them. You can do much on small platform like nRF52840 with only 256 KB of RAM.
GC isn't a problem but an advantage of Go.
But try to convince C/C++ people to it.
Of course there are some problems like fat binaries or the lack of the way to leave read-only data in read-only memory which are difficult to argue with. But surprisingly, nobody of C people bothers about it. You hear mainly how bad is GC for embedded programming.
abarisani
commented
4 years ago The embedded world desperately needs powerful, yet easy to use and memory safe, libraries for security as well as operational tasks.
So far you either have to use low level bare metal firmware (e.g. C) or high level languages but with the burden of complex OS runtimes and all its dependencies involved, just shifting away C/low level code as a dependency.
This is the direct consequence for low quality/security firmware in most IoT devices.
Being able to run something like Go on the bare metal removes this burden and greatly reduces the attack surface (please see the presentation linked in the very first message).
Go mission is to enable building expressive, concise, clean, and efficient code. Being able to do so without OS or non-Go dependencies on embedded systems, to me, greatly follows the spirit of why this language was created. It just takes it to unexpected, but welcome, territories.
My personal smartcard is now written in pure Go, if this doesn't excite the Go community I am not sure what can.
cherrymui
commented
4 years ago It is definitely exciting!
The question is, why it needs to be in this repository. Go is a language with a spec, so it can have different implementations. Besides this repository, there are other implementations of Go, like gccgo, tinygo, etc., with different targets. All live in different repositories. This could be just another one.
abarisani
commented
4 years ago In my humble opinion because the required changes to the runtime and linker are minimal, and having them incorporated and maintained as part of upstream Go provides more orchestrated long term support, more overall trust (I am security oriented) and gives exposure to a wider audience.
Also we want to use this specific implementation because it's the only one that can well adapted for this task, there is a reason why we achieved this by modifying vanilla Go and not using gccgo or tinygo. These projects have different goals, different requirements and therefore need different implementations.
We specifically aimed at having the least possible amount of changes against the standard Go distribution because to us that was the most sensible and clean way to achieve this.
As our overall changes to the distribution are minimal it really doesn't make sense to treat this separately in our opinion. I encourage you to check my initial presentation and slides as well as the patch to fully understand the motivation and differences with tinygo et al.
networkimprov
commented
4 years ago There is a third way; release tamago as a patchset and script that deploys tamago into the Go versions you support.
If your customers require it, you can also ship binary Go tarballs that have the patchset applied.
Hello,
I am unsure if to go directly with a CL (as this has been discussed already on golang-dev) or if to go through this issue first, apologies if I should have done otherwise.
On this golang-dev thread I described the new GOOS=tamago created to support bare metal ARMv7 architectures using the original Go compiler.
The use case for such support is to run native Go applications on Single Board Computers without and underlying Operating System. Currently we are testing this on our own USB armory but we are working on Raspberry PI Zero support as well.
We have working examples of SSH and HTTPS servers running on Ethernet over USB with all USB drivers and stack implemented in pure Go. We are also working on a pure Go OpenPGP card leveraging on this.
We think there is great value in having this kind of support in the Go language as it considerably improves the security and ease of development on embedded systems and allows to purge C, with ease, on otherwise complex firmware efforts.
Our work is split across the following 3 repositories:
For upstream consideration only the first repository matters, we intentionally kept the compiler changes separate from SoC specific drivers and ensure that Go applications can run mostly unencumbered and with a single import required for the board package.
You can find the following resources to understand more about our effort:
Our compiler modification approach tries to be as minimal and clean as possible, respecting or reusing existing Go runtime code. A new
GOOS=tamagois added only for GOARCH=arm, you can see the changes here.We are committed in maintaining the required patches long term, we also understand that any potential upstream inclusion would require renaming
tamagoto something more suited (others have been usingnoosin other proposals), which is totally fine.The patches should also be easy to revert if necessary.
To further prove the maintainability and isolation of this new GOOS from the existing code base, we began our work on go1.12, then rebased it to go1.13. Bumping it to 14.1 only took a small effort.
We built
GOOS=tamagospecifically to make it upstream friendly and have the minimum possible set of changes as our focus is trust for security applications and maintainability without reinventing or adding too much code to the existing runtime.We would much like to start a conversation to have these changes accepted upstream and take responsibility to maintain them.
Thanks