search

tinyos / tinyos-main

Main development repository for TinyOS (an OS for embedded, wireless devices).
1.41k stars 518 forks source link

Porting tinyos to OS X #297

Open andrasbiro opened 10 years ago

andrasbiro commented 10 years ago

This was discussed in various github and tinyos-devel threads, but it never had it's own thread.

We want to use macports, if you missed the previous threads. However, this raises a question: Where do we want to distribute the portfiles? We could commit them to macports (I'm not sure if it's difficult or not), or create our on repository.

So this is where we are at the moment:

gcc: Nothing seems to work with Apple's LLVM compiler. I've used gcc-mp-4.8, it seems OK. This is fine, but we should probably mention this in the install guide

nesc: There is a portfile at the macports (by Razvan) repository, but it doesn't work https://github.com/tinyos/nesc/issues/22 Can be compiled/installed from source with gcc48. We should eighter update or remove the portfile from macports when we're done.

tinyos-tools: Trunk version compiled with gcc-mp-4.8 is ok, no portfile yet

AVR toolchain: The avr-gcc on macports doesn't includes the atmel patches, so I wouldn't recommend it, I will try to create a portfile/package on the next days.

MSP430 toolchain: This seems to work out of box with ports, although I'm not sure how much it differ from the linux packages.

ARM toolchain: I don't know much about ARM, probably someone else should test it.

Testing I will post portfiles somehow, if you want to test it, please follow this guide to create a local repo, with the portfiles

I'm just a beginner at macports (or os x, for that matter), so there might be big problems, if you have any recommendation, please comment it.

andrasbiro commented 10 years ago

Nesc portfile is done: https://dl.dropboxusercontent.com/u/363226/munka/packages/macports/nesc/Portfile Put it in a local repository at [your repo's directory]/lang/nesc/

cire831 commented 10 years ago

there already exists a portfile for nesc (1.3.4). we should update this port file.

It is easy to do, follow the instructions at https://guide.macports.org/chunked/project.contributing.html

phil-levis commented 10 years ago

Andras,

This is great work. Can you update the nesC portfile as Eric suggests? I think that going through standard channels for port is better than trying to set up a new repository.

bradjc commented 10 years ago

re nesc: one thing to be careful of is that building from the raw github source requires many more dependencies (including emacs) than building from the tarballs on sourceforge (where the ./Bootstrap step is already done). I think the portfile should either pull from sourceforge or we should upload the tarballs to the github releases and have the portfile pull from there.

Another note is that certain versions of emacs can't build nesc 1.3.5 (https://github.com/tinyos/nesc/issues/12, the patch is in nesc master).

andrasbiro commented 10 years ago

Yes, I know about that, fortunately osx provides quite old emacs. I don't want to use tarballs or sourceforge. Since github, noone seems to care with the source tarball (which is understandable). Therefor I would have to change the script back to github with the new release - and it's quite a difference with macports. I would rather see a nesc bugfix release...

phil-levis commented 10 years ago

I find that msp430 isn't working out of the box with port; the msp430-gcc port doesn't include inttypes.h. I tried msp430-gcc-devel @4.7.0-20120911_0 as well as msp430-gcc @4.6.3-20120406_3. What worked for you? I run into the same issues for the avr-gcc ports (4.8.2).

andrasbiro commented 10 years ago

Did you install the full toolchain? It seems like you doesn't have libc. I followed this doc (loosely - I set the prefix for nesc and tinyos-tools, and set the gcc to gcc-mp, etc): https://github.com/tinyos/tinyos-main/blob/master/doc/00d_MacOSX_Development

phil-levis commented 10 years ago

Ah, ooops! I missed msp430-libc, incorrectly thinking that msp430-gcc had triggered its installation (it hadn't). Seems to work now. Thanks!

phil-levis commented 10 years ago

OK, so it seems like right now all that's left is the tools problems (particularly Java, prefix, python, etc.) as per https://github.com/tinyos/tinyos-main/issues/298

bradjc commented 10 years ago

I'm using a fresh install of mac ports to test this and write new portfiles.

  1. I couldn't get nesc 1.3.5 to work because of the issue with gcc/clang and nescc-mig (this is fixed in nesc/master). I think we need to tag nesc/master with v1.3.6 and point to that.

  2. First pass at tinyos-tools. LIttle bit of a hackjob to get it out of tinyos-main repo.

    # $Id$

PortSystem      1.0
PortGroup       github 1.0

name            tinyos-tools
version         2.2.0
github.setup    tinyos tinyos-main master
categories      devel
maintainers     ?
license         BSD

description     Supporting tools for TinyOS operating system.
homepage        http://tinyos.net
platforms       darwin

long_description The TinyOS tools package includes scripts, serial port \
   utilities, and other necessary utilities to install and \
   communicate with TinyOS programs. TinyOS is an operating \
   system for micro-controller based low-power wireless sensors.

checksums       rmd160  a36431cd937e93c3da19d4de07d54b3412d88d69 \
               sha256  35897899d4b5402ade994325d5e98de9fa08325ee84938c4beeb1b5bf5a5ff7d

configure.compiler  macports-gcc-4.8
configure.cmd       cd tools && ./Bootstrap && ./configure

build.cmd           cd tools && make

depends_build       port:autoconf \
                   port:automake \
                   port:python27 \
                   port:python34 \
                   port:nesc

  3. And one for just tinyos. It puts the repo in /opt currently. It could either not download any source or put it elsewhere. Also need to figure out what ports should be dependencies when one installs tinyos.

    # $Id$

PortSystem      1.0
PortGroup       github 1.0

name            tinyos
version         2.2.0
github.setup    tinyos tinyos-main master
categories      devel
maintainers     ?
license         BSD

description     Operating system for low-power wireless devices.
homepage        http://tinyos.net
platforms       darwin

checksums       rmd160  a36431cd937e93c3da19d4de07d54b3412d88d69 \
               sha256  35897899d4b5402ade994325d5e98de9fa08325ee84938c4beeb1b5bf5a5ff7d

long_description TinyOS is an operating system designed for low-power \
   wireless devices, such as those used in sensor networks, \
   ubiquitous computing, personal area networks, smart buildings, \
   and smart meters.

use_configure no
build {}
destroot {
   copy ${worksrcpath} ${destroot}/opt/tinyos-${version}
}

depends_build   port:nesc \
               port:tinyos-tools \
               port:msp430-libc \
               port:avr-libc \
               port:arm-none-eabi-gcc
phil-levis commented 10 years ago

Here's the port file I made:

-- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4

$Id$

PortSystem 1.0 name tinyos-tools version 2.2 categories tools platforms darwin maintainers cs.stanford.edu:pal description Required tools for the TinyOS operating system long_description The TinyOS tools package includes scripts, serial port \ utilities, and other necessary utilities to install and \ communicate with TinyOS programs. TinyOS is an operating \ system for micro-controller based low-power wireless sensors. homepage http://www.tinyos.net master_sites http://tinyos.stanford.edu/tinyos/ distname tinyos-tools

checksums sha256 4fbf7bd7e21ccf4c58fe4b07f41d0fa64946a2cba936f6251aa379c1b20008a6

depends_lib

configure.args

configure.compiler macports-gcc-4.8 build.cmd cd tools && make configure.cmd cd tools && ./Bootstrap && ./configure destroot.cmd cd tools && make install depends_build port:autoconf \ port:automake \ port:nesc \ port:python33

It looks like we are very close -- that's a good sign. Your category makes more sense. How did you get destroot to work without adding a new command? Also, the license isn't GPL-2+, it's BSD?

bradjc commented 10 years ago

Updated the license. As for not needing destroot.cmd, I just didn't specify one and it worked. macports seems to use build.cmd with the argument install for destroot.cmd if one isn't specified.

andrasbiro commented 10 years ago

I finally finished the avr toolchain: https://dl.dropboxusercontent.com/u/363226/munka/macports.zip It's structured as it should, so just unzip it to your local repo and run portindex. Please remove the official avr toolchain before installing this one, I didn't care with conflicts yet. Use avrdude from the official repo, it's working fine. I had to make a portfile for automake-1.11, since the gnu toolchain (even the newest) is preconfigured with that, and when I tried autoreconf, it returned some compatibility problems which didn't seemed too easy to fix. I hope the macports developers still accept it this way. Tinyos-tools: I think @bradjc's version is better than @phil-levis's: The linux/cygwin scripts are using the github tag download method, so we wouldn't need to create the source tarball if the portfile would use the github portgroup.

bradjc commented 10 years ago

I finally finished the avr toolchain: https://dl.dropboxusercontent.com/u/363226/munka/macports.zip It's structured as it should, so just unzip it to your local repo and run portindex. Please remove the official avr toolchain before installing this one, I didn't care with conflicts yet. Use avrdude from the official repo, it's working fine.

I gave this a go.

  1. I believe you are missing a / on line 51 of the avr-binutils-atmel portfile.

  2. I get a macport warning when building avr-binutils-atmel:

       Warning: violation by /opt/local/avr
   Warning: avr-binutils-atmel violates the layout of the port-filesystem!
   Warning: Please fix or indicate this misbehavior...

    Given this is a toolchain I'm not sure why it would need to violate the port file structure.

  3. It is unfortunate that binutils requires automake 1.11 (of all things). I imagine that would make it hard/impossible to upstream the portfile to macports proper. Is there any pushback on Atmel to get their configure scripts to not try to specify a particular autotools version? Doing hacks like that is just broken, and it's unfortunate that our install process would be based on it. The end goal should be to get sudo port install tinyos to work without some step were you have to edit some macports config file and add a url for some third party repo first.
phil-levis commented 10 years ago

On Aug 11, 2014, at 8:58 AM, András Bíró notifications@github.com wrote:

Tinyos-tools: I think @bradjc's version is better than @phil-levis's: The linux/cygwin scripts are using the github tag download method, so we wouldn't need to create the source tarball if the portfile would use the github port group.

I'm on the fence for github tags vs. files. On one hand, the github approach is cleaner. On the other, I don't have 100% confidence that developers won't accidentally screw up tags (which yes, has happened in the past).

Phil

bradjc commented 10 years ago

I'm on the fence for github tags vs. files. On one hand, the github approach is cleaner. On the other, I don't have 100% confidence that developers won't accidentally screw up tags (which yes, has happened in the past).

I think the biggest issue would come from a tag accidentally getting deleted and macports not being able to find the source the on github. Hopefully this won't ever happen. If a new tag gets accidentally created it won't propagate to the portfile immediately, allowing for some time to correct any errors (say if the tag pointed at the wrong commit). I would say the long-term convenience of just being able to tag releases outweighs the possible issues.

andrasbiro commented 10 years ago

@bradjc Thanks for the testing!

  1. Where does it need a "/"? It's just "--enable-doc\"
  2. Yes, I know, this is because it puts the include/libdir to ${prefix}/avr/ The official avr-binutils does the same, so I didn't really cared. But the warning shouldn't be there, "destroot.violate_mtree yes" should disable that. I will take a look.
  3. I'm afraid too that they won't accept the portfiles this way. But this is not an atmel bug. The newest GNU binutils/gcc/gdb uses automake 1.11. Usually, that's not a problem, since they're done the autotools stuff in the source tarball, but Atmel patched some file, so we have to run it manually. It would probably work with the newest automake, but you have to autoreconf it, and it fails with "cygnus tree is depraceted" or something like that. It's way beyond my knowledge to patch it. The other solution is to do all the magic stuff on binutils/gcc before the configure step, and create a huge patch, something like this: https://github.com/arduino/toolchain-avr/blob/atmel-3.4.3/gcc-patches/00-gcc-4.8.1-atmel.patch And I'm starting to think that this would be better.

@phil-levis I see your point. I still think the tag solution would be better, but just slightly.

bradjc commented 10 years ago
  1. Where does it need a "/"? It's just "--enable-doc\"

My bad, I meant in avr-gcc-atmel for the multiple dependencies.

phil-levis commented 10 years ago

On Aug 14, 2014, at 4:15 AM, András Bíró notifications@github.com wrote:

@bradjc Thanks for the testing!

  1. Where does it need a "/"? It's just "--enable-doc\"
  2. Yes, I know, this is because it puts the include/libdir to ${prefix}/avr/ The official avr-binutils does the same, so I didn't really cared. But the warning shouldn't be there, "destroot.violate_mtree yes" should disable that. I will take a look.

OK.

  1. I'm afraid too that they won't accept the portfiles this way. But this is not an atmel bug. The newest GNU binutils/gcc/gdb uses automake 1.11. Usually, that's not a problem, since they're done the autotools stuff in the source tarball, but Atmel patched some file, so we have to run it manually. It would probably work with the newest automake, but you have to autoreconf it, and it fails with "cygnus tree is depraceted" or something like that. It's way beyond my knowledge to patch it. The other solution is to do all the magic stuff on binutils/gcc before the configure step, and create a huge patch, something like this: https://github.com/arduino/toolchain-avr/blob/atmel-3.4.3/gcc-patches/00-gcc-4.8.1-atmel.patch And I'm starting to think that this would be better.

I agree, especially if it's in line with what others are doing (Arduino).

@phil-levis I see your point. I still think the tag solution would be better, but just slightly.

OK.

So it sounds like we're (finally) almost there. OS X certainly is non-trivial. The new server has arrived, it should be up and running next week. We can then put the debian (squeeze + wheezy) packages up for the compilers and tinyos-tools, and start testing the TinyOS code. One to-do is the long-ago raised arbiter issue. I'm going to fix the arbiters to properly follow the correct calling semantics; this might (but probably not!) break some core components, we'll see. I'll fix the arbiters on a branch, synced with master, so that it's easy to isolate a problem to the arbiters.

Phil

cire831 commented 10 years ago

On Sat, Aug 16, 2014 at 6:48 PM, Philip Levis notifications@github.com wrote:

.One to-do is the long-ago raised arbiter issue. I'm going to fix the arbiters to properly follow the correct calling semantics; this might (but probably not!) break some core components, we'll see. I'll fix the arbiters on a branch, synced with master, so that it's easy to isolate a problem to the arbiters.

There is also an arbiter bug that prevents ping ponging. ie.

ResourceUser1 requests and gets the resource and is holding on to it.

ResourceUser2 requests the resource. this is signaled to User1, which releases the resource, and then immediately rerequests the resource. It wants the resource after User2 is done.

The above doesn't work correctly. User2 never gets the resource.

I am working on replicating the failure mechanism. And after I replicate it, I'll file a bug report and also submit the fix.

Phil

— Reply to this email directly or view it on GitHub https://github.com/tinyos/tinyos-main/issues/297#issuecomment-52411221.

Eric B. Decker Senior (over 50 :-) Researcher

phil-levis commented 10 years ago

Which arbiter? I think a specific bug issue is a step back -- rather than deal with bugs piecemeal, I want to precisely specify the correct calling semantics for an arbiter (as per the TEP) and then ensure each arbiter implementation follows them. Right now there are differences.

Phil

On Aug 16, 2014, at 11:05 PM, Eric Decker notifications@github.com wrote:

On Sat, Aug 16, 2014 at 6:48 PM, Philip Levis notifications@github.com wrote:

.One to-do is the long-ago raised arbiter issue. I'm going to fix the arbiters to properly follow the correct calling semantics; this might (but probably not!) break some core components, we'll see. I'll fix the arbiters on a branch, synced with master, so that it's easy to isolate a problem to the arbiters.

There is also an arbiter bug that prevents ping ponging. ie.

ResourceUser1 requests and gets the resource and is holding on to it.

ResourceUser2 requests the resource. this is signaled to User1, which releases the resource, and then immediately rerequests the resource. It wants the resource after User2 is done.

The above doesn't work correctly. User2 never gets the resource.

I am working on replicating the failure mechanism. And after I replicate it, I'll file a bug report and also submit the fix.

Phil

— Reply to this email directly or view it on GitHub https://github.com/tinyos/tinyos-main/issues/297#issuecomment-52411221.

Eric B. Decker Senior (over 50 :-) Researcher — Reply to this email directly or view it on GitHub.

andrasbiro commented 10 years ago

The huge patch method really creates a huge patch: 4.3MB for binutils (most of it is deleted files in the atmel tree). So I think I will still use the atmel tarballs instead of the gnu ones, but I will do all autotool related stuff in a patch. This eliminates the version dependent difficulties, but creates a much smaller patch (500k, but it will be much smaller when I do it on my build system with the correct autoconf/automake version - right know it's full with licence year changes). By the way, I think the arduino project uses a much older toolchain. That patch is probably still in developement.

andrasbiro commented 10 years ago

I can't go lower than 500k for binutils: atmel or gnu didn't follow they're own documentation, they used various autoconf/automake versions which causes a lot of garbage in the patch. But it turned out that gcc doesn't need autotools at all (the patch was empty), and they packaged libc quite differently, so the autotool patch became more then 10MB, but it works with any auto* version, so I don't care about that. Result:

I will update the linux and mac scripts according to this early next week.

andrasbiro commented 10 years ago

I've updated the buildscript for linux and mac (I didn't regenerated the linux pkgs. These changes shouldn't affect the binaries). Changlog: 54a0e88d808f0bd102caa1e30ef9d4f1ee28ecdf d745ec0bd18e5118266e34cf45c1f5ba03d0ec79 I've fixed the binutils warning: turned out those files are misplaced in the official binutils as well. I've also added the conflicts lines for the corresponding GNU version. https://dl.dropboxusercontent.com/u/363226/munka/macports.zip I'm quite satisfied with these scripts. I will send a pull request to macports soon, if there's no objection.

bradjc commented 10 years ago

I just used the new avr portfiles and they worked great. Post a link to the macports request when you pull the trigger so we can keep an eye on it/motivate macports if necessary.

andrasbiro commented 10 years ago

I've opened the submission tickets for the avr toolchain: https://trac.macports.org/ticket/45328 https://trac.macports.org/ticket/45329 https://trac.macports.org/ticket/45330

NathanBlais commented 5 years ago

We still can't install TinyOS on OS X :( Is possible to do so, or would it take a significant rewrite to achieve it. Is it possible to use llvm instead of gcc?

cire831 commented 5 years ago

its been done but isnt supported (i need some one to manage it)

On Sun, Apr 7, 2019 at 6:22 PM Nathan_Blais notifications@github.com wrote:

We still can't install TinyOS on OS X :( Is possible to do so, or would it take a significant rewrite to achieve it. Is it possible to use llvm instead of gcc?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tinyos/tinyos-main/issues/297#issuecomment-480652415, or mute the thread https://github.com/notifications/unsubscribe-auth/AAY46WMaJuUgMrPJEbwuidLRalx5WvYbks5vepm9gaJpZM4COn-g .

-- Eric B. Decker Senior (over 50 :-) Researcher

ytlw commented 4 years ago

I think this software should outlast the machine. It can still download the binary of msp430-gcc from http://sing.stanford.edu/klueska/tinyos2_mac_install/msp430-tools-i686-apple-darwin.tar.gz The problem is that the recent version OS X(10.15 Catalina) no longer supports i386, so can everybody contribute a method for compiling msp430-gcc from source code?

cire831 commented 4 years ago

the msp430 toolchain has a number of issues that makes it difficult to build and use. I've personally switched to ARM based platforms.

I do not have any motivation to fix the msp430 toolchain.