StratifyLabs / StratifyOS

A Powerful embedded RTOS for ARM Cortex M microcontrollers
https://stratifylabs.co
Other
114 stars 23 forks source link

Is it possible to install the SDK without a cloud login? #230

Closed syntroniks closed 4 years ago

syntroniks commented 4 years ago

I'm developing a Docker image that will be used for kernel builds.

The Docker image aims to install & build a SDK (with specific libraries called out).

This image will serve as the base for application development as well.

All of this will be happening in a continuous build system, so the interaction is different from an independent developer setting up their workspace.

There are a number of solutions that should fit the bill. Here are some proposals:

When I mention SDK below, I am talking about the results of sl sdk.install, not sl sdk.update

  1. Supply the SDK as a package that can be downloaded without authentication
  2. Allow users to build their own SDK (presumably using the same steps StratifyOS authors do)
  3. Supply Docker base images

Personally I am partial to (2) since it gives flexibility in 'toolchain' (i.e. gcc version) selection and has the capability of excluding arches that won't be used (STM).

tyler-gilbert commented 4 years ago

@syntroniks Yeah. That is interesting. Right now, sl sdk.install downloads a file from Firebase storage and there isn't a way to do that unless you are logged in. But it just downloads a zip/7z file extracts it and deletes it. I think you can use: sl sdk.publish:dryrun to pull the local machine's SDK into an archive.

I don't have much experience with Docker but I am interested in how this works out. I run CI on several self-hosted servers. That way I can actually connect hardware to them and execute tests.

syntroniks commented 4 years ago

That's really good news. I ended up harvesting the cached toolchain from ~/.sl. No need to sdk install now, we can update the toolchain on an as-needed basis (but would prefer a wget link)

The current issue is that sdk.update requires a login, although I would estimate (for our needs) this just needs to hit git repos and "make" them.

I suspect in the backround this may perform toolchain updates.

Is it feasible to decouple these activities?

We wish to build our "SDK" (read: kernel image using specific repos and tags) programmatically. Is there a way of either removing the login dependency for sdk.update or to login as a machine user?

I think our pipeline would be (on change of toolchain or dockerfile):

  1. Update docker image (as needed, new toolchain)

(on change of sl_workspace_settings)

  1. Using the docker image, sl sdk.update
  2. Publish kernel artifacts for app developers

(interactive kernel development) a. Use the docker image, update, iterate, commit to sub-repos & change sl_workspace_settings as needed) b. Run native, no docker, trigger a new kernel build when finished

tyler-gilbert commented 4 years ago

You can use sl sdk.update --offline to prevent the login requirement. Also, the latest version of sl which is 0.68 or so only requires a commands that use the cloud features. I switched the update mechanism recently so you need to do the curl command (see Getting Started).

With that version, I also added support for multiple SDKs installed on the same machine. You can use the environment variable (or CMAKE variable) SOS_SDK_PATH to specify. You can see how that works with the HelloWorld CMake file.

syntroniks commented 4 years ago

I think that resolves our issues!

For reference:

# escape=`

# To build run this command from this folder: docker build -t stratify-sdk-windows -f windows.dockerfile .

FROM mcr.microsoft.com/windows/servercore:ltsc2019

SHELL ["powershell","-command"]

WORKDIR /StratifyLabs-SDK

# Add and extract the configured stratify sdk
ADD ./windows/sdk_windows_Tools-2019-3-26.exe .
RUN ./sdk_windows_Tools-2019-3-26.exe

# add some tools to the path (so cmake can find mingw32-make.exe)
RUN setx /M PATH $($Env:PATH + ';C:\StratifyLabs-SDK\Tools\gcc\bin')

# Add requirements for building the kernel
ADD sl_workspace_settings.json .
# Add the sl binary
ADD ./windows/sl.exe .

# Get choco! 🍫
RUN Set-ExecutionPolicy Bypass -Scope Process -Force;` 
    [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072;`
    iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

# Get git and cmake
RUN choco install -y git.install
RUN choco install -y cmake --installargs 'ADD_CMAKE_TO_PATH=System'

# now build it
RUN ./sl.exe sdk.update --offline
tyler-gilbert commented 4 years ago

Very nice. Thanks!