nim-lang / fusion

Fusion is for now an idea about how to grow Nim's ecosystem without the pain points of more traditional approaches.
MIT License
128 stars 16 forks source link

CI: don't build nim from scratch (can take majority of the CI time) #27

Closed timotheecour closed 3 years ago

timotheecour commented 4 years ago

there are better options, eg:

      - uses: jiro4989/setup-nim-action@v1
        with:
          nim-version: ${{ matrix.nimVersion }}

further, for devel (which changes on every nim commit), we should use caching; somehow it rebuilds nim even though there is caching in the yml file specified, I need to investigate further this point

alaviss commented 4 years ago

It tracks Nim's latest commits of each major branch. Due to the nature of devel it will have to be rebuilt all the time, but since csources is cached the process is very fast.

We use custom code here because we don't want to use choosenim for this.

alaviss commented 4 years ago

With recent changes to nightlies I might craft a simple fetcher to pull those instead, though when I crafted Fusion CI @Araq wanted to use a "traditional" Nim building approach, so here we are :P

timotheecour commented 4 years ago

here's what I'm getting: looks like nim-cache doesn't work, while csources-cache works:

Run actions/cache@v1
  with:
    path: nim
    key: nim-linux-amd64-dc1c251a2c2779f3f12d7bf2255f01355bbb886d
Cache not found for input keys: nim-linux-amd64-dc1c251a2c2779f3f12d7bf2255f01355bbb886d.
2s
Run actions/cache@v1
  with:
    path: csources/bin
    key: csources-linux-amd64-64e34778fa7e114b4afc753c7845dee250584167
Cache Size: ~2 MB (1972957 B)
/bin/tar -xz -f /home/runner/work/_temp/4239afe8-fc51-4b5c-ab1b-07ad1337c22e/cache.tgz -C /home/runner/work/fusion/fusion/csources/bin
Cache restored from key: csources-linux-amd64-64e34778fa7e114b4afc753c7845dee250584167

=> 64e34778fa7e114b4afc753c7845dee250584167 is indeed HEAD of csources repo, but dc1c251a2c2779f3f12d7bf2255f01355bbb886d doesn't look like a valid commit in nim repo, hence the cache miss

alaviss commented 4 years ago

As you can see in your own PR, the cache is working.

The issue here is that there hasn't been a commit to fusion since the last merge two days ago, so the cache wasn't updated with the recent backports/commits to the various nim branches.

Note that only commits to the main repo may update the cache of the main repo. PR cache is derived from the main repo and changes stays local to the PR.

timotheecour commented 4 years ago

maybe it's good enough currently and this could be closed, just want to check whether caching works as intended on next few PR's first, or whether a pre-built solution would lead to faster CI by reducing cache misses in practice. Caching somehow ended working after a few commits on my PR but not yet sure why.

wanted to use a "traditional" Nim building approach We use custom code here because we don't want to use choosenim for this.

if cache misses are an issue in practice even for nim point releases, it'd be worth considering doing what other packages are using (eg the setup-nim-action@v1 or choosenim approach; docker probably not an option as it wouldn't help with windows/osx); nim-devel could still use the traditional cached-csources + cached-nim build so that code path would still be tested.

juancarlospaco commented 3 years ago

Works:

- uses: jiro4989/setup-nim-action@v1
  with:
    nim-version: 'devel'
timotheecour commented 3 years ago

@juancarlospaco can you make a PR? if it improves CI time, it's great for turnaround