misc CI (excluding flaky tests)

timotheecour / Nim

Nim is a compiled, garbage-collected systems programming language with a design that focuses on efficiency, expressiveness, and elegance (in that order of priority).

http://nim-lang.org/

Other

2 stars 0 forks source link

misc CI (excluding flaky tests) #98

Open timotheecour opened 4 years ago

timotheecour commented 4 years ago

[ ] need regression test for cross compilation refs: https://github.com/nim-lang/Nim/pull/13907#issuecomment-612028458

timotheecour commented 4 years ago

[ ] wrapper module files are not tested in CI

As a side note, I wanted to add some unit tests for this but apparently wrapper module files are not tested in CI :( Fixes issues with dynamic loading OpenSSL. Fixes #13903. by dom96 · Pull Request #13919 · nim-lang/Nim : Fixes issues with dynamic loading OpenSSL. Fixes #13903. by dom96 · Pull Request #13919 · nim-lang/Nim

timotheecour commented 4 years ago

[x] test CI important_packages on OSX as well
[ ] test CI important_packages on windows as well
refs: Test packages on Linux by narimiran · Pull Request #13921 · nim-lang/Nim : Test packages on Linux by narimiran · Pull Request #13921 · nim-lang/Nim
azure-pipelines: test packages on more configuration by alaviss · Pull Request #13348 · nim-lang/Nim : azure-pipelines: test packages on more configuration by alaviss · Pull Request #13348 · nim-lang/Nim
[ ] run other pipelines (eg that would take too much resources if run each time) on some schedule ; eg w some probability or using https://docs.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml

=> see https://github.com/nim-lang/Nim/pull/13954#issue-402180448

timotheecour commented 4 years ago

=> moved to https://github.com/timotheecour/Nim/issues/167

timotheecour commented 4 years ago

bisect bot triggered on github issues that contains a keyword + runnable code

see: automated bisection for github issues · Issue #8 · juancarlospaco/nimbug : automated bisection for github issues · Issue #8 · juancarlospaco/nimbug

links

https://help.github.com/en/actions/reference/events-that-trigger-workflows#issue-comment-event-issue_comment

on:
  issue_comment:
    types: [created, deleted]

on:
  issues:
    types: [opened, edited, milestoned]

name: deploy_to_channel_on_demand

on: issue_comment

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: logging
        run: echo "$GITHUB_REF"

links

https://gitgitgadget.github.io/

timotheecour commented 4 years ago

avoid rebuilding csources => use choosenim or docker image or some fixed release tarball instead?

timotheecour commented 4 years ago

32bit linux VM: can't we use docker for that?

timotheecour commented 4 years ago

docker based nim CI's eg https://github.com/nitely/nim-regex/blob/e6efb506662721656567f0dab236b1ea052d518f/.travis.yml#L2

services:
  - docker
env:
  - NIM=1.0.4
  - NIM=1.2.0
before_install:
  - docker pull nimlang/nim:$NIM
script:
  - docker run --rm -v `pwd`:/usr/src/app -w /usr/src/app nimlang/nim:$NIM /bin/bash -c "nimble install -y; nimble test"

timotheecour commented 4 years ago

important_pacakges: auto-detect whether to set hasDeps=true or false depending on whether there are any requires by parsing nimble file. (see https://github.com/nim-lang/Nim/pull/13984#issue-403474485)

timotheecour commented 4 years ago

cache builds/files with s3cmd, eg: https://github.com/ziglang/zig/blob/480deacbab9c86d3cff32e59665e9f70b45e453d/ci/azure/macos_script#L78

links

https://github.com/MicrosoftDocs/azure-devops-docs/blob/master/docs/pipelines/tasks/utility/download-secure-file.md DOWNLOADSECUREFILE_SECUREFILEPATH https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/download-secure-file?view=azure-devops https://github.com/ziglang/zig/blob/480deacbab9c86d3cff32e59665e9f70b45e453d/ci/azure/macos_script#L77

timotheecour commented 4 years ago

does this run with cpp only in NIM_COMPILE_TO_CPP? etc

discard """
  target: "c cpp js"
"""

timotheecour commented 4 years ago

implement proc coverage, simpler than line or branch coverage; useful to see what's missing (or accidentally missing) in tests

timotheecour commented 4 years ago

skip parts of CI based on commit msg

[Skip build if there's [skip ci] in the commit message. · Issue #858 · microsoft/azure-pipelines-agent](https://github.com/microsoft/azure-pipelines-agent/issues/858)

s there a way to get the git commit message so I can use it as condition?
This one is not working:
not(contains(variables['Build.SourceVersionMessage'], '[skip ci]'))

Build.SourceVersionMessage returns some commit id.

Are you sure? My condition work just fine.
See predefined variables where you can read on Build.SourceVersionMessage.

docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml

timotheecour commented 4 years ago

[ ] D20200424T205650 automatically test for module imports for different targets, eg js, nims

links:

https://github.com/nim-lang/Nim/pull/14095/files#r414969432

glob approach, similar to what I did in https://github.com/nim-lang/Nim/pull/13221 ; this could work by adding a test inside tests/trunner.nim (fmt"{nim} js -r {autogenerated}") that would populate the list of all modules (via glob), and use a blacklist for things that don't work (instead of what you're doing which is a whitelist).

The point of a blacklist (instead of whitelist) is to keep working even if modules are added, removed, or moved.

Note that the same code could be reused (with a different blacklist exclusion) for other targets and/or compiler options, eg nimscript, --gc:arg etc

kochdocs: use a glob instead of hardcoded list; generate docs for compiler/; bugfixes by timotheecour · Pull Request #13221 · nim-lang/Nim

timotheecour commented 4 years ago

[ ] need to test VCC windows in CI

IIRC, ABI checks failed for VCC!

links

https://github.com/nim-lang/Nim/pull/14149