search

minio / mint

Collection of tests to detect overall correctness of MinIO server.
Apache License 2.0
78 stars 50 forks source link

|Proposal] How to avoid travis mint occasional failures #294

Closed vadmeste closed 5 years ago

vadmeste commented 5 years ago

Current problem:

6 mint SDKs/tools issue API calls to api.github.com to fetch package latest release information. However Github has a rate limit of unauthenticated requests: 60 requests per hour, which is little low especially if we push the same PR multiple times.

Possible solutions:

  1. Hard-coding latest release string in each SDK/tool so we can avoid calling api.github.com

  2. Checking out the source code of the SDK and running this git describe --abbrev=0 --tags to get the latest annotated tag, this will also avoid calling api.github.com

  3. A authenticated user is allowed to make 5000 requests per hour, so we can create a new user in Github like minio-guest, which doesn't have write access to anything, get its Github Token and then follow this documentation (http://blog.code4hire.com/2016/06/adding-github-token-to-travis-ci-configuration/)

vadmeste commented 5 years ago

This is an example of a travis failure, I made mc install script verbose to see better what's going on:

Step 29/52 : COPY build/mc /mint/build/mc
 ---> e9e8aa76354c
Step 30/52 : RUN build/mc/install.sh
 ---> Running in ac73c0e3ae7c
++ jq -r .tag_name
++ curl --retry 10 -s https://api.github.com/repos/minio/mc/releases/latest
+ MC_VERSION=null
+ '[' -z null ']'
+ test_run_dir=/mint/run/core/mc
+ wget --quiet --no-check-certificate --output-document=/mint/run/core/mc/mc https://dl.minio.io/client/mc/release/linux-amd64/mc.null
The command '/bin/sh -c build/mc/install.sh' returned a non-zero code: 8
nitisht commented 5 years ago

Checking out the source code of the SDK and running this git describe --abbrev=0 --tags to get the latest annotated tag, this will also avoid calling api.github.com

IMHO this is the best way as we already do git clone

harshavardhana commented 5 years ago
  • Hard-coding latest release string in each SDK/tool so we can avoid calling api.github.com

This is easier to do for each SDK, mc, minio releases we update the mint with a commit which updates the version for relevant tools etc. This will be automated from our release automation scripts for each sdk, mc and minio

Checking out the source code of the SDK and running this git describe --abbrev=0 --tags to get the latest annotated tag, this will also avoid calling api.github.com

This won't work for mc and any other binary downloads @nitisht

nitisht commented 5 years ago

fixed by https://github.com/minio/mint/pull/315 ?

harshavardhana commented 5 years ago

Yes most probably.

nitisht commented 5 years ago

Closing for now then, we can revisit if seen again