CI configuration

[evdokim]

Hi @alberskib ! We don't use any CI for our Scala projects, but we have all configuration of release process in nice-sbt-settings. So it would be easy to use any of CI server. Only one technical issue that I see here is the AWS credentials for publishing artifacts to S3. Indeed we have to grant to CI server write permissions to the buckets that we are using for release. But in same time we don't want to make these credentials publicly available. Today I've added an experimental support for Travis for one of our repository https://github.com/ohnosequences/aws-scala-tools and it works fine (travis page - https://travis-ci.org/ohnosequences/aws-scala-tools). AWS credentiasl for publishing were provided by environment variables (AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY). It seams to me as a secure solution? but honestly I haven't read carefully http://docs.travis-ci.com/user/encryption-keys/ so probably I missed something important. Also we have to decide which AWS account we will use for this project.

[alberskib]

Hello @evdokim, @bio4j/dynamograph Ok it is really good idea just to use CI in order to publish artifacts but I also believe that great application of CI server is just to running unit tests after each commit. Thanks to it we will have higher probability that commit does not break anything

[alberskib]

When it comes to security: Generally it is secure but I see one potential vulnerability. Take a look what is encrypted: 'NAME=VALUE' so assuming that someone know that first X characters of encrypted data is encryption of known NAME it will be easier to decrypt (but still difficult). Generally when we assume that potantial intruder does not know Name of the variable it is secure

[evdokim]

My point is to use the same release process in CI as we are using locally, it includes building, running test and publishing artifacts, probably with minor changes with versioning. For this we don't need to configure anything and aren't biased with some particular CI.

[alberskib]

There is one additional thing: Travis contains hook that will invoke build process after each pull to master or PR. Let's imagine situation that someone create some pull request after such creation the whole travis process starts. Next code review reveal that there is some bugs and some code should be changed. Adding next commit to PR will trigger next travis build. Presented travis configuration deploys package to repository every time it is runned. As a result of described actions repository will contain at least 2 packages.

[laughedelic]

@alberskib I'm not sure that I understand exactly what's going on here, but regarding your last concern, if there is a bug, the project won't compile and therefore won't be published. If there are no bugs, fine — a snapshot will be published (and it will be overwritten in the repository).

[alberskib]

@laughedelic Ok now I see. I do not know that running publish command will result in publishing snapshot version - I was thinking that each publish will result in seperate version (non snapshot version). As I understand it I think everything is great. Thanks a lot for explanation.

[laughedelic]

@alberskib You're welcome. I copied for you a part of the docs wiki related to coding. Take a look at it, especially the sbt page and don't hesitate to ask anything :wink:

[alberskib]

@laughedelic .gitignore file contains credentials.sbt file - I assume that there should be placed "access Key Id" as well as "Secret access key" for releasing on http://releases.era7.com.s3.amazonaws.com. Could you send me template for this file

[laughedelic]

@alberskib nope, it shouldn't be in .gitignore (it was needed before, but not now). Credentials for publishing should be stored in ~/.sbt/.s3credentials. See format here

[evdokim]

@alberskib as I mentioned it before Travis support encoding for environmental variables, so you can set AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY instead using credentials.sbt

[laughedelic]

@alberskib @evdokim is right, you can use environment variables instead (again see the link). And again, no any credentials.sbt, just to be clear — it's deprecated.

[alberskib]

@evdokim @laughedelic Great, thanks for clearing it up.

[evdokim]

@laughedelic

it's deprecated. But what the alternative now for local publishing?

[laughedelic]

@evdokim for "local publishing"? (you don't need credentials) or for setting credentials in a file? ~/.sbt/.s3credentials (here)

[evdokim]

I mean not at the EC2 instance, Travis server etc

[laughedelic]

@evdokim I don't understand you "/

[evdokim]

ok from other side: local machine is you laptop, all others instances aren't

[laughedelic]

I don't understand what you want, let's move this to the chat:

[alberskib]

Is there any doc generator for project? Actually docs for scarp are outdated after commit. If no I will update docs for it

[laughedelic]

Well, "docs" there are generated just from the sources. You can generate them with generateDocs in sbt (and then commit).

[alberskib]

Invocation of gtm connect result in error:

No commits between ohnosequences:master and ohnosequences:docs/docs-update
Couldn't transform issue to pull-request. Either it's already a pull-request, or you didn't push anything yet. Try gtm push.

[laughedelic]

Yes, it's ok. It's like that, because you cannot create a pull-request from a branch, that doesn't have any difference with master. So this is the usual workflow when you want to create a branch+issue:

1. create a new branch and an issue on github with the given name

   gtm task feature/foo/bar -m "Developing a new cool feature"

   or if you already have an issue for that

   gtm task feature/foo/bar -i 5   # (issue number)

2. then you do some stuff in this branch (i.e. at least one commit) and push it

   gtm start
   # ... then you work on the feature
   git add <some files>
   git commit -am "Worked on the foo-bar feature"  # this will record the time you spend, you don't need to do "gtm stop"
   gtm push

   here gtm push will do three things:

   • push timetracker notes
   • push this branch
   • make a pull request from the corresponding issue (without creating a new issue) (this what gtm connect does, so you don't need to use connect, use just push)

   then any commit to this branch will be show in the corresponding pull-request. You can see all such branch+issue bindings with gtm list.

3. When you are done, you just do

   gtm close

   it will say you what it does (basically, just merge you branch, push everything (closing the pull-request) and archive the branch)

P.S. don't hesitate to ask about gtm and open issues, because it's a hacky tool, it can be non-intuitive sometimes (and I don't notice that, so I'm thankful if you point it out) and I know that it lacks documentation... :wink: