search

lucasdicioccio / deptrack-project

monorepo for DepTrack
40 stars 5 forks source link

Provide devops code and example to deploy DO nodes and Spark cluster #1

Closed abailly closed 7 years ago

abailly commented 7 years ago
lucasdicioccio commented 7 years ago

Lovely and awesome. Thanks! The only thing I'm not fan of is the system-extra git submodule dependency on deptrack-devops-recipes. I'm open to have another subprojects like deptrack-devops-extra and have the examples depend on it (long term I'm wondering what's a good way to distribute and break apart new/fixes to recipes: I guess long term we can ask people to git-pull (and maybe git-submodule?) to vendor the recipes they need). Do you think it's a fine strategy?

I just got my DO account validated and I'll give it a try. I'm excited.

abailly commented 7 years ago

Honestly, I am not a big fan of submodules either : This is a transitory situation I created because I wanted to move fast, it should go away. Given the system-extra module does depend on standard packages, the simplest solution would be to integrate the Build.hs file in the deptrack source tree. If that suits you, I can update the PR along that line.

re. the general question of handling changes to recipes : Ideally, I would rather package them independently and let people depend on what they need exactly. That's one of the things I am not pleased with propellor: I am dragging along 100+ files for features I don't need with a dramatic effect on build time (and also impact on code editiing given I am using intero which also needs dependencies to be built). I like the approach taken by amazonka.

lucasdicioccio commented 7 years ago

ACK. This SGTM, please add another bullet in the README within "Using DepTrack" to run a git submodule update + a note that we'll likely remove this dependency later.

abailly commented 7 years ago

OK, will do that.

But then, let's do the Right Thing(tm) :) This package is not really a package anyway, just a bunch of code I reuse across various projects because I need some way to handle running processes and define docker configs more or less elegantly. Given you have already done some work to expose a docker API, that should fit nicely in.

lucasdicioccio commented 7 years ago

About splitting modules apart: I definitely agree that having a mega repository is too annoying. However I'm wondering how versioning will look like freezing snapshots on hackage periodically from small packages in a same git repo seems like the least worst idea.

abailly commented 7 years ago

Totally agree. I like having multiple packages but a single source tree (handling multiple source repos is a PITA as your remark on system-extra rightfully points to). The base process would be to release everything periodically at a single version but letting open the door to specific bug fixes/hot patches at the singular package level.

Years struggling with dependency managers gave me the feeling this problem space does not have any best solution, only a set of optimal solutions forming a pareto border depending on the constraints you are willing to relax/tighten :)