Global CI configs

[CJ-Wright]

Add a script for getting the global ci configs

[CJ-Wright]

This could be helpful for testing code against the NSLS-II stack from conda and from master. This in turn could help us when it comes time to release (as we hope that all the API breakage doesn't happen at once :smiling_imp: ). I don't like writing this in bash though so maybe xonsh?

[CJ-Wright]

We may also want a build which only uses the NSLS-II conda stack (to really check that our builds will pass muster).

[CJ-Wright]

So there are a couple cases here (on a per package basis):

1. Package only from conda (eg numpy)
2. Package only from master branch via pip (eg streams) (discouraged but necessary)
3. Package from pip or conda (either conda-forge or NSLS-II) (things where we may want to have cutting edge code but also test against the current standard).

Maybe we have three files per repo?

1. req.txt the conda only reqs
2. ce_req.txt for conda or cutting edge code
3. pip_req.txt for pip only code

[CJ-Wright]

@chiahaoliu @sbillinge

[CJ-Wright]

Continuing with the specification build out: For the reqs we just need a valid name on conda-forge/NSLS-II internal For the ce_reqs we need both the conda name and the repo url For the pip_reqs we just need the repo url

Note that for the NSLS-II conda stuff we need to install the main stack so we can find any dep collisions as they come up.

[CJ-Wright]

@danielballan

[sbillinge]

Btw, I don't have a strong opinion on this do go with what makes sense. On Thu, Aug 3, 2017 at 7:41 PM Christopher J. Wright < notifications@github.com> wrote:

@danielballan https://github.com/danielballan

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/xpdAcq/wishlist/issues/13#issuecomment-320118478, or mute the thread https://github.com/notifications/unsubscribe-auth/AEDrUfNlJ-iPiRggRYwkojnfkXMwwT76ks5sUlrBgaJpZM4OkKwy .

[danielballan]

Since you tagged me, I'll offer my opinion: keep it simple. You need to install at least one thing from pip, so I suggest you simply install everything from pip using a standard requirements.txt, which Travis-CI will recognize automatically.

The alternative is an overengineered CI scheme which others will have to maintain someday. I've been simplifying ours lately, and I like the results.

[CJ-Wright]

My only issues with that is a) some things we never get from pip (numpy), b) it means we will always test against bleeding edge code and sometimes we need to straddle released and bleeding edge. How do you handle these issues at NSLS-II?

[danielballan]

a) some things we never get from pip (numpy)

Why? You can get numpy from pip, and now that manylinux wheels exist it's as fast as conda. We do this for some bluesky CI builds.

b) it means we will always test against bleeding edge code and sometimes we need to straddle released and bleeding edge.

You can install latest stable releases using pip. I don't understand why you say using pip means always testing against bleeding-edge code.

[CJ-Wright]

Ah sorry, I forgot that I could pin pip github to tags (my understanding that some of the NSLS-II stack was not on pypi)

[danielballan]

Right, you'll have to pin databroker to a tag until we put it on PyPI. Bluesky and ophyd are now released on PyPI in the standard way.

[CJ-Wright]

I wish that there was a way for pip+github to got to the latest release

[danielballan]

That would be a GitHub thing. There might be a URL to access that latest tag of a repo.

[danielballan]

But don't burn much time on that -- databroker will be on PyPI soon. :- )