search

fsprojects / Paket

A dependency manager for .NET with support for NuGet packages and Git repositories.
https://fsprojects.github.io/Paket/
MIT License
2.02k stars 525 forks source link

[Testing] Set up a set of test nuget repositories to test against #2511

Open baronfel opened 7 years ago

baronfel commented 7 years ago

Description

We should provide a docker-compose file that stands up, at a minimum,

with packages in some predetermined format, that the tests could execute against. This would help prevent regressions like have shown up recently.

Useful Links: https://hub.docker.com/r/athieriot/docker-klondike/ https://hub.docker.com/r/sonatype/nexus/ https://hub.docker.com/r/sonatype/nexus3/

They could all point at the same test package store, and then it'd just be a matter of docker-compose up -d && ./build.sh UrlTests or something like that.

We wouldn't have myget out of the box like this, or nuget proper, but what do you all think?

forki commented 7 years ago

❤️

baronfel commented 7 years ago

Sounds good then. Setting up the configuration for those two at least shouldn't be too bad, but the larger question will be where we want to store the backing test data sets (package versions and all that jazz). I'd argue that ~100 versions of a package isn't exactly viable to store in this repo :D

forki commented 7 years ago

100 versions of a package - make them empty. shouldn't hurt that much

matthid commented 7 years ago

I think we could generate most packages from within the tests and upload them. Just how https://github.com/TheAngryByrd/simplenugetserverpaketrepro is doing it.

I started marking interesting issues with the https://github.com/fsprojects/Paket/issues?q=is%3Aissue+is%3Aopen+label%3Aneeds-test-case label once we have the necessary infrastructure in place we can add tests for those...

baronfel commented 7 years ago

@matthid I'm slowly working on this, on this branch: https://github.com/baronfel/Paket/tree/local_hosted_feeds

Right now I've got three feeds set up via docker-compose and at some point I've gotta prepopulate them :D

You can test/replicate by running docker-compose up -d in the /tests/feeds directory

matthid commented 7 years ago

you are targeting travis I guess?

cdrnet commented 7 years ago

I wonder whether we could also include Artifactory, which sadly also has its own quirks (and doesn't work with Paket v5 for us anymore at this point - unfortunately I did not have time to properly analyze what exactly is broken, report and propose a fix yet; we had to downgraded to v4.8.8 temporarily).

forki commented 7 years ago

@cdrnet is there a public Artifactory around? with a repro we could take a look...

cdrnet commented 7 years ago

@forki they do offer hosted instances, I'll have a look whether I can find one (which we would be allowed to use). Maybe we could even apply for their free OSS SaaS offering. There are also docker images available.

forki commented 7 years ago

repro on docker image would be fine

0x53A commented 7 years ago

Gitlab CI is based on docker (and is free), so that may be an easiysh route to add coverage.

Afaik travis has relatively hard disk limits, so running docker from inside Travis would probably fail.

I've played a little with it and

a) Gitlab has an automatic sync feature, so we would only need to set it up once and could then continue everything on Github. The mirrored repo is here: https://gitlab.com/paket-mirror/Paket b) CI is also based on a yaml file. Setting it up was easy. Green build including the integration suite: https://gitlab.com/0x53A/Paket/-/jobs/33122580

forki commented 7 years ago

Sounds good. I'm a big gitlab fan. Is it possible to get it hooked into pull requests?

Am 19.09.2017 18:58 schrieb "Lukas Rieger" notifications@github.com:

Gitlab CI is based on docker (and is free), so that may be an easiysh route to add coverage.

Afaik travis has relatively hard disk limits, so running docker from inside Travis would probably fail.

I've played a little with it and

a) Gitlab has an automatic sync feature, so we would only need to set it up once and could then continue everything on Github. The mirrored repo is here: https://gitlab.com/paket-mirror/Paket b) CI is also based on a yaml file. Setting it up was easy. Green build including the integration suite: https://gitlab.com/0x53A/ Paket/-/jobs/33122580

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fsprojects/Paket/issues/2511#issuecomment-330604441, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNCyAoFAF29iUuPCSP2QY7ugHZgAmks5sj_K7gaJpZM4OUQLS .

0x53A commented 7 years ago

Is it possible to get it hooked into pull requests?

I doubt it.

It would run after each commit on all branches, so if you or matthid work directly on a branch in this repo, it would trigger, but not for external pullrequests.

There seems to be something similar as a dedicated CI with circle-ci, I think @matthid experimented with it?

But they seem to only allow 1 container for free accounts: image

As I understand it, we would need to run multiple containers in parallel (one for the base, then one for each repository)? This doesn't seem possible with a free circle-ci account.

Reading the gitlab help on services seems to cover exactly this scenario.

matthid commented 7 years ago

In fact I tried out Circle-CI and it is really really nice. In pricing Circle-CI has:

How do I get started?

Linux plans start with the ability to run one build simultaneously without parallelism at no charge. Open source projects get 3 additional free containers, so you can use the power of parallelism and/or run more than one build concurrently.

Also, I'm not sure how pricing in fact counts multiple "images"/"container" in a single build, which is what we actually need.

matthid commented 7 years ago

GitLab is only really viable if we can make PR integration work via hooks/hacks otherwise feedback is too late.

0x53A commented 7 years ago

a) I assume that circle-ci and gitlab would run the same script (minus maybe slightly different configuration in their yaml files), so if we get one to run, porting to the other should be straightforward. b) If circle-ci supports the multi-container scenario, great, if not, having gitlab-ci after commit would be better than nothing.

As to hacks, it would probably be possible to write a bot that monitors github PRs, and commits to branches in gitlab (e.g. PR 12345 here in github would be directly commited to branch prs/12345 in gitlab and auto-synced).

Or we could move the paket development off github to gitlab ;-)

matthid commented 7 years ago

@0x53A Obviously I agree with both points. But I'd still argue that "after" doesn't really fit with our workflow (at least not how I'm using PRs, can't speak for @forki but I'd assume from observation he is working very similar). So it would not really prevent "bad" releases which is the important point here.

The only thing which might work is if we create a release branch merge stuff there and have GitLab pick that up. Than I think gitlab status might even show up in the PR for this release branch.