publish dependency testing scripts as module?

ssbc / ssb-server

The gossip and replication server for Secure Scuttlebutt - a distributed social network

1.69k stars 164 forks source link

publish dependency testing scripts as module? #650

Open dominictarr opened 5 years ago

dominictarr commented 5 years ago

@christianbundy I think we should publish the various scripts we have to test deps (scripts/pretest.js and test/deps.sh) as a module that makes this pattern easy to reuse.

I would do it but christian you wrote most of it so I'd rather see your name on it on npm

christianbundy commented 5 years ago

I think that's a great idea, mind if I add it under the SSBC org instead?

dominictarr commented 5 years ago

sure, go ahead

dominictarr commented 5 years ago

I needed this for testing flume, so I did it: https://github.com/ssbc/compatibility

christianbundy commented 5 years ago

Oh, I fixed up some things and was working on that in this branch: https://github.com/ssbc/ssb-server/tree/extract-pretest

I defaulted to testing all dependencies and was waiting until we could get pull-stream using compatible versions again.

dominictarr commented 5 years ago

@christianbundy oh I didn't realize. did you change the logic about how you check dependencies? I remember I said I wanted to preserve the history... but for simplicity's sake I just attributed the initial commit to you. (you can commit as anyone in git!) I'm wary of going down a testing rabbit hole... I used to have an testing based omega project...

I think we should set a concrete goal: getting someone who is maintaining an alternate ssb-server distribution (i.e. @mixmix ;) to run the tests as part of their release cycle. see https://github.com/ssbc/patchbay/pull/340

christianbundy commented 5 years ago

@dominictarr Yeah, by default this runs tests for all parent modules (e.g. patchbay) who have intersecting dependencies with child modules (e.g. ssb-server) so that you don't have to hardcode any deps you'd like to test. For example:

patchbay depends on ssb-foo and ssb-bar@2
ssb-foo has tests but only tests against ssb-bar@1
we want to test ssb-foo against ssb-bar@2 to make sure nothing breaks

I was thinking I'd likely add back the customization of an allowlist/blocklist, but I wanted to make sure I got all of the logic working correctly. I think we're on the same page, my stretch goal was just to make it so that no customization was needed (because it's so easy to forget to add stuff to a hardcoded list).

dominictarr commented 5 years ago

okay, but the problem I'm encountering is that your script was quite pedantic about ensuring that the extra devdeps didn't conflict. in many cases you have bumped a major version, but it doesn't mean it's actually broken a particular dep. what if it only added extra deps but didn't over ride deps the root module has hard coded?

stale[bot] commented 3 years ago

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?