search

promises-aplus / promises-tests

Compliances tests for Promises/A+
Other
943 stars 107 forks source link

Benchmarks #35

Open ZJONSSON opened 11 years ago

ZJONSSON commented 11 years ago

Perhaps this is not the right place for this, but I think centralized benchmarks (using the promises-tests adaptors) would be a great addition under promises-aplus, given general lack of proper benchmarks in the wild (see for example https://github.com/thanpolas/perf-promises/pull/2)

stefanpenner commented 11 years ago

+1

johnyanarella commented 11 years ago

:thumbsup:

briancavalier commented 11 years ago

I do think a generalized, and properly reviewed, benchmarking tool for promises would be a good thing. As we've seen, if you create a benchmark, people will try to win at it, which means that people will innovate to make promises faster.

That said, I'm not sure that a standards organization such as Promises/A+ is the right place to house such a tool. I bet members would be willing to contribute, review, etc., tho. I'd certainly be willing, but I can't dedicate a large amount of time to it.

domenic commented 11 years ago

Yeah, I agree with @briancavalier I think. Promises/A+ community members will almost certainly be involved in this effort, which I am probably going to start soon, but it's not something we'd like to necessarily "endorse" as an organization.

ForbesLindesay commented 11 years ago

I actually started something today but realized so far it's no better than anything that already exists. I think the difficulty is that we keep creating really tiny micro benchmarks. What we need is to come up with something that's a genuinely realistic approximation of how promises are used.

I'm sure lots of us would be happy to collaborate on something like this. Personally I'd be happy to see it as part of the Promises/A+ organisation.

ForbesLindesay commented 11 years ago

I'd really love to see a benchmark created by someone who doesn't also maintain a promise library. People have a tenancy to "win" their own benchmarks. Alternatively someone like @kriskowal could make one since Q is unlikely to win on speed as that's not its priority.

briancavalier commented 11 years ago

I agree with @ForbesLindesay that a benchmark tool shouldn't be driven primarily by one library's owner(s), and ultimately, it needs to be developed in the open.

gobwas commented 9 years ago

I think we still need this benchmarks... Any news? :sunglasses: