search

duojs / duo

A next-generation package manager for the front-end
3.42k stars 118 forks source link

caching a list of packages internally, to prevent duplicating work #492

Closed dominicbarnes closed 8 years ago

dominicbarnes commented 8 years ago

It turns out that in complex projects, it's not hard to hit the API limit. I've discovered that packages are being resolved over and over again.

Since they are kicked off asynchronously, there's not an indication that the given package is already being resolved. (particularly if there are multiple pkg objects referencing the same package)

This changes it so that they same pkg object will be used in cases where the slug is identical. I'll also be adding changes to duo-package so that it also knows when the package is already in flight. (similar to how it checks before downloading)

matthewmueller commented 8 years ago

Hm, I thought the inflight stuff in duojs/package handled this.

dominicbarnes commented 8 years ago

That only deals with the tar download. Resolve had no caching in this regard.

dominicbarnes commented 8 years ago

I had to put out a fire, since the number of API requests was exceeding the rate-limit because of this issue. But I can explain more of the rationale this evening. :)

matthewmueller commented 8 years ago

Ahh gotcha no that makes sense, since we're pinging the API now for resolves.