search

OrkoHunter / depends

[Unmaintained] Analyze dependencies of Python libraries
13 stars 6 forks source link

How it this service so fast? #4

Closed laike9m closed 7 years ago

laike9m commented 7 years ago

Hi Mishra, I tried the dependency analysis service and is amazed by its speed. I viewed your source code and found you did have cache for previous results. Then I tried some libs which I'm pretty sure nobody has searched using your service before, and it returns results just as fast.

So I'm just curious if you fetch all the packages from pypi periodically.

OrkoHunter commented 7 years ago

@laike9m Hi ! Thanks for trying it out. The cache is developed by new queries only and are not fetched by any cronjob or any other bot on the server. So if any one runs the service for a package once, then only it will be saved. You can see the difference between the speeds by querying a rare library two times. Rest of the speed comes from the script, which I'm pretty sure can be improved for better.

laike9m commented 7 years ago

Good to hear your response. So downloading and iterating over files are faster than I thought.