Issue with long load times for initiating the Flickr API

[Kinetic-Pulse]

Hey There, I read in your read me that when you start up the app it can take quite a bit of time to start up and that it doesn't matter after the startup. I am developing an App on Meteor and have integrated your package into the app. It works well, but the issue is every time i start up Meteor it takes around 3 or 4 min for the flickr API to start up after which pictures load in less than a second. I would like to ask is this something that the rest of the uses also face? And if you deploy the app on a live server will it then be be running always without that initial loadup?

[Pomax]

that's not supposed to happen. Does Meteor "wipe" the data directory that the flickrAPI tries to write to? Because if it exists and the data is still there, it should take maybe a few seconds to load at most...

[benlowry]

I have a fork that I've been optimizing for fast startup time and reduced dependencies:

https://github.com/benlowry/node-flickrapi/tree/master/src

'build-api.js' created 'built-api.json' from the data then 'flickr-api-object.js' gets everything from that. The first load is about 0 seconds and each reload is about 0 seconds.

I don't see the point in continuing to download those files on each npm install - they never get updated after that so it's effectively just a really inefficient way of bundling a static copy .

I think the dependencies have some low hanging fruits too - I'm currently working on replacing request.

1) async isn't used by anything

2) glob is used once to look for files with a key in their name

3) open is used once and just spawns a process

4) request is used once and bundles a ton more depedencies

[benlowry]

PS. For me the biggest strike against your method of loading the API is modern developer tools watch our code for changes and reload our apps as we make changes... your module is both a constant bottleneck during development and the slowest part of my project to load.

[benlowry]

Here's a WIP alternative for 'request', there was more dependency on it than I thought but still only GET, POST, and GET + write images:

https://github.com/benlowry/node-flickrapi/blob/remove-dependencies/src/request.js

I also decided to do some benchmarks of what I've been working on cause for my needs it's largely complete now:

require('dotenv').config()
const config = {
    api_key: process.env.FLICKR_API_KEY,
    secret: process.env.FLICKR_SECRET_KEY
  }

console.time('flickrapi-start')
require('flickrapi').tokenOnly(config, function () {
   console.timeEnd('flickrapi-start')
    return process.exit(0)
})

@Kinetic-Pulse this is the combined effect of ~60 fewer dependencies and consolidating all the JSON into a single file that is bundled with the module:

flickrapi-start: 168.509ms vs 1350778.591ms
flickrapi-start: 173.079ms vs 904.563ms
flickrapi-start: 171.009ms vs 889.867ms
flickrapi-start: 171.247ms vs 898.451ms
flickrapi-start: 171.778ms vs 922.448ms
flickrapi-start: 169.674ms vs 919.577ms
flickrapi-start: 169.942ms vs 896.615ms
flickrapi-start: 171.198ms vs 898.636ms
flickrapi-start: 171.483ms vs 881.462ms
flickrapi-start: 173.541ms vs 886.090ms

[Pomax]

Nice. I'm no longer actively working on this code but if you think the changes are PR-able I'll be happy to merge them in and spin a new major release

[benlowry]

This is now pull request #99.

[karlpokus]

@Pomax Is this PR happening?

[Pomax]

@karlpokus I am not finding a lot of time, but if you want to review this to see if it works as expected I'm willing to merge it in without having reviewed it myself.

[karlpokus]

@Pomax Sorry. I don't have the time either. I wouldn't wanna break anything. I found another module that don't require auth that suits my needs better. Best of luck!

[Pomax]

nice, which one?

[karlpokus]

Well, I started with this and ended up with this. This way you only need to auth the app (with the api key) not auth by user.