search

veliovgroup / jazeee-meteor-spiderable

Fork of Meteor Spiderable with longer timeout, caching, better server handling
https://atmospherejs.com/jazeee/spiderable-longer-timeout
33 stars 9 forks source link

v1.2.4 #11

Closed dr-dimitru closed 9 years ago

dr-dimitru commented 9 years ago
dr-dimitru commented 9 years ago

@jazeee I've added test app and few new features Also i thought to rewrite package into Spiderable class, but it will bring major changes, so to start package developer will need to run something like:

spiderable = new Spiderable
   cacheLifetimeInMinutes: 120
   debug: true
   ignoredRoutes: ['a', 'b', 'c']
   userAgentRegExps: [/^vkShare/i]

This will bring more structure to the code. Let me know what do you think.

jazeee commented 9 years ago

You accidentally included a symlink to your home directory in packages. It should be a relative link to the parent directory.

However, it does test correctly, with a 404. The main difference is that you are using the Session.set, which I will test further tomorrow.

dr-dimitru commented 9 years ago

But this session is not used anywhere, I've put this into code just for example

jazeee commented 9 years ago

@dr-dimitru - Regarding Spiderable class, I am unsure about this. It seems like we could have a Spiderable class within our code, however, we should not require users to create the object. I believe we should stick with a configuration object that centralizes the various Spiderable parameters.

In addition, it will break existing users, since they are reliant on the existing configuration.