googlearchive / flashlight

A pluggable integration with ElasticSearch to provide advanced content searches in Firebase.
http://firebase.github.io/flashlight/
756 stars 158 forks source link

Adding a refBuilder hangs flashlight #129

Open tallminator opened 7 years ago

tallminator commented 7 years ago

Before refBuilder: Path: exports.paths = [{ path: "nuggets", index: "firebase", type: "nugget", filter: function(data){ return (data.hasOwnProperty('published') && (data.published == true))||(!data.hasOwnProperty('published')); } },{ path: "tags", index: "firebase", type: "tag" } ]; Result: image

After refBuilder: exports.paths = [{ path: "nuggets", index: "firebase", type: "nugget", refBuilder: function(ref, path) { return ref.orderByChild('createdAt').startAt(Date.now());}, filter: function(data){ return (data.hasOwnProperty('published') && (data.published == true))||(!data.hasOwnProperty('published')); } },{ path: "tags", index: "firebase", type: "tag" } ]; Result: image

katowulf commented 7 years ago

Would need a complete, minimal repro with version info to be any help here.

vcrepin commented 7 years ago

Same problem here. Using

  refBuilder: function(ref, path) {
     return ref.orderByChild('timestamp').startAt(Date.now());
  }

It does not index anything.

sanvir10 commented 7 years ago

I've the same problem, who can help?, i added indexOn on timestamp, but this don't work, thanx.

vcrepin commented 7 years ago

Put a try catch around the inner code of PathMonitor constructor to see the error in the console. The problem is there. The statement that blows is

console.log('Indexing %s/%s from DB "%s"'.grey, path.index, path.type, fbutil.pathName(this.ref));

because of fbutil.pathName().

refBuilder creates a wrong ref for a reason that I do not understand. Seems a race condition because adding some console.log here and there solved the problem for me.

sanvir10 commented 7 years ago

Can you share your PathMonitor?, i put console.log

function PathMonitor(esc, path) { console.log("PathMonitor: "+JSON.stringify(path)); ...

In the config i've: ... { path : "users", index: "prototipo-flash", type : "user", refBuilder: function(ref, path) { return ref.orderBy("timestamp").startAt(Date.now()); } ...

But in the console i only see: {"path":"users","index":"prototipo-flash","type":"user"} <- this don't contain refBuilder.

vcrepin commented 7 years ago

JSON.stringify does not show functions.

Add console.log() for the ref before and after is has been replaced by refBuilder.

In my case the replacement gave a wrong ref. But after adding other console.logs, it started to give the right one. In case, console.log(ref) should be the same thing before and after the replacement.

console.log(ref) gives the firebase path to your document.

vcrepin commented 7 years ago

Here is my path monitorconstructor:

function PathMonitor(esc, path) {

this.ref = fbutil.fbRef(path.path);
console.log("Created REF: "+this.ref);

if(fbutil.isFunction(path.refBuilder)) {
    this.ref = path.refBuilder(this.ref, path);
    console.log("Changed REF TO REFBUILDER: "+this.ref);
}
console.log('Indexing %s/%s from DB "%s"'.grey, path.index, path.type, fbutil.pathName(this.ref));

this.esc = esc;

this.index = path.index; this.type = path.type; this.filter = path.filter || function() { return true; }; this.parse = path.parser || function(data) { return parseKeys(data, path.fields, path.omit) };

this._init(); }

vcrepin commented 7 years ago

By the way in fbutils you should adjust pathName like so:

exports.pathName = function(ref) { var p = ref.parent; return (p ? p.key+'/' : '')+ref.key; };

sanvir10 commented 7 years ago

Thanx, but the result is not indexed u.u:

` Connecting to ElasticSearch host 89b1ead13acc93f9402a2f3107f7exxx.xx-xxxx-1.xxx.found.io:9200 Connected to ElasticSearch host 89b1ead13acc93f9402a2f3107f7exxx.xx-xxxx-1.xxx.found.io:9200 Connecting to Firebase https://xxxxxx-9cc16.firebaseio.com Created REF: https://xxxxx-9cc16.firebaseio.com/users

`

I did the changes but not work yet, i will review later thanx.

vcrepin commented 7 years ago

print your ref once replaced by refBuilder. If should be the same as before ( https://flashligth-9cc16.firebaseio.com/users)