Closed mef closed 7 years ago
Hey @mef . I've started with the implementation. Here is what i'm doing
{hashtag: counts}
This is happening in the beggining. My idea was to send the sorted objects to the client and on a new tweet, update that array. I dont know if this is efficient.
a: 5
b: 4
c: 3
Lets say a new tag b
arrives. Loop through all hashtags, find property b
and ++ the counter, resort the array objects . If we don't find the property b just push it to the end.
Do you think that this is efficient to be done on the client side? Should we do the calculation on the server, everytime there is a new tweet? What do you think?
Here are some suggestions:
entitiesStats
or entitiesCount
, instea of topStats
eventTopStats
accordingly.entitiesCount.hashtags = {
belgium: 55
, toto: 32
...
}
topEntities = {
hashtags: [
// key,value again, only for the top n.
]
, lowestHashtagCountCount: 10
...
}
metrics
controller, to be used at both sides to update the stats.updateTweetStats
call, for some reason it has been commented out in the previous commit.Hope this helps,
Thanks a lot. The reason for the array of arrays, was that I couldn't sort the object. Do you have any ideas for that?
You are going to have a very long tail of hashtags that are here only once. This is why you want to avoid to have to iterate through this array, and also you don't want to sort it.
I'll give it a try, thanks
You don't need to sort the object. Inside a reduce function, just identify whether the element's count is higher than previous elements, and keep the top n records.
On Tue, Sep 20, 2016 at 2:19 PM, 11 Digit Labs notifications@github.com wrote:
Thanks a lot. The reason for the array of arrays, was that I couldn't sort the object. Do you have any ideas for that?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Mango-information-systems/twitto_be/issues/100#issuecomment-248298925, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4fAzk2tFDdUAwXzQ45MiZWmap63WIzks5qr916gaJpZM4J_X55 .
Does not seem to be working with an empty database:
/home/c-user/development/twitto_be/model/tweets.js:197
return {'top': arr, 'all': obj, 'lowestCount': arr[slice-1].value}
^
TypeError: Cannot read property 'value' of undefined
at orderSliceObject (/home/c-user/development/twitto_be/model/tweets.js:197:62)
at calculateEntitiesStats (/home/c-user/development/twitto_be/model/tweets.js:136:14)
at new Tweets (/home/c-user/development/twitto_be/model/tweets.js:37:2)
at new Storage (/home/c-user/development/twitto_be/model/index.js:24:16)
at Object.<anonymous> (/home/c-user/development/twitto_be/app.js:18:16)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Function.Module.runMain (module.js:441:10)
[nodemon] app crashed - waiting for file changes before starting...
Just like @mef suggested, no arrays, no sorting and slicing \m/ (still WIP!)
2.2.0-pre
has just been deployed in the production server, data will be usable in 24h :smile: .
@11digitlabs I think I've fixed the various small things that remained, please have a look.
Apart from that, I thought the refactoring removing the sort of all hashtags, using reduce to pick the top 10, had been done, but it looks like it's not in place. Has it been done in some other branch? Or perhaps I misunderstood?
Nope, no misunderstanding. Seems like I didnt do it, if you want, please do the reduce, you JS master :) Will check the rest later tonight
What do you think of this alternative presentation?
Trending hashtags / mentions is more relevant to users, so pushing it to the top seems like a good move to me:
Makes more sense, plus its like grouped by color :) Grey ones on top Green ones at the bottom
Should we make the hastags/mentions clickable too?
A bug I just noticed, couldnt resist to download the latest version, im a junkie ;)
A bug I just noticed, couldnt resist to download the latest version, im a junkie ;)
Yes, reported in #101
@11digitlabs the features looks ready for deployment to me. What do you think?
Alright, lets go for it. Seems to be working/looking fine
Display a set of metrics / charts related to the tweets:
Remaining from #85