Open weppami opened 4 years ago
Thanks for the issue! We'll take your request into consideration and follow up if we decide to tackle this issue.
To our amazing contributors: issues labeled type: bug
are always up for grabs, but for feature requests, please wait until we add a ready for dev
before starting to work on it.
To claim an issue to work on, please leave a comment. If you've claimed the issue and need help, please ping @thepracticaldev/oss and we will follow up within 3 business days.
For full info on how to contribute, please check out our contributors guide.
I experienced the same, not sure what happened
Hi there, If this bug is not taken I would like to help
@Rafi993 go ahead!
Thank you @rhymes
Hi there, I have been caught up in other things lately. I'll just remove my self as assignee from this issue and comeback later if this issue is still there. Thank you @rhymes
Hello there, I'll take up this issue again if it is not solved yet.
Thanks @rhymes
I tried to reproduce this issue in local I could not. I tried with sees size 3 It is still not there. I am going to check the actual dev.to site to see if this issue still there and if it is issue in lazyloading or this issue in the data that reaches the front end.
My guess this is some issue in insertArticles function in app/assets/javascripts/initializers/initScrolling.js. I'll try to verify this.
I was wrong it issue in the data of the feed itself
When I hit url https://dev.to/search/feed_content?per_page=15&page=1&sort_by=hotness_score&sort_direction=desc&approved=&class_name=Article I get
and when I call the next page https://dev.to/search/feed_content?per_page=15&page=2&sort_by=hotness_score&sort_direction=desc&approved=&class_name=Article I get
between those two API calls article with title TC39er.us with Allen Wirfs-Brock!' repeats. I tried the API calls couple of times I was able to recreate this in the first two pages (this happens in current dev.to). I am yet to reproduce this in local.
Is this related to elastic search (app/services/search/query_builders/feed_content.rb)? @thepracticaldev/oss can you help me out here (Thanks in advance for taking your time to read my comments).
@Rafi993 The team name changed with the repo move, to ping us you now need to use @forem/oss :smiley:
For your actual question, I'll leave it to someone who is more familiar with the feed, maybe @joshpuetz can weigh in here?
I suspect this might be some sort of collision between the articles we pull back directly from the database/cache (it's the first 10-30 IIRC), and the rest of the feed which comes back from Elasticsearch. I'm unsure if we actually check to eliminate duplicates!
One idea would be to record the article ids serviced initially, and exclude them from showing in the sets retrieved from Elasticsearch
I suspect this might be some sort of collision between the articles we pull back directly from the database/cache (it's the first 10-30 IIRC), and the rest of the feed which comes back from Elasticsearch. I'm unsure if we actually check to eliminate duplicates!
One idea would be to record the article ids serviced initially, and exclude them from showing in the sets retrieved from Elasticsearch
I think you are right I couldn't find any code that checks to eliminate duplicates. So if the issue is due collision between database/ cache and elastic search then there shouldn't be any duplicates after first 30 articles right? (I'll try to verify this).
I think you are right I couldn't find any code that checks to eliminate duplicates. So if the issue is due collision between database/ cache and elastic search then there shouldn't be any duplicates after first 30 articles right? (I'll try to verify this).
Yes, fantastic point @Rafi993! Duplicates occurring only between the very beginning and later in the feed would indicate this is the problem!
I suspect this might be some sort of collision between the articles we pull back directly from the database/cache (it's the first 10-30 IIRC), and the rest of the feed which comes back from Elasticsearch. I'm unsure if we actually check to eliminate duplicates!
One idea would be to record the article ids serviced initially, and exclude them from showing in the sets retrieved from Elasticsearch
You were right the duplication occurs only within the first 30 entries. That is between url https://dev.to/search/feed_content?per_page=15&page=1&sort_by=hotness_score&sort_direction=desc&approved=&class_name=Article
and
after that there is no duplication
I suspect this might be some sort of collision between the articles we pull back directly from the database/cache (it's the first 10-30 IIRC), and the rest of the feed which comes back from Elasticsearch. I'm unsure if we actually check to eliminate duplicates! One idea would be to record the article ids serviced initially, and exclude them from showing in the sets retrieved from Elasticsearch
You were right the duplication occurs only within the first 30 entries. That is between url https://dev.to/search/feed_content?per_page=15&page=1&sort_by=hotness_score&sort_direction=desc&approved=&class_name=Article
and
after that there is no duplication
Weirdly enough this not happening in my local development environment with same API calls.
@Rafi993 are you still working on this? I know you're assigned to quite a few issues so it's understandable if you haven't gotten the time to get back to this. Let me know if you still want to work on this or else if you have new information you can leave in the issue here so to pass the torch if you decide so. Thank you!
@Rafi993 are you still working on this? I know you're assigned to quite a few issues so it's understandable if you haven't gotten the time to get back to this. Let me know if you still want to work on this or else if you have new information you can leave in the issue here so to pass the torch if you decide so. Thank you!
Thank you @rhymes I was not able to recreate this issue in local even though it happens in dev.to. Please feel free to remove me as assignee from this issue so that If someone is able to recreate this issue again they could take it up.
I have removed myself as assignee from this issue so that if anyone is able to recreate this issue in local environment can take it up.
Describe the bug
Same entries are repeated in latest feed view
To Reproduce
Choose latest tab and scroll
Expected behavior No duplicates
Screenshots
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context