ltenzil / scrape

Scrape talks about building reports on user entered keywords and its search results (Google)
0 stars 0 forks source link

Keywords scoping by user #15

Open longnd opened 2 years ago

longnd commented 2 years ago

The behavior of keyword listing on the homepage is a bit awkward: it lists all keywords uploaded by all users without requiring the users to log in. But to view a specific keyword, it requires login. Why should it work that way? 🤔

Besides that, the current DB structure looks like this: users <-> users_keywords <-> keywords

and the scrapping method in the Keyword model will update the search result of a keyword if it exists https://github.com/ltenzil/scrape/blob/6eb163261a9102f748fc789147840f7856584ef8/app/models/keyword.rb#L42-L47

so if two users upload the same keyword, the result of the second upload will override the first one.

shouldn't each report (the search result for each keyword) be independent by each user?

ltenzil commented 2 years ago

Yes, we usually let users to see something (like what we have in our website) before asking the users to sign up I tried searching same keywords in different browser as logged user, guest user. I don't see much difference.

User => UserKeywords => Keywords So i don't want users to see to same keywords like 'Ipad' or 'BTS' or 'GOT' or other trending keywords multiple time. This reduces redundant data.

Initially i want update keyword only if the hits are higher or with lower search time. but it (higher hits or lower search time) happens only on second or third time.

ltenzil commented 2 years ago

@longnd , is the above comment is sufficient? or is there any issues you see in my db design. Please do let me know. do let me know incase if you have other / more suggestions.

longnd commented 2 years ago

Yes, we usually let users to see something (like what we have in our website) before asking the users to sign up

What do you mean by our website? I assume it's a product you're building in your current job?

I tried searching same keywords in different browser as logged user, guest user. I don't see much difference.

it's because you're performing the search from the same geolocation (same city/country). It 2 users search for the same keyword from different geolocation, the results are also different. Here is an example Search from Singapore Search from Vietnam
Screen Shot 2021-12-31 at 09 34 32 Screen Shot 2021-12-31 at 09 34 40

even for the same user, if they search for the same keyword on different days, the search results are also different.

So i don't want users to see to same keywords like 'Ipad' or 'BTS' or 'GOT' or other trending keywords multiple time For the trending keywords, you'll see the search results changing frequently, even on the first page. Doesn't it make sense to capture the search result every time?

So the DB design, by using the same keyword record and override it isn't sufficient IMO.

ltenzil commented 2 years ago

What do you mean by our website? I assume it's a product you're building in your current job?

Nope, usually websites show little info to user before asking them signup. (in twitter you can see tweets before logging in). So i let users to see index without authentication.

it's because you're performing the search from the same geolocation (same city/country)

I didn't think on this, if i had i would have went with user -> keywords (without uniqueness check). that changes everything. thanks @longnd , its very helpful. Will refactor it.