Filter search results by time

[Treora]

Oliver and I discussed two ways to express the desired time range:

1. by text query: one could write "since March", or "last week" (like WorldBrain&Falcon already do it)
2. by a separate input; perhaps a nice range slider or a date picker

Both are nice, but we can start with just one.

[blackforestboi]

I would tend to the date picker for the MVP now and then maybe add the slider afterwards. If it is feasible, I would leave the query parser library of Falcon also in action, because it has other cool features, like minus word recognition, date recognition, quote recognition etc. This way people still can type "after:yesterday" in a quick fashion.

I really do like the range slider from a usability perspective :), but I see a couple of problems with it though to make it work smoothly without much work: 1) It would need be somewhat dynamic in terms of the time frame that is displayed. Means with time passing by the slider would have the same length, but has to encapsule more time. With growing usage, it would contribute to a second problem: 2) Precision: If people want to add precise dates it could be hard to get this done without much fiddly work by the user on the slider itself. Magnification could solve this. but yeah. for now it might be a little too much work.

There might be some solutions to this that don't require too much work, just thinking about potential time-eaters.

[reficul31]

I would like to take up this task. According to me the course of action should be

1. Design how time should be represented in the overview
2. Add the timeline component and update the state with the time(might not be an independent component)
3. Filter the search results using the filter method and the state updation
4. Display the results

In my opinion the text search thing might not be that user friendly and will also require a lot of time to implement. Instead if we have something like a slider it would be much better. ps: wouldn't it be cool if we could order the results by oldest and newest?

[blackforestboi]

Hey @reficul31

Thanks for your enthusiasm and your clear communication around what you understood :) really appreciated!

If you don't mind, lets first finish the other issue you are working on entirely, as there probably will be some review adjustments. Also there are a lot of people interested in contributing, so by leaving this issue open, we can give others the chance to make their first contribution to the project. I can assure you, we wont run out of things to do anytimes soon 😄

As for your text search remark, the WorldBrain (Re)search-Engine extension does this already quite well. You can test it in the address bar. To get it running we can take inspiration from this file: https://github.com/WorldBrain/Research-Engine/blob/master/src/js/queryparser.js

It could indeed take quite some time to refactor and integrate it into the WebMemex, so a very manual with 2 drop downs from and to could be a good first step.

Note to myself: Make some mockups for this

ps: wouldn't it be cool if we could order the results by oldest and newest?

Good idea! Would put it up as a separate issue though. @Treora, for now I add it as a note to the Visual Memory project: https://github.com/WebMemex/webmemex-extension/projects/2

[reficul31]

So sorry @oliversauter I thought I had already sent the PR. Will not take on multiple issues at the same time from now on. Thanks.

[RajPratim21]

I plan to take on this issue.

[RajPratim21]

I plan to take on by implementing the basic text search by implementing the querry parser. 1) (text message to numerical value) first to get the intent of the message and then mapping the intent to some key words and getting the time bound. let's calll it querry_period.

2) As of after studying the code what I understood , that search/index.js fetches whole data till defaultResultLimit (30 in our case ) if no inputt given else it searches the page via ids at src/find-pages.js by Searchpage fuction by getting the data from data base. 3) It then calls the find-visits.js to get the visiting times and maps them to these pages and finally we get the required searched web links that too in a sorted manner (most recent visited first).

I plan to querry the database to get the doc.id by selecting the docs with timestamp within a range of the querried_period .

Doubt : I have where exactly I can find the timestamp of pages numerical values of times that I can compare with the given querry_period.

[RajPratim21]

I have sucessfully implemented time search as can be seen this is my default tab history regarding You Tube as it's very popular. I ussually use it.

with current fuctionality integrated . I have initially left a default search by last 25 minutes . So we can see though there are many instances of use of You Tube. only the one within time limit is getting showed. So it has been implemented sucessfully . I will now soon Implement the date picker with it. and integrate the natural language parser for the searches like "yesterday' s search on youtube" or etc. will be happy to recieve feedback and suggestions for the natural language parser.

[blackforestboi]

Hey @RajPratim21

Great stuff, thanks for implementing this so quickly!

I think the date picker would be a very good next step.

As for your questions regarding the language parser, here is the library that is currently in use by the WorldBrain extension.

Also another good thing to look at is how WorldBrain currently recognises that the language inside the query is meant to be a time filter. (file1) (file2) Currently you would type "climate change after:"2 weeks ago" before:"1 week ago"" to find everything containing "climate change" between 2 and 1 week ago.

I would find "from:" and "to:" more intuitive than "after:" and "before:" though. But that's just me.

I have the feeling that the way you imagine the language query ("yesterday's searches on youtube") could be a little bit too complicated to distinguish between a time filter and a query. (at least for now) So as a first step I would leave it a little bit more "programming language"-like.

Regarding the code, Gerben is the right person to give feedback to.

Great stuff, thanks a lot so far :)

[RajPratim21]

@oliversauter yeah surely my next step will be towards integrating the date picker and will also work with the natural language parser and that "yesterday's searches on You Tube was jsut my way of speaking as I was short of vocablary" definetly the from to way is lot more intutive , and if I will proceed it in the simpler way. Thank you for your feedback.

[RajPratim21]

I have successfully implemented the date-time search via using date-timepicker . it's working perfectly fine .

[blackforestboi]

Nice. If thats feasible, you can make a screencast and send it as a gif. With that we can see the whole work-flow: http://www.omgubuntu.co.uk/2016/08/peek-desktop-gif-screen-recorder-linux

PS: don't forget to plug your battery, soon empty ;)

[RajPratim21]

@oliversauter

All the feature requests are kept in mind while designing. Initially a placeholder ask user to select start date and select end Date

if user selcts none of them then the default search will be updated between present date to 100 days back.

if user selects just one then other is set up by default value

default end date.

Default start Date

special care has been taken so that user can't selct invalid dates of future. by disabling future dates . so that he can't select future dates that haven't occur.

Now some details about technical hacks @Treora I already send you some snippets of my code . Now in a nutshell I have done some non trivial stuff or non traditional way I have to do it. I have tested the feature and it remains robust and works perfectly fine. I have one file called DAte_value_store.js (it act like a pseudo store class) which contains a class containg Global start date and Global end Date . They are made globally available through out the code base. they are updated by date picker from Overview.jsx on onChange variable under DatePicker calls mapDispatchToProps method where MyClass.Global_start_date and MyClass.Global_end_Date' are updated and then the UI of date picker is updated via selected={Myclass.Global_startDate }andselected={Myclass.Global_endDate } and sameMyClass.Global_startDate&MyClass.Global_endDateare accessed byfind_visit.js` and checked if they are updated then used as a range for querring the data base else default values are chosen.

[RajPratim21]

There have been one edit as earlier there was an issue the UI use to got stuck and don't seems to be updating that's why we do this in this way.

store the instantaneous value of date in MyClass.Global_start/end_date then update it in reducer via

function handleStartChange(state, {startDate}) { //MyClss.Global_startDate = startDate; return Object.assign({},state, {startDate :MyClss.Global_startDate}) }

function handleEndChange(state, {endDate}) { //MyClss.Global_endDate = endDate; return Object.assign({},state, {endDate : MyClss.Global_endDate}) }

and that resolved all the issues looking forward to the PR.

[RajPratim21]

when none of the fields are selected

[RajPratim21]

When just End date field is selected

[RajPratim21]

Start Field selected

[RajPratim21]

Both fields selected

[blackforestboi]

Great stuff. Looking forward what Gerben has to say for the code :)

Thanks for getting this run so quickly and all the sweat and tears!

[RajPratim21]

I have resubmitted PR #74 fot it.

[RajPratim21]

@oliversauter , @Treora I have added the natural language parser by using moment.js as it supports various languages, intially I am using only those functionalities like "yesteday", "r days before last month", etc so as to get the startDate of the range and end Date is taken as either current Date or date selected by date -picker, that from to statement is not implemented yet, but present parser comes with lot of functionalities. https://momentjs.com/ and for references https://github.com/juttle/moment-parser,