We deferred work on the broken date search on the front page advanced search, but this is still important because it is Time layered cultural map. See: Date Search Broken #45.
This could not be completed because the database query and handling is necessarily inefficient. Dates need to be recorded as entered as many systems, including our database cannot handle all forms of dates that we need to be able to handle for Humanities purposes (eg: BC dates, years less or greater than 4 digits, dates that are a year alone, or with dd and mm etc etc). So any query has to get this free text field for all records and convert the dates into some format that can be queried, and then present results. Now there are more than 700,000 records these all have to be got and converted to check, and so we cannot simply limit the query to max 200, etc.
The solution will probably be similar to the changes made to coordinates for the ST Analytics handling. In that case, for PostGIS to function, it needed all coordinates to be converted to a specific kind of numeric value. We created additional fields for these values, rather than convert on the fly, for efficiency. Similarly, we can convert all dates into a single numeric value, stored in a seperate field. If a user has only provided a year, for StartDate we should assume January 1st of that year, and for date end we should assume December 31 of that year, during the conversion.
Need to check and decide on the best numeric value for dates:
One possibility for this numeric value is the 'Unix epoch', as there will be existing methods for transforming dates to Unix epoch dates. This may result in numbers too large. Otherwise we could consider the year 0 as 0, and use positive and negative values and run our own conversions. If using date transformation functions built into the database or programming language, we must check they can handle all TLCMap compliant dates (BC dates, and years with less or more than 4 digits).
We deferred work on the broken date search on the front page advanced search, but this is still important because it is Time layered cultural map. See: Date Search Broken #45. This could not be completed because the database query and handling is necessarily inefficient. Dates need to be recorded as entered as many systems, including our database cannot handle all forms of dates that we need to be able to handle for Humanities purposes (eg: BC dates, years less or greater than 4 digits, dates that are a year alone, or with dd and mm etc etc). So any query has to get this free text field for all records and convert the dates into some format that can be queried, and then present results. Now there are more than 700,000 records these all have to be got and converted to check, and so we cannot simply limit the query to max 200, etc. The solution will probably be similar to the changes made to coordinates for the ST Analytics handling. In that case, for PostGIS to function, it needed all coordinates to be converted to a specific kind of numeric value. We created additional fields for these values, rather than convert on the fly, for efficiency. Similarly, we can convert all dates into a single numeric value, stored in a seperate field. If a user has only provided a year, for StartDate we should assume January 1st of that year, and for date end we should assume December 31 of that year, during the conversion. Need to check and decide on the best numeric value for dates: One possibility for this numeric value is the 'Unix epoch', as there will be existing methods for transforming dates to Unix epoch dates. This may result in numbers too large. Otherwise we could consider the year 0 as 0, and use positive and negative values and run our own conversions. If using date transformation functions built into the database or programming language, we must check they can handle all TLCMap compliant dates (BC dates, and years with less or more than 4 digits).