Enable advanced searching

[smalers]

It would be nice to have advanced search features such as regular expressions, pipe for or, etc. This is a placeholder for future discussion of such features.

[smalers]

This is now a high priority item given that InfoMapper sites can be complex and we are trying to encourage more use. Below are some thoughts on the design.

Technical Considerations

1. Search filters
   1. Provide checkboxes to allow selecting what gets searched. Maps, Documents, Datasets. Initially maps and documents are what is important since those are the major displays.
   2. This cross-references with the content type in the next item.
2. Search results presentation - It would be good to see a sorted table with the following:
   1. Human-readable link. Maybe show the map title, Markdown document title (from main heading), etc.
   2. Content type? Maybe it will help for users to know whether it is a map, a document, etc. They probably won't know what Markdown means so perhaps just say "document". This will also cross-reference to the original selections for what to search.
   3. Relevance or score. Show them something. That way a score of 100 compared to the next one of 5 would clearly indicate a high-value item. If all the results have a score of 1, then a match is less obvious. I don't think we need to worry about popularity so whatever score we come up with is enough.

Data Design

The content that needs to be searched must have a URL corresponding to the content. We also want to separate the concerns so that content can stand on its own. In some cases, the searchable content can explicitly indicate keywords and in others it cannot, as discussed below.

1. Map configurations
   1. Because map configurations are already complex objects, adding keywords is just another element. I recommend adding a top-level property keywords : [ "keyword1", "key word 2" ]. These can be explicit searched. The search results can use the map title and the content type can be "Map".
   2. Also search the documentation associated with the map. Again, the map title can be used and content type is Map.
2. Content Pages - Because content pages are Markdown, they don't explicitly have a place for keywords. Therefore, there are the following options:
   1. Rely only on the content page words and people will just need to make sure those words show up. This is the default behavior.
   2. Allow comments in Markdown, for example as HTML comments. This will make the Markdown uglier but could be an option where other searches don't return the desired output.
   3. Add a keywords property in the configuration that links the documentation. For example, the application configuration file could allow this. This makes the content less stand-alone, but is perhaps appropriate when the content does not make sense on its own. I would avoid this for now in favor of the Markdown content.
   4. For pages such as map layer or dataset info pages, keywords can be included in the Markdown in some standard way since those pages are more technical. Let's talk more about this once other features have been implemented.
3. Spatial data metadata (somewhat advanced):
   1. This would search the layer information Markdown files.
   2. The results would be the dataset title and URL would be the map with dataset info popup.
   3. Problem - what if layer is used in more than one map?
4. Spatial data layer data (very advanced and probably slow):
   1. This would search the spatial data layer properties, for example to find a specific station ID.
   2. The results would be the dataset title and URL would be the map with data table popup.
   3. Problem - what if the layer is used in more than one map?
5. Time series data metadata (advanced and possibly slow):
   1. Need a way to search time series metadata, perhaps by using a catalog file.

Code design

It would be good if the code in the InfoMapper was not overly hard-coded. For example, using the concepts of an interface would be good, something like "implements keyword search". Then, as we add components with content, the search tool would just need to check if a behavior is implemented and if so call the function. The results that are returned will need to be standardized so they can be accumulated into the overall results. It is not clear how the scoping of URLs would be handled in that case. Are they relative URLs (relative to the app location?), absolute URLs (less flexible, especially if the site is moved).

Pregeneration of the Search Index

The software to walk the application will need to read the application configuration file and then dig into components. Although this could be run at any time in the InfoMapper, it may make sense to do it as a tool in the InfoMapper Builder, or in a GeoProcessor command. Being able to automate would be good to keep the search index up to date. Similar concepts apply to the sitemap page generator, which can be used for troubleshooting. Both artifacts would allow search engines to index static content. It is OK to develop the code in InfoMapper (Builder) for now and we'll see where it goes.

[Nightsphere]

Add keywords property to the top level GeoMapProject element as an array of strings.

The first commit for the global search has been merged and pushed. The searchable items right now are

• Content Pages - To successfully add the content page's markdown content to the search index, the title of each markdown file must be parsed out and must follow the pattern:

  # Some Title #

  If the title is not surrounded by hashes on either side, a warning message will be printed to the DevTools window that the file's content's were not added to the search index.

• Maps - The keywords property in the map config's top level GeoMapProject element is an array of strings. Each of these strings will contain the keywords associated with this map, and the Lunr search package will add a boosted score for each found, as they will mean more than words found in markdown file Content Pages. An example of a map's GeoMapProject element could look something like the folowing:

  {
  "geoMapProjectId": "...",
  "name": "...",
  "description": "...",
  "projectType": "...",
  "properties": {
    "author": "...",
    "specificationFlavor": "...",
    "specificationVersion": "..."
  },
  "keywords": [
    "Ditch service areas",
    "Water districts raster",
    "Colorado municipal boundaries",
    "Polygon geometry map"
  ],
  "geoMaps": []
  }

---

I tried to implement the query parameters for dialogs so map doc markdown files could also be searchable, but ran into issues. When searching through Content Pages or Maps with keywords, it is simple enough to route the user to that page without much hassle. Markdown files in a map, however, require much more handling.

Before I get into query parameter issues, here's how I was using them for clarity. To help ease parsing, there are 2 query parameters per dialog. They are in the form:

'dialog' + windowNumber
'dialog + windowNumber + 'Id'

so in a URL, the first opened dialog would look like:

https://localhost:4200/map/some-map-id?dialog1=map-layer-Text&dialog1Id=map-layer-id

Here is why they are not currently supported:

1. Because these files are shown in a dialog, it cannot be simply routed to. Query parameters would need to be used to uniquely identify each dialog. Time was spent researching them to get them working.
2. Multiple dialogs open simultaneously would need to be supported. There is already a Window Manager in the Common package, so logic would need to be added to dynamically add and remove query parameters as they are opened & closed (on top of making sure a dialog can only be opened once). Dynamically closing dialogs also meant logic would need to be implemented to check if any other dialogs are currently open when it's closed, and the URL with its query parameters would need to be updated to accurately reflect this change. This means adding code every time a dialog is opened.
3. If query parameters are used, users could theoretically copy-paste a URL with a dialog open, paste it into another browser tab, and have the map shown with the open dialog and its query parameter in the URL. This means there needs to be logic that checks the URL for query parameters when its loaded. The only thing I found was using RXJS and subscribe to the Angular provided Activated Route service, which will run the code inside it whenever a query parameter change is detected. The issue with this is clicking on a button on the map that opens up a dialog will a) run the code after the click event, and b) change the URL, which will run the code in the subscription, so a conditional will need to be used so it's not run twice.
4. Another issue with using dynamic map & layer names (they're coming from the map config file) is that the map config file needs to be read in first before the check for a valid query parameter can be performed. Going through the flow of a user copy-pasting a URL with a query parameter would look like this:
   • After the URL is pasted, the Angular project is created, the router determines what component should be created, and creates it.
   • The map configuration file needs to be read in first so it can be in memory.
   • The subscription to the activated route observable is done after the map config file is read in and detects there has been a change to the query parameters, and runs the code in itself.
   • The component parses the necessary query parameter and gets the id it will look for in the map config object.
   • Some component or service (the component the dialog is opening from, or maybe the window manager?) traverses the map config object and compares each of its ids (geoMapProjectId, geoMapId, etc.) with the id given from the query parameter. If they match, use the query parameters and window manager to add the dialog to it and open it up. If bad query parameters are given, don't do anything.

There were other issues to keep in mind, but they weren't as difficult to figure out. After writing out this documentation and talking to Steve, I realized I came up with a solution to most of the above issue, and at least might be able to implement the query params for map documentation files. No other dialog in the InfoMapper would be supported (except for the already supported opening of only one dialog with the same Id at a time), but at least the search could be improved because of this. I'll look into the code I've already done and see what I can do.