Full text search of HTML spec

GallLeo commented 5 years ago

How do you search through the full HTML spec? https://specification.modelica.org/v3.4/MLS.html

Using browser search, I only can search on the current page.

For FMI, it's one very long HTML page, so browser search does work for the full document: https://fmi-standard.org/docs/2.0.1-develop/

sjoelund commented 5 years ago

You search on github :( See also #2320

beutlich commented 5 years ago

I search in PDF to find the section number I am looking for.

Or use https://dev.openmodelica.org/~marsj/tmp/MLS/index.html from #2320.

GallLeo commented 5 years ago

Okay, so currently no search option available for the HTML version :( I think we should leave this ticket open in order to not loose it in the more general Sphinx discussion.

casella commented 5 years ago

This is a big deal. I very often search the Specification PDF document for keywords, no longer being able to do so is a major drawback.

Do we have a PDF of 3.5 to search as suggested by @beutlich?

HansOlsson commented 5 years ago

This is a big deal. I very often search the Specification PDF document for keywords, no longer being able to do so is a major drawback. Do we have a PDF of 3.5 to search as suggested by @beutlich?

Yes, https://specification.modelica.org/master/MLS.pdf according to the listing in main page (pdf for 3.5-dev).

casella commented 5 years ago

Ok, now I see it. Maybe we should link this page from https://www.modelica.org/documents?

beutlich commented 5 years ago

It is linked from https://specification.modelica.org.

casella commented 5 years ago

Yeah, I mean, what if I don't know the address https://specification.modelica.org?

If I google 'Modelica Specification 3.5' I get here, but from that page how to get to the PDF is not obvious, and if I google 'Modelica Specification 3.5 pdf' I don't get anwhere near the right place. Which was the reason why I posted my previous comment :)

I guess a link from www.modelica.org/documents would enhance the accessibility from search engines a lot. Shall I do that?

HansOlsson commented 5 years ago

Since 3.5 is still under development I don't think we should advertise it that much;; and similarly for MSL 4.0.0.

Obviously when released, the situation changes.

beutlich commented 5 years ago

OT: But if you like you can update the two links to the Modelica Trac which are outdated.

casella commented 5 years ago

This is a specific instance of a more general issue. I happen to be able to contribute to MA work during some periods in the year, but not during others, so I'm not working on the whole thing continuously on a daily basis. I guess I'm not alone in this situation. After a few months, stuff may have been renamed, or moved elsewhere, or maybe you simply forgot where it is.

Hence, it is of paramount importance to be able to retrieve all the important links and documents from some well-known web entry page, which I'd say is either https://www.modelica.org/ or https://github.com/modelica/.

Furthermore, having everything linked from there also allows one to use search engines to quickly locate what is needed.

casella commented 5 years ago

OT: But if you like you can update the two links to the Modelica Trac which are outdated.

Do you mean here?

beutlich commented 5 years ago

OT: But if you like you can update the two links to the Modelica Trac which are outdated.

Do you mean here?

No. See

grafik

casella commented 5 years ago

Since 3.5 is still under development I don't think we should advertise it that much;; and similarly for MSL 4.0.0.

Why not?

On the contrary, I am convinced that being open about the process would improve the overall quality of the process.

Of course we don't necessarily need to make all the discussions public. But on the other hand 3.5 is almost finalized now, and the decision to go for 4.0.0 is not something we are going to reverse, so the more people are involved, the better.

Of course it is only MA members who take decisions, but the more input we have from the outside, the better.

What do you think?

casella commented 5 years ago

@beultlich, I'll update the website page myself. I also opened #3117 to propose taking trac offline.

casella commented 5 years ago

Done, see the updated page.

@GallLeo I think this solves your problem, you can just to a full text search on the PDF. Or use Google adding site:specification.modelica.org to the search string.

I guess we can close this ticket now.

sjoelund commented 5 years ago

Searching in a PDF is not very reliable. You have hyphens and other things added by LaTeX that mess up your search strings.

GallLeo commented 5 years ago

I tried the google search with an example: site:specification.modelica.org/master "inverse ="

For the google search you have to first find the right URL:

specification.modelica.org is to general. I do not know where it is going to find stuff for me.
specification.modelica.org/master/MLS.html is what you can copy from your browser's address bar. But, this does not work as a site for google
specification.modelica.org/master is the URL which worked for me.

So:

Google site search works, but still I think a search window on the top left would be much nicer. I do not know how many Modelica users are familiar with google site search.
PDF search is very helpful, but not what we wanted: One important idea of the HTML spec was to be able to quickly point other people to the right section. If I would search in the PDF, I then would have to remember the correct section, click on this section in the HTML and then send this link to other people.

beutlich commented 5 years ago

Reopen, since the actual issue topic "Full text search of HTML spec" was not resolved.

HansOlsson commented 5 years ago

I see two obvious possibilities:

Integrate Google (or Bing or duckduckgo or...) custom search in that page in some way, similarly as "Custom Search" on modelica.org; so that it just works. Unfortunately it seems we either get ads (Google) or it costs fractions of a cent per search (Bing).
Generate the HTML as one file as a complement to other variants, but that is likely too slow to load due to all of the equations and images. (But should be quick to test.)

On a separate note: having an actual index would be nice, but: it's a lot of work to do in a good way (auto-indexing don't work well enough, especially not when so many keywords are common words) - and the risk is that we just gets an index that isn't useful.

casella commented 5 years ago

I checked what Duckduckgo can do for us, without subscription or anything. I think it's a good starting point. I made a test page, please try it out. We may eventually provide one page with different search boxes, one for each version of the specification.

beutlich commented 4 years ago

As part of https://github.com/modelica/ModelicaStandardLibrary/issues/3287 @pylypko proposed to use the https://docsearch.algolia.com/ service for indexing and filtering search results of community-based technical information.

We already set it up for the Modelica libraries (Modelica, ModelicaReference, ModelicaServices, Complex and ObsoleteModelica). In https://github.com/algolia/docsearch-configs/commit/06cf7b2f8a86f58747774d99f8591f29e26d558d I added the Modelica specification to give you an impression how it could look. Currently, filtering is possible on the different version "3.4" and "latest" of the specification. As usual, there is plenty of room for improvements. At least, there is the chance now to update site https://specification.modelica.org/index.html with a search input field.

grafik

henrikt-ma commented 4 years ago

It sure looks nice.

Henrik

On 12 May 2020, at 12:17, Thomas Beutlich notifications@github.com wrote:

As part of modelica/ModelicaStandardLibrary#3287 @pylypko proposed to use the https://docsearch.algolia.com/ service for indexing and filtering search results of community-based technical information.

We already set it up for the Modelica libraries (Modelica, ModelicaReference, ModelicaServices, Complex and ObsoleteModelica). In algolia/docsearch-configs@06cf7b2 I added the Modelica specification to give you an impression how it could look. Currently, filtering is possible on the different version "3.4" and "latest" of the specification. As usual, there is plenty of room for improvements. At least, there is the chance now to update site https://specification.modelica.org/index.html with a search input field.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

HansOlsson commented 3 years ago

It seems this can be decoupled from the version of the Modelica spec.

beutlich commented 3 years ago

It seems this can be decoupled from the version of the Modelica spec.

If you are referring to my screenshot above, yes. We can set up the algolia search for the MLS easily. The search index already is prepared.

beutlich commented 3 years ago

Here's a full proof of concept using the same search engine and layout as for the Modelica Standard Library: https://beutlich.github.io/MLS/index.html

henrikt-ma commented 3 years ago

Nice. Do you have any ideas for how it could be integrated with the document?

beutlich commented 3 years ago

No, I haven't. Would be sufficient for me to have it as landing page for https://specification.modelica.org.

henrikt-ma commented 3 years ago

No, I haven't. Would be sufficient for me to have it as landing page for https://specification.modelica.org.

Sounds like a good start, but I hope it will turn out so useful that it in the end will deserve a more convenient location for someone already reading the document.

eshmoylova commented 3 years ago

It looks good. One question. If I search for "constant" (for example) it looks like it is searching only in the headings and bullet items. It doesn't mention regular appearances in text or examples. Is that expected?

beutlich commented 3 years ago

Yes, it is expected that not every occurence is found. That is due to the current HTML layout and the laborious configuration of the search framework, e.g., there is no hit for "OtherExample", though it is available in section 12.9.4.

grafik

We can still decide which (HTML) classes to include in the search index. Luckily they are properly marked.

beutlich commented 3 years ago

This issue is still open while targeting the closed milestone 3.5. I propose to first get it resolved before the MA approval of MLS 3.5.

HansOlsson commented 3 years ago

This issue is still open while targeting the closed milestone 3.5. I propose to first get it resolved before the MA approval of MLS 3.5.

Since it shouldn't impact the text of the specification I don't see that it needs to be synchronized.

beutlich commented 3 years ago

It looks good. One question. If I search for "constant" (for example) it looks like it is searching only in the headings and bullet items. It doesn't mention regular appearances in text or examples. Is that expected?

@eshmoylova FYI: This is now fixed, i.e. searching in text (e.g. "OtherExample" or "component3") no longer results in missing search hits.

modelica / ModelicaSpecification

Full text search of HTML spec #2404