Improve the HTML documentation generation

[Moult]

I tried to use http://ifc43-docs.standards.buildingsmart.org for daily doc browsing but really struggled and often had to revert to looking at existing IFC4 docs. This issue is to list all the things I'd like to see resolved (and help write the code to fix it too!) so I don't forget.

Content

• Does search not do partial matches of page titles? E.g. searching IfcAir won't come up with IfcAirTerminal
• EXPRESS functions don't link to their definition, for example in here there is a link to here
• Computer interpretable listings is missing IFC XSD long form schema, psets in IFC, qtos in IFC, "MVD-XML model view definitions", and "EXPRESS XSD configuration" (copy pasted from existing docs)
• No examples in the examples section
• Changelogs should show inherited changes
• Changelogs on psets
• New functions such as http://ifc43-docs.standards.buildingsmart.org:51734/IFC/RELEASE/IFC4x3/HTML/lexical/IfcAssociatedSurface.htm apparently haven't made it into the UML yet and no list of functions
• No list of rules

Aesthetics

Edit: all fixed

Code

Edit: all fixed

Just a running list. Keen to hack but I need some handholding to make it easier to work with docker :(

[Moult]

Are examples working? I have no examples dir? Do I have to fetch examples somehow?

[Moult]

Little screenshot showing syntax highlighting. If we get motivated we can write a syntax file for EXPRESS:

[Moult]

I got motivated

[Moult]

Examples :) The second most valuable thing for devs!

[aothms]

Oh this is gold!

[aothms]

Fyi I see quite a bit of 404s related to the examples:

--2022-02-17 09:46:29--  http://localhost/IFC/RELEASE/IFC4x3/HTML/annex_e/nageSystem_2_3.JPG
  HTTP/1.1 404 NOT FOUND
--2022-02-17 09:46:29--  http://localhost/IFC/RELEASE/IFC4x3/HTML/annex_e/nageSystem_2_4.JPG
  HTTP/1.1 404 NOT FOUND
--2022-02-17 09:46:29--  http://localhost/IFC/RELEASE/IFC4x3/HTML/annex_e/nageSystem_2_5.JPG
  HTTP/1.1 404 NOT FOUND

[Moult]

I reverted back to getting concepts from the XMI, but left the old code there so it is easy to compare. For example, with the XMI approach, it looks like IfcRoot has no concepts at all?

[aothms]

• [ ] Concepts don't show their sub concepts anymore http://ifc43-docs.standards.buildingsmart.org/IFC/RELEASE/IFC4x3/HTML/concepts/Object_Connectivity/content.html

[Moult]

Did I mess something up? I don't recall actually changing that code at all - it still doesn't use Jinja templates. I'm not sure what I should see or how to debug?

[Moult]

https://github.com/buildingSMART/IFC4.3.x-development/commit/704a40df1cb9d1a23e6e21b984ea3fa2946121a7

This commit is awesome! I now see things like Revision / Identity which I expect to see. Is IfcObjectDefinition missing classification association? e.g. on this page http://ifc43-docs.standards.buildingsmart.org/IFC/RELEASE/IFC4x3/HTML/lexical/IfcWall.htm

[aothms]

It is related to this thing:

https://standards.buildingsmart.org/IFC/DEV/IFC4_3/RC4-voting/HTML/link/ifcdistributioncontrolelement.htm

(note that we currently haven't modelled this one yet)

We're still a bit in doubt what to do with this and how to evaluate the semantics of concept usage, but especially the semantics for the elements not parametrized. In this case it's quite obvious. ClassificationAssocation is a general concept and this one particular parametrization highlights some recommended usage for a specific entity. It doesn't preclude other entities from using the concept even if not directly associated. In other cases though the listing of parametrizations is "more or less exhaustive" and non-bound usage is not recommended (e.g you're not supposed to associate a material layer set to a furnishing element). I haven't really understood yet how to distinguish this.

Edit note that this applies tot the concepts currently set to NO_PARAMETRIZATION https://github.com/buildingSMART/IFC4.3.x-development/blob/master/code/concept_interpretation.py

[aothms]

I did a very rough sketch of the listings. As expected from me, it's nasty. We fire up a temporary redis as a global map. We use wget to spider the site. That's it.

There's two observations:

• some of the images have weird labels, they're quite easily distinguishable especially apparent in the listing because of the usage of li.number which html-escapes
• the approach of auto-numbering by section creates a lot of duplicates, especially for e.g some of the widely used concept images. Ideally each figure (uniquely identified by URL) should get one and only one number. We do have the mechanism now to globally attach a label to any image URL. What I would propose is somehow adjusting the figure numbering to keep the smallest number for each and every img url.

http://ifc43-docs.standards.buildingsmart.org/IFC/RELEASE/IFC4x3/HTML/listing-figures.html

Thoughts?

[Moult]

For the object classification, I reckon that table is not quite correct - BACnet is not a classification system, that should be migrated to use a library association. Same would go for the others. Classification references should be restricted to clearly published classification databases, not an arbitrary identifier in an external system.

[Moult]

Agreed that auto numbering by sections creates duplicates and the approach could be flawed. I'll be happy to have a shot at numbering by image URL which will be unique, shouldn't be too hard. The only downside is that it won't work on SVG generated diagrams.

This will also make images waaay easier to handle in the listings.

Redis + wget seems overkill, couldn't a route simply call other routes within Flask and spider itself and parse the incoming HTML for < figure > ? I can have a shot at this too.

[aothms]

couldn't a route simply call other routes within Flask and spider itself and parse the incoming HTML for < figure > ? I can have a shot at this too.

Maybe... I don't see immediately how it would work, stackoverflows, memory, performance, etc.. Also keep in mind that diagrams can be updated in subsequent changes that are live pulled. (I know I also shut down redis after the spider, but at least there would be an easy way to refresh it).

The only downside is that it won't work on SVG generated diagrams.

True, we already hash the content though so should be easy to solve. Also they're probably not yet in use in any of the content that gets repeated over multiple pages.

[Moult]

Closing as the majority of issues are solved, and the remaining issues are now tracked separately in:

257 #384 #385 #386 #387 #388 #389