Review of MapML-Proposal explainer

zcorpan commented 4 years ago

I've reviewed the explainer yesterday and today. I haven't yet reviewed other MapML material, so take this as my raw first reaction to reading this explainer.

At a high level, this explainer doesn't help me understand the problem very well. I see a series of claims that I don't understand how they follow from one to the other. Different concerns are not separated; the proposal is mixed with the problem statement.

I have concerns about the extensions of existing HTML elements being risky for web compatibility.

Accessibility, security and privacy are mentioned in passing but not really addressed.

I think the explainer should be rewritten to more clearly explain the high-level problems (based on the Use Cases and Requirements doc), explain the proposal (for a web developer audience; no need to cover all aspects of the proposal), alternatives, how those proposals address those problems.

The explainer is also too long; too much detail of the proposal.

More details inline:

(Note: I found the bold formatting makes this harder for me to read. I've removed bolding here.)

(Edit by @Malvoz: Bold formatting removed in https://github.com/Maps4HTML/MapML-Proposal/pull/19/commits/f8866400525be9a375ac8e71925911b6f953c865.)

Introduction

Web maps are a well-established domain of Web design, and there exist popular, mature open and closed source JavaScript libraries to create and manage Web maps. JavaScript web maps are often containers for publicly available and funded open geospatial and statistical data. Yet despite established JavaScript libraries and server-side API standards, Web maps remain a complex Web niche that is difficult to learn, due to their extensive prerequisite knowledge requirements. As a result, there exists a community of Web map developers which contributes very little to the Web platform and which may possess little understanding that the Web exists as a distinct and standards-based platform. Similarly, the Web platform seems mostly oblivious to Web maps and their requirements, and provides no direct support for maps. In other words, Web maps existence in the Web platform depends on intermediaries which “abstract away” the Web platform.

The goal of this proposal is to bridge the gap between the two communities in a way that may have positive benefits for both sides.

I can follow perfectly this far. 🙂

On the one hand, the Web mapping community is burdened by intermediaries and the consequent barriers to widespread creation and use of maps and public map information.

It would be good to provide evidence of this (not necessarily in the introduction).

On the other hand, the Web platform, especially the mobile Web, needs more and better high-level features and less JavaScript.

Why does the web platform, especially the mobile web, need that?

Simple yet extensible Web maps in HTML, that equally leverage the other platform standards, is the feature that both communities need to come together to improve usability and accessibility for users.

The Problem

Web maps today are created using a wide range of technology stacks on both the client and server, some standard, some open, and some proprietary.

Obligatory XKCD: https://xkcd.com/927/

The complexity of choices and the wide variety of technologies required to create Web maps results in maps of highly variable usability and accessibility.

I don't really follow this. Usability and accessibility are used but without substance.

(Edit by @Malvoz: Adressed by linking to the Web map tools WCAG 2.1 evaluation, in https://github.com/Maps4HTML/MapML-Proposal/pull/21/commits/edf64d6fb006f61ead9873df8c9a3416a708ba53.)

This has in turn led to the creation of centralized mapping services, that may or may not be implemented using Web technology;

Any citation for this cause and effect claim?

in some cases, mapping services which work well on desktop Web browsers mostly bypass the mobile Web through creation of mobile platform mapping apps, where the ‘rules of the Web platform’ (such as device permissions) do not apply.

How is this a problem?

Is the device permissions model for native mobile apps here still accurate?

Some centralized mapping services, both on the Web but especially on mobile technology platforms, are constructed for the purpose of tracking the user’s location and their locations of (search) interest, and using that private location information to market and re-sell highly targeted advertising.

Is the tracking and ads problem solved by this proposal?

Is the existance of centralized mapping services a problem?

So, What Is The Problem to Solve?

How is this section different from the previous one? (Should this be "The Proposal"?)

(Edit by @Malvoz: Changed the section heading to "The Proposal" in https://github.com/Maps4HTML/MapML-Proposal/pull/21/commits/9183add07aa0f238a32491c70ecfc5e13e2fdf18.)

The problem to be solved, therefore, is to reduce the threshold complexity of creating accessible and usable Web maps, and to enable full use of Web platform standards such as HTML, URL, SVG, CSS and JavaScript in map creation, styling, presentation and interaction.

The claim is to reduce complexity. But the proposal has considerable complexity. It's not clear that the existing stuff will go away once this exists. So it's not reassuring that complexity will be reduced for anyone.

To solve the problem, our approach is to identify the Web map processing that is currently performed by JavaScript libraries which should instead be defined as elements and attributes supported by CSS, while at the same time, we identify the Web map processing that should remain in the JavaScript domain as a standardized DOM API. By building the core behaviour of maps and layers into HTML, Web authors who want to build simple maps into their pages can easily do so, supported by core platform technologies, with the power of JavaScript available to enhance the core map and layer behaviour.

This is briefly discussing the method and also a solution.

By lowering the barriers for Web map authors in this way, we will improve the usability,

How? Usability for whom?

and standardize the accessibility of Web maps.

How?

Through making map creation a matter of applying appropriately crafted Web platform standards, we will create the conditions to multiply the choices of mapping services offered to authors and users of the Web.

How?

In improving the choices among mapping services available through the Web platform, we will enable the growth of services that offer alternate means of paying for maps other than in exchange for the user’s personal private information,

What do tracking-free mapping services need?

and we will enable standardized Web map accessibility through addition of maps to HTML.

How will accessibility be addressed?

Finally, by making it cheaper to create Web maps than to build mobile apps, we will improve the business rationale for choosing the mobile Web as a development platform, and in doing so we hope the (mobile) Web will benefit from increased ‘success’, or network effects.

How will it be cheaper?

Goals

Follow the HTML Design Principles to create HTML elements and attributes that

This is part of the proposal and doesn't belong in the goals.

(Edit by @Malvoz: Moved this part from the goals section in https://github.com/Maps4HTML/MapML-Proposal/pull/21/commits/8d983f62e631fb78d9a759bf1086b93e2288a7db.)

allow authors to create dynamic, usable and accessible Web maps about as easily as they can embed an image, a video or a podcast today.

This by itself is a goal.

(Edit by @Malvoz: Extracted this part to describe a goal in https://github.com/Maps4HTML/MapML-Proposal/pull/21/commits/8d983f62e631fb78d9a759bf1086b93e2288a7db.)

Define and embed accessibility of map feature and location information into HTML for use by screen readers and other assistive technology.

Define and design security of map information considerations into the Web platform.

Define the markup to create mapping mashups that doesn’t necessarily require scripting or detailed mapping server technology knowledge i.e. that can be accomplished about as easily as linking to a document.

Simplify the use of public spatial data infrastructures (SDI), such as Open Street Map and national and international SDIs, by designing the integration of those services into the proposed Web platform mapping standards.

Defining and (advocate for) adding map-enabled HTML to the serialization formats available from existing spatial (map) content management systems, APIs and Web Services.

This last bullet point also seems to mix in the proposal. Is the goal to have one serialization format that is adopted by all spatial content management systems, APIs and Web Services?

Non-goals

Changing the operating model or availability of existing spatial (map) content management systems, APIs and Web Services.

Backwards-incompatible changes to HTML / breaking the Web

Breaking the web is not a non-goal, it's an anti-goal (or requirement to not do).

(Edit by @Malvoz: Removed this non-goal in https://github.com/Maps4HTML/MapML-Proposal/pull/21/commits/2a7800b0c924e86e72c2b78174fbbc52213dfa6e.)

Alternative Approaches

SVGMap - is it possible to merge the SVGMap proposal and this proposal? Or are they competing proposals?

APIs: Leaflet, OpenLayers and others, (albeit others without any notion of cross-origin resource sharing) provide excellent map scripting APIs and events. Can these or similar APIs be built on top of the proposed HTML infrastructure? Would life be simpler for authors with the proposed HTML?

Status quo

Good questions, and we should answer them and evaluate different proposals against the use cases and requirements.

A High Level API

The Extensible Web Manifesto calls for iterative development and evolution of platform features, starting with low-level ‘primitives’ and resulting eventually in high-level features. Although there are several low-level primitive proposals inherent or implicated in this proposal, overall this can be seen as a proposal for a high-level feature. That feature is declarative dynamic Web maps in HTML. Web mapping is a mature category of JavaScript library that is well into the stage of its development life cycle that some of the aggregate characteristics of those libraries should be incorporated into the platform. As such, this proposal captures some of the ‘cow paths’ of open and closed source JavaScript Web mapping libraries, as well as taking into consideration how to incorporate server-side mapping services and APIs.

That maps is a mature problem space and we can reach for a high-level API makes sense, but those APIs should still be on top of low-level primitives (existing or new ones).

The map element

To allow authors to create declarative Web maps that have approximately the same authoring complexity as <video> or <audio> elements do today, we propose to extend the processing model of the currently existing HTML <map> element. The current HTML <map> element, with its associated <img> and child <area> elements constitutes a simple Web map, wherein the pixel-based coordinate system of the element is shared and used by the <map>’s child elements to draw geometries to act as hyperlinks. This relationship, that of parent node-child node with a shared coordinate system is an ideal extension point for more dynamic Web mapping. If it turns out that the code involved in the <map> element implementation is too much like spaghetti, we will have to create a new and similar element anyway. So if possible we should avoid that particular form of technical debt and adopt, maintain and extend the existing <map> element.

map is a legacy element in HTML and is treated as essentially obsolete; it hasn't gotten new features or improvements, other than for web compat and interop. Reusing legacy elements in HTML for new features is usually not a good choice, as web compat constraints what can be done and it can be a source of confusion for web developers.

The web compat constraints cannot be understated. map is used on 1.96% of the web. Even things that seems minor can break pages. If you want to reuse a legacy element for a new feature, you need a very compelling case to make it worth the risk.

(Edit: filed https://github.com/Maps4HTML/HTML-Map-Element/issues/30 )

The proposed extension would create a standard <map> widget that contains controls in a closed shadow root,

This is opening a can of worms. Does this happen for all map elements? If yes, how does that not break web compat? If no, and it's toggled by an attribute, that seems like considerable implementation complexity and an unnecessary source of bugs.

(similar to <video> today), with child <layer> elements which are in, and may contain, light DOM map-related markup, the vocabulary of which is also part of this proposal.

What markup goes in a layer?

A working example custom element prototype is available (although it’s not yet a fully compliant ‘polyfill’. For one thing, it doesn’t work on Webkit due to the use of custom built-in elements. A parallel <geo-map> autonomous custom element suite is available for Webkit. For another, the light DOM content of <layer> is not currently active or available as an API).

Move the polyfill reference to an appendix or separate section.

(Edit by @Malvoz: Moved to a separate section in https://github.com/Maps4HTML/MapML-Proposal/pull/21/commits/15bfd7c5b21a2d8a2569232ef20fea09731c15ab.)

The map element

The layer element

A layer is a primitive abstraction that is shared by many if not most maps, both on and off the Web. Even in the case that a map doesn’t have more than a single layer, the map itself can be considered to be composed of that single layer. A key characteristic of map layers is that they share the same coordinate system as other layers in the map, allowing the rendering engine to lay out and render content over the same coordinate space. This is in contrast to the normal rendering of HTML elements, which are laid out in document order down the page. There is precedent for the proposed rendering model for maps in HTML, however. The “client side image map” of HTML, implemented by the <img>, <map> and <area> elements, has this rendering model; the <area>s “coords” attribute are expressed in units of the pixel-based coordinate system of the <map>-associated <img> element.

Other examples: <canvas> (imperative), SVG (declarative). While they are things that have a coordinate space, and maps have that as well, it doesn't follow that it necessarily makes sense to build a maps feature on top of them.

An author-friendly system for dynamic Web mapping, such as that intended by this proposal, would make it as simple as possible to automatically layer map content, ideally re-using and extending the markup for <img> <map> and <area> that has become familiar. A more thorough discussion of how this proposal conforms or relates to the HTML Design Principles is beyond the scope of this explainer, but is available here.

(I'll review that separately.)

The layer element

Layers are added to maps by adding <layer> elements as children of the <map> element. Every <layer> added to the map must share the same coordinate system declared by the <map> in its projection attribute. If the layer added does not share the same coordinate system as the <map>, it will not be displayed and the layer will be ‘disabled’ in the layer control of the map widget. The set of possible coordinate systems is small, and the values enumerated by the MapML specification. If no projection attribute is specified, the default coordinate system is that of Web Mercator, signified by the “OSMTILE” value.

The <layer> element renders content by one of two alternate means. The first is by referring to remote content at a URL specified by the src attribute. This is described under Key Scenarios below. The response to a request the the src’s value should be “MapML” content, identified over the network by the “text/mapml” media type. MapML is intended to be part of the HTML vocabulary and to be parsed with the (almost) unmodified HTML parser.

"(almost) unmodified HTML parser" implies modifications. Which ones, and why? Proposals to change to the HTML parser is usually met with resistance from browser vendors due to the risk of resulting security problems. See https://github.com/whatwg/html/issues/2253#issuecomment-272100126

To achieve that, some elements in the MapML specification rely on and extend the definitions of elements of the same name in HTML. The second rendering path for <layer> is to include “MapML” content inline in light DOM.

(I'll review the spec separately.)

It’s great that we can imagine a way to extend HTML to include dynamic Web maps using almost-existing browser code infrastructure,

What does almost-existing browser code infrastructure mean? What infrastructure is missing?

but where will layer content as envisioned come from? Without content, new browser code would not be useful. What’s needed is a way to get the vast amount of existing content from services that have well-defined standard and not-so-standard interfaces, into the new HTML-MapML format available at simple URLs, without requiring significant change to the content, but especially without requiring browsers to ‘understand’ interfaces other than the uniform interface of HTTP. This might seem like a tall order, but a key design constraint followed by HTML-MapML is that the vocabulary itself should represent abstractions that ‘fit’ the existing content, so that if the services by which the content is accessed today were to provide HTML-MapML as a serialization format, then such content would be rendered inherently compatible with browsers, even before script execution. Such experimentation has formed the basis of significant experimentation by several participants through the OGC Testbed program, and has proven to be readily possible in practice, through the creation of server shims. We anticipate that should the Web community agree to work with the mapping community on this proposal, that we will have success in persuading the public spatial data infrastructures around the world to be made available to the browser in HTML-MapML.

I appreciate the chicken-and-egg problem, but I think it's not really a problem. Browsers implement features based on the feature's own merit all the time, without needing prior existing content. The motivation for a polyfill should be to get real-world experience and iterate on the API shape to inform what the standard API should look like, and to demonstrate support from web developers.

Moreover, I think creating a legacy of web content before a proposal is implemented can hurt the proposal more than it helps. Bugs in the polyfill can get relied on by web content, resulting in the final API needing to use different names from what is used in the polyfill. (Examples: Array.prototype.contains; Node.prototype.rootNode)

For the situation of no browser implementation and only a polyfill, I suggest not using the names that you want to reserve for the final API until it is shipped by at least one browser. (This was practiced by PictureFill, which used spans until picture was implemented in browsers.)

Even using a feature after it has been shipped by one browser is not without problems: bugs in the first implementation can get relied on (accidentally) by web content, leading to those bugs getting "stuck", and other vendors having to copy those bugs, or it can take considerate effort over many years to remove the original API. (Example: Shadow DOM, which had a spec, then shipped, then the spec changed significantly, but the original implementation couldn't be removed due to web compatibility. "v0" was finally removed this year, 6 years after it was introduced, and 4 years after "v1" shipped.)

I'll skip reviewing the remainder of the explainer for now. If getting maps a first-class feature in the web platform is a goal, the process is imperative to its success.

(snip)

Use Cases and Requirements

This proposal is being evaluated against the Use Cases and Requirements for Web mapping, to identify gaps between the required functionality and the polyfilled behaviour.

Considered alternatives

TBD - we have considered many alternatives, I have just run out of steam to document them, at the moment. Also this document is already quite long. As things progress, I will add content here.

I'm looking forward to seeing those alternatives documented.

Stakeholder Feedback / Opposition

Some participants have said we should start over, because of the sunk costs fallacy; that doesn’t seem to be in the spirit of iteration, nor hopefully is it correct to see this proposal as a waste of energy or money. A better strategy would be to solicit concrete, actionable and incremental change requests. It is in that spirit that the current explainer is offered.

It's not a waste to develop a concrete proposal, but I think it should be positioned as one proposal out of several when several exist, and be open to being evaluated as such.

The objective of this project is to get Web browser projects to agree that Web maps as a high level feature are desirable and that the proposal is implementable, and then to implement and ship the feature. To get there from here, we need browser developer participation, the absence of which appears equivalent to opposition. So, there is work to do.

I hope my feedback here is helpful for the getting from here to there.

(snip)

prushforth commented 4 years ago

Hi Simon, Welcome to the community group, and thank you for this review! Yes, even this explainer needs iteration.

I hope my feedback here is helpful for the getting from here to there.

Yes, it is, and insofar as I am able to fix it, I will. We are willing to do what it takes to get there from here. Hopefully we can use your and others' feedback to improve the proposal. Keep in mind the proposal as summarized is from the OGC community who have worked on it so far, with myself as the editor. We have been waiting for more of the Web platform community to join us, and so your review is very welcome in that regard.

I think it should be positioned as one proposal out of several when several exist, and be open to being evaluated as such.

The biggest challenge is to gain collaborators even willing to read your proposal (I am no better than anyone else in this, probably worse actually), much less make independent proposals. In preference to choosing among proposals, which I think might waste people's good will, it has always been the intention of this group to work in the open so that people could fork and propose/pull requests, so as to obtain the best result for the Web that can be agreed upon, and one which responds to the HTML Design Principles. You mention process above:

I'll skip reviewing the remainder of the explainer for now. If getting maps a first-class feature in the web platform is a goal, the process is imperative to its success.

I think we should create an issue about what the process is we're going to follow. That would help have something to refer to and discuss.

The ideas of the HTML Design Principles and Extensible Web Manifesto are hopefully burned into the process. The challenge in the latter is to scope out what primitives (that don't yet exist) that are common to others needs but on which Web maps, a high-level feature - and the goal of the EWM process, might be built.

zcorpan commented 4 years ago

I've started reading https://maps4html.org/HTML-Map-Element-UseCases-Requirements/ today, and my feedback so far is that it is much clearer and and doesn't leave me with questions and concerns like reading the explainer did. It also seems to have an idea about the process for the work, and a plan for accessibility, privacy, security. Very nice!

I think a summary that document would make for an excellent part of a new explainer.

zcorpan commented 4 years ago

For evaluating different proposals, or comparing them to existing alternatives, a table like https://github.com/jackbsteinberg/std-toast/tree/master/study-group#table-of-comparison could be used to get an overview of how different solutions compare on use cases, features, or other aspect you want to compare.

prushforth commented 4 years ago

For evaluating different proposals, or comparing them to existing alternatives, a table like https://github.com/jackbsteinberg/std-toast/tree/master/study-group#table-of-comparison could be used to get an overview of how different solutions compare on use cases, features, or other aspect you want to compare.

Good suggestion.

I think this could be used to show how / if the proposal responds to the Use Cases and Requirements, because that document provides examples of the use case as implemented by a number of popular mapping frameworks. I started writing this issue how this proposal could respond to use cases. My intention was / is to ask the community to iterate on the proposal until it responds appropriately to all use cases.

Similar to the toast study table, the Web map use case could be listed down the page and the framework/polyfill across the top. I think the answers might be more complex than yes or no, or at least they could have links to the use case, the example Web maps, proposals, polyfill example page and so on.

Such use case/framework tabulations might also be valuable in determining how the use case should be supported i.e. does it need to be built into the platform as markup, or could it be accomplished by progressive enhancement. Currently almost all map use cases are accomplished through progressive div enhancement :-).

prushforth commented 4 years ago

Picking another specific conversation from your comments, perhaps better to have over on that repo but for context starting here.

map is a legacy element in HTML and is treated as essentially obsolete; it hasn't gotten new features or improvements, other than for web compat and interop.

I can see this POV, but I don't fully understand it. map has definitely been lacking maintainer support, despite requests over the years for it to be made responsive. With the <picture> element it would seem to be possible. I've read platform developers recommend to use svg instead of using map to support the responsive use case, I guess because they do consider map to be legacy.

Reusing legacy elements in HTML for new features is usually not a good choice, as web compat constraints what can be done

Yes. I was hoping to better understand what the constraints are, specifically, in proposing to provide new enhanced behavior for this element set. If it can't be done I guess there's no choice and we'll have to propose something new.

and it can be a source of confusion for web developers.

Instead of implementers potentially spending authors' time, allowing them to investigate how to use a map element only to find that isn't maintained properly, it might be better to maintain the element and keep it up to date. I've read that authors use the map element because it is inherently accessible for simple maps, and it would be less confusing (for authors, imo) to make a "map" element work for both simple and advanced maps instead of introducing a second element set for the advanced scenario and neglecting, or recommending SVG for the simple scenario. I think the latter situation is a potential source of confusion for authors, too.

The web compat constraints cannot be understated. map is used on 1.96% of the web. Even things that seems minor can break pages. If you want to reuse a legacy element for a new feature, you need a very compelling case to make it worth the risk.

map/img/area implement layering using a shared coordinate system. In addition to the priority of constituencies argument, this makes it a logical extension point of HTML for more advanced maps.

We haven't really tackled the entire MapML vocabulary in these comments yet, but in developing that proposed specification, we have identified, for example, new input type(s), new attributes for link, and probably other extensions that I don't recall right now, but for instance: img, head, body etc. These proposals were done for these elements for much the same reason as for the map element: extending existing "semantics" instead of adding new, slightly incompatible forms, which I believe is aligned with various principles in section 2 of the Design Principles.

The proposed extension would create a standard widget that contains controls in a closed shadow root,

This is opening a can of worms. Does this happen for all map elements? If yes, how does that not break web compat?

Some time ago the Web platform community developed a denylist for elements that should not be allowed for developers to add a shadowRoot to, because it might be the case that the Web platform would want someday to add its own shadowRoot there, much like the video element has. I made sure to ask that map be on the denylist for that reason. As a result, I don't think the platform will come across a map element in the wild that has a shadowRoot.

If no, and it's toggled by an attribute, that seems like considerable implementation complexity and an unnecessary source of bugs.

Regardless of whether it is map or some new element, there will need to be a container for browser provided controls, similar to video, I believe.

Malvoz commented 4 years ago

The proposed extension would create a standard <map> widget that contains controls in a closed shadow root,

This is opening a can of worms. Does this happen for all map elements? If yes, how does that not break web compat? If no, and it's toggled by an attribute, that seems like considerable implementation complexity and an unnecessary source of bugs.

I believe the answer is no, it should only apply to map elements with one or more descendant <layer> elements.