osmandapp / OsmAnd

OsmAnd
https://osmand.net
Other
4.4k stars 988 forks source link

Nautical rendering style: 1. Add display of light characteristics in the nautical map 2. Add display of sectorial lights #16894

Open josail opened 1 year ago

josail commented 1 year ago

🚀 feature request

Nautical rendering style: 1. Add display of light characteristics in the nautical map 2. Add display of sctorial lights

Description

1. The display of the nautical rendering style lacks the important information on the light characteristics, which is essential for navigation by the light information. It is readily available in a well structured list of seamark tags as described in : https://wiki.openstreetmap.org/wiki/Seamarks/Lights which can be evaluated in a appropriate text as in the openseamap.

  1. A special display with lines and coloured arcs is required for sectorial lights, as e.g. displayed here: https://wiki.openstreetmap.org/wiki/Seamarks/Sectored_and_Directional_Lights

For 1. and 2. I'm willing to support with limited/beginners experience in the rendering style syntax. Thus the success will require engagement of a second person with more experience on the rendering style programming. I'm much willing to support by feedback on the openseamk syntax (https://wiki.openstreetmap.org/wiki/Seamarks/Light) and intended style (similar to openseamap and international standard INT-1 for nautical charts, https://wiki.openstreetmap.org/wiki/Seamarks/INT-1_Section_P). And by repeated testing of adjustments to the nautical rendering style on an iOS iphone.

Describe the solution you'd like

  1. Text display right of or below the navigational light in the nautical rendering style reflecting INT-1 nomenclature of the light characteristics (e.g. https://wiki.openstreetmap.org/wiki/Seamarks/INT-1_Section_P, www.openseamap.org)

  2. lines, dashed lines and coloured arcs of sectorial lights around the navigational light like in https://wiki.openstreetmap.org/wiki/Seamarks/Sectored_and_Directional_Lights

  1. set up a composed string for text display in the nautical rendering style, some depence on zoom level according to major, minor and other navigational lights rating of importance

  2. draw lines and arcs, graphical features to be implemented, I don't know how to do this

regarding 2.: alternatively as a first step to allow for the information of sectorial navigational lights as a workaround: Option for the display of a short 2nd text indicating the light sectors definitions

sonora commented 1 year ago

Let me cross-link to #16341. It's a frequent request from the boating community, so really quite a bit more than "nice to have" for the practical usability, is my understanding.

josail commented 1 year ago

Yes, I can underline that the seamark light characteristics are essential for nautical chart usability as aids to navigation at night. The flashing time, duration and multiple repetitions are unique and characteristic to clearly identify a seamark and its position in an area with certainty. Currently only the colour is indicated in the OSMAND nautical rendering style. This is insufficient, if you have e.g. three white cardinal lights around a point of danger. Only the flashing characteristic will tell, whether you need to go south or north of a specific light (seamark). So it is essential, and without light characteristics, it's of very limited use for navigation really.

Thus request part 1. is most important, which is just representing light characteristics by INT-1 standardised text display and likely much simpler to implement in the natical rendering. Request part 2. (linked to #16341) would be important at mid-term, too, but there could be a workaround by text only, as a first step.

josail commented 1 year ago

A starting point would be a strategy for the rendering_types.xml (see OsmAnd-resources obf_creation folder) and its seamark section in lines 6742 to 7107:

preprocess rules

  1. seamark:type=VAR seamark:VAR='' Example : seamark:type=dyke seamark=dyke' (main entity)
  2. seamark:VAR:* + seamark:VAR='' Example : seamark:light:colour=red add pair seamark=light (main) seamark:colour=red
    • tag seamark:attached=$(seamark:type)
  3. seamark:*:VAR seamark:VAR Example : seamark:buoy_lateral:category seamark:category (additional)
josail commented 1 year ago

https://github.com/osmandapp/OsmAnd-resources/pull/965 addresses part 1 of the issue as far as possible today

quantenschaum commented 10 months ago

example of how it should/could look like

image

quantenschaum commented 10 months ago

I came here from https://github.com/osmandapp/OsmAnd/issues/16342#issuecomment-1721330495

Sector lights can be displayed by adding dedicated lines to the OBFs for the sector limits and arcs. I wrote a script that does this and adjusted the render style accordingly.

Grab the OBF and style from https://github.com/quantenschaum/mapping/releases/

Using these files it can look like this (with depth contours)

Screenshot_20230918-204835

quantenschaum commented 10 months ago

I would like to propose a change of the light characteristics string. Currently the string does not follow the usual convention, it is similar but not the same. The light characteristics string should look like as described in the INT1 section P, page 82.

image

Main things to change

vshcherb commented 10 months ago
vshcherb commented 10 months ago

Mostly for simple text processing we use rendering_types.xml and only complex we hard code in java (less flexible and not transparent)

quantenschaum commented 10 months ago

Line breaks in a label are not possible?

quantenschaum commented 10 months ago

It seems to be like this: The text in nameTag and nameTag2 are joined into a single string nameTag (nameTag2) that gets wrappend if it exceeds textWrapWidth. The wrapping occurs at space, period, comma, .... Non-breaking spaces (unicode 00A0) are not wrapped. This can be used to force the light characteristics onto the second line and not to break them in the middle.

vshcherb commented 10 months ago

Yes you are right textWrapWidth controls number of charactersl.

josail commented 10 months ago

I can confirm that "nameTag (nameTag2)" is what we could achieve June 2023, when I tried to imitate and allow display of light characteristics in the nameTag2 as far as possible, also by selection of different lightDetail to modify in the app the nameTag2 and allow even some basic light sector text info.

Nevertheless, this is still far from replicating IHO Int-1, which should be the aim for next steps regarding both:

  1. more complete text info on seamark lights

  2. sector light graphical display in colour arcs

My hint for a development in Java required for this certainly is:

The OpenSeaMap and its rendering in OpenNavigationalCharts (ONC) is already able to provide text labels very close to Int-1 and draw arcs according to this standard. A lot of this rendering code is on Github and Sourceforge.

I have had direct contact to the person achieving the Int-1 like display from OSM seamark tags in the ONC rendering. The code behind is very likely also based on java. I am not the right person to continue work in java. But I could forward an e-mail address, if you are interested to get in touch directly for next steps in OsmAnd. Please let me know.

Regards Jo

quantenschaum commented 10 months ago

@josail What is "OpenNavigationalCharts"? Is there a link to the project/code?

It can be achieved as described in https://github.com/osmandapp/OsmAnd/issues/16894#issuecomment-1724202350. Either these extra lines and labels are generated during the creation of the OBFs or this data is generated separately and can be downloaded as extra OBFs by uses who need these features like it is the case with the depth data.

josail commented 10 months ago

Sorry, ONC means OpenNauticalCharts:

http://www.opennauticalchart.org/?permalink=true&zoom=16&lat=54.67277&lon=10.03644&seamarks=true&coordinate_grid=true

josail commented 10 months ago

Regarding the two options: Generation with the OBF in the app, i.e. rendering in the app would be best and allow better for up-to-date nautical chart display, much preferable in my view as active sailor

Several month delay of seamark changes would be a big disadvantage for OsmAnd. The OpenSeaMap and ONC do the rendering of seamark changes within few hours, sometimes even minutes. If a new download could lead to such up-to-date seamark light information then OsmAnd becomes a very valuable Offline source of most recent seamark and light information in the hands of sailors, more up-to-date than paper charts in many and perhaps some very relevant cases.

E.g. when lit seamarks are discontinued/removed or newly installed, marking new dangers for navigation, protected areas, etc. Or when light characteristics change. It does not occur very often, but where and when, then the information on this change is sometimes picked up directly in OSM by the community of sailing mappers and very relevant to navigation.

quantenschaum commented 10 months ago

Yes, if this would happen in the app, this would be the best solution IMHO. But this is currently not possible, whereas generated extra lines in the OBFs are doable, as I have demonstrated.

Concerning updates, OsmAnd does not make OSM data accessible directly (You could use OSM raster tiles, though). It relies on the OBFs that are generated from the OSM data and which are updated monthly. So, generated light sectors in the OBFs could be updated monthly as well. I do not see a problem here. BTW: The ONC tiles are currently not updated!

And generally OSM data is not suitable for navigation anyways without comparing it thoroughly to other sources. I think it can be used for navigation, but you really have to know about the state of the data and data sources.

quantenschaum commented 10 months ago

I adjusted my script such that a light characteristics string is generated and displayed in favor of the standard string. The characteristics of each individual sector are given in the sector arc. This is actually quite usable.

image

josail commented 10 months ago

I perfectly agree.

Good to learn that ONC is not up to date currently. Then something went out of work. Can you confirm at your example check that the OpenSeaMap is updated better, or the same update problem applies? I would then inform the group managing this accordingly, at best based on your OSM seamark data example.

josail commented 10 months ago

Thanks for the Schleimünde example from your updated script. This is a big improvement! I like to motivate you to implement this for use of all in the OsmAnd. I would love to use this and prefer it much over my "symplistic" approach of June. The intuitive understandability of your Int-1 like display will be very helpful.

quantenschaum commented 10 months ago

I am not part of the OsmAnd team, but they are free to use the code from my repository. Would be nice to see this to become part of the OsmAnd nautical charts plugin. But you can already use it.

Does not belong here, but because you asked: AFAIK the services

are essentially the same. Both show map tiles from openseamap.org and they are not up to date.

Bildschirmfoto vom 2023-09-21 09-46-13

quantenschaum commented 9 months ago

Is there a keyword in the render.xml for scaling the icons? Or is there a path in the osmand directory where I can place custom icons and "shaders"?

josail commented 9 months ago

To my knowledge there are two folders

  1. source for creating the icons for rendering Osmand-resources/ icons/...

  2. actual source for rendering: Osmand-resources/ rendering_styles/style-icons/...

The processing from 1. to 2. involves a batch processing of icons, with the

Osmand-resources/ icons/tools/...

which was difficult for me to find out two years ago. I hope I took some notes on it and if i find them, I post them here.

quantenschaum commented 9 months ago

Thanks. But there icons are inside the APK. Is there a location outside the APK, like in the osmand dir on the device, where I can place additional icons (w/o need to recompile)?

josail commented 9 months ago

I think the style-icons data from 2. is taken to the osmand APK or to some hidden dir. At least on my iOS device, which has limited file managment access, I do not have a visible seperate directory in the OsmAnd dir containing the style-icons. So, if it is there, it is hidden. I think there is a chance, that on Android devices, more access to the file structrure is granted (?).

josail commented 9 months ago

Both show map tiles from openseamap.org and they are not up to date.

I got a fast reply on this processing for updating: "... ran out of disk space ... will rebuild that server. This may take a week or so". So I guess we have a good chance to get back to normally fast updates of seamarks in the OpenNavigationalChart and OpenSeaMap in October.

josail commented 9 months ago

Here are my notes for managing and creating new icons: https://github.com/osmandapp/OsmAnd-resources/blob/master/icons/tools/Generating_OSMAND_map_icons.txt

josail commented 9 months ago

@vshcherb and @sonora : Would someone more deeply involved in the java development of OSMAND be able to make good use of the code suggested by quantenschaum ?

I am not part of the OsmAnd team, but they are free to use the code from my repository. Would be nice to see this to become part of the OsmAnd nautical charts plugin. But you can already use it.

It sounds like a perfect "winter project" and likely quantenschaum and certainly myself would be happy testers of such new features.

The rendering of arcs from nautical sectorial light information in offline vector OSM data would be an outstanding and unique feature to my knowledge. Up to now this is only availabel in OpenNautiCalcharts and OpenSeaMap. Both of these can be used on mobile devices only with online connection (which is often missing on the water) or as raster tile download (very data intensive and laboursome for large areas of potential navigational approaches). So there is clearly added value and request for such a new feature in OSMAND, as pointed out by sonora:

Let me cross-link to #16341. It's a frequent request from the boating community, so really quite a bit more than "nice to have" for the practical usability, is my understanding.

vshcherb commented 9 months ago

Yes we monitor @quantenschaum his work progress and so far he is doing pretty well. First of all most of his work is processing contour lines which is really hard work in QGis. Rendering arcs as scripts was my idea as well and it can't be improved with java, we can't render such complicated arcs in rendering engine. If @quantenschaum continue his work, we will have no option as to switch to his work as primary source of data and make nautical plugin.

I feel more & more confident that work is on the right track and today we won't have even good expertise to replicate this work

quantenschaum commented 9 months ago

What about generating OBFs with light sectors as in my proof of concept for several areas of the world and updating them every month as it already happens with the other OBFs provided by OsmAnd? The style for rendering them in the app can be added to the nautical style, it does not interfere with the existing stuff. Then, users interested in nautical charts can download these additional OBFs containing the light sectors, like they already can download additional OBFs with contour lines. You can use my python script or we modify/migrate it to fit the needs. That would be a quite easy solution go get going with light sectors and proper light labels quickly.

vshcherb commented 9 months ago

@quantenschaum we're providing Nautical seamarks already as a separate file and we will keep doing it. So in that case indeed we need to merge as much rendering style features as possible into nautical.render.xml and think about files distribution.

Files could be distributed by our servers if you can generate it monthly and give us permission to distribute it.

quantenschaum commented 9 months ago

@vshcherb You mean the world seamarks? Seamarks are also included in the regular OBFs for each country/region. This means, that if you download an OBF for a country and the world seamarks, you have duplicate data in OsmAnd. If the data in both is the same, it does not matter, but if not (live updates!), objects might appear twice. OK, but this just as a thought.

I do not want to replace any of the existing OBFs, just add another one that optionally can be downloaded. There are about 37500 lights worldwide with a range tag. They can be put into a single OBF of ~15MB.

To make this work in OsmAnd OOTB, the nautical.render.xml has to be adjusted to contain rendering instructions for lightsectors. I created a modified marine style based on the default nautical style. It contains the instructions for the light sectors and other customisations. Search for lightsector to find the relevant sections. Please compare the marine style to the nautical style, maybe you want to incorporate more/all of my changes.

Here is a short description of lightsectors.

vshcherb commented 8 months ago

Yes it should be alternative file I believe cause combination make things difficult. So user should install either your version of obf or standard seamarks

vshcherb commented 8 months ago

You can use https://creator.osmand.net/osm-extract/World_seamarks/ our pbf to generate file with more features built-in

quantenschaum commented 8 months ago

Well, IMHO it is simpler to provide an additional file that contains the light sectors only. I prefer to use the individual country OBFs instead of the world seamarks, because I also want an accurate coastline and features on land. So it is desirable to have the light sectors in a separate file to be able to use them in conjunction with whatever other OBFs. Since the file with worldwide sectors is only ~15MB, it is not worth splitting it into smaller regions.

quantenschaum commented 8 months ago

I set up a cronjob that generates the sectors at least once a week.

http://waddenzee.duckdns.org/download/

:exclamation: only works with the marine rendering style

josail commented 8 months ago

The solution offered by quantenschaum is very convincing. I have tested it successfully on iOS iphone 14 with following OsmAnd result:

Weser:

Helgoland:

quantenschaum commented 6 months ago

Could anyone make use of the light sectors in https://github.com/osmandapp/OsmAnd/issues/16894#issuecomment-1780749215 ?

josail commented 6 months ago

@vshcherb and others quantenschaum and I lack the abilities of putting his solution into work in OsmAnd, but we are both longing very much for it. Could you or some other person of the team put some time on making this important step for the boating community?

As demonstrated step by step manually, all elements described by quantenschaum work perfectly. However, I would not know, how to implement that professionally (not manually, and sufficient in the OsmAnd world, i.e. independent of updating and downloading from a file provided on a server by quantenschaum), so that it becomes available to the whole community.

More correct seamark light display would certainly enhance the usability of the OsmAnd Nautical render map for boaters significantly!

a) I think the basis would be to enhance the creation of the .obf data taking the path provided in the tools of quantenschaum.

b) When it comes to integrating quantenschaums marine rendering style with OsmAnd's nautical rendering style:

Who can support this?

vshcherb commented 6 months ago

Hi @josail we certainly need to take ownership of it. I do appreciate the work you have done and I see that we need to integrate marine style replace our default boat rendering style. Also we need to integrate the data and schedule it on our side to produce monthly with World_seamarks.obf

I have 1 question and 1 concern:

We're certainly willing to take leadership and automate it further. We don't know how to approach it yet (transfer maintenance is not easy unfortunately)


josail commented 6 months ago

Hi Vshcherb, thanks for your engaging reply. The additional work proposed for the 2nd step of improving seamark lights originates from @quantenschaum. Thus I like to forward your question and concern to him/her?

Should the solution proposed above in this issue by quantenshaum really be related to a lot of manual effort, then I can suggest to also consult with @malcolmh, who has implemented java solutions for highly automate, global and "within minutes" OSM data seamark rendering with very similiar result, i.e. including light arcs and INT-1 standard text display next to lights, both for the opennauticalchart and openseamap here: https://github.com/OpenNauticalChart/renderer, see also some more description at https://wiki.opennauticalchart.org/, see the map rendering results corresponding to the three examples illustrated above in this issue:

quantenschaum commented 5 months ago

Yes, please use my work and integrate it into OsmAnd if you like.

data created via cronjob is available at http://waddenzee.duckdns.org/download/

OsmAnd with light sectors vs OSM online

image

quantenschaum commented 2 months ago

Hi, is there any progress on this topic?

It would be nice get my marine rendering style merged (could replace the nautical style) It contains many changes/enhancement to make the chart much more usable as a nautical chart, but is not that easy to merge and it get's worse the more they diverge. I also integrated the nautical depth style into it, because nautical depth are an integral part of a nautical chart (and currently the depth style gets overwritten with every update of OsmAnd).

There also needs to be some additional data (i.e. light sectors, depth contours and areas, depth expositions, etc.) in the OBFs themselves, so the rendering types have to be updated as well.

marine vs nautical

hiddensee

norderney

vshcherb commented 2 months ago

@quantenschaum progress was interrupted by complicate 4.7 release. Just yesterday discussed that we need to do it this / next week at least start. We discussed the main problem here is to create or not create primitive to draw light sectors. So they will be visible on all scales equally but that's quite a task.

quantenschaum commented 2 months ago

Ok, nice. What do you mean by?

the main problem here is to create or not create primitive to draw light sectors. So they will be visible on all scales equally but that's quite a task.

The marine style contains the styling for the light sectors but also contains other modifications, this could be merged now. The creation of the OBFs containing the light sector primitives is a separate task. One could start with a single set of light sectors as in the lightsectors.obf that I provide. This is already very usable. At a later stage one could generate an OBF that contains that contains multiple sets of lightsectors for different zoom level.

quantenschaum commented 2 months ago

Using my script one can create lightsectors of different size using the -a and -f options. Then these sectors are compiled into OBFs at diffrent zoom levels, the OBFs are combined into a single OBF and you get diffrently scaled lightsectors at different zoom levels. This way you can avoid to make any changes to render engine or to generate the geometry for the light sectors in the renderer on the fly.

I just implement this for 3 levels. It looks like this.

https://github.com/osmandapp/OsmAnd/assets/3985599/3384ff77-b234-4fd5-96ce-f50ad5afbfc8

vshcherb commented 2 months ago
quantenschaum commented 2 months ago

What exactly do you mean? Yes, the sector arc is a line consisting of several points which have lat/lon positions, so they "do not know" anything about pixels on screen. By generating multiple arcs for different zoom levels, the arcs are always within the viewport. One can also set the range threshold such that low range lights are only displayed at higher zoom levels.

Is there a possibility to give the sector arc radius in pixels? That would require the arc to be generated by the renderer, in a geometry shader. That would be nice solution. Are geometry shaders available on all current mobile devices? Alternative the arcs have to generate in demand in software (not opengl).

quantenschaum commented 2 months ago

I just updated the lightsectors file with now 4 different levels with different arc radii and thresholds to include lights based on their range. Try it, I think, this is quite usable.

http://waddenzee.duckdns.org/download/

josail commented 1 month ago

quantenschaum, the nautical chart displays on your website look really promising. That's a whole new level of 1st accurate nautical chart display and 2nd so much needed reliable depth information for Netherlands and Germany, where hydrographical offices depth information is openly accessible.

From some displays by Melcolm I saw that he suggested similar displays for UK coast. So I guess, there must be some open access depth information from UKHO, too? and Norway seemed pretty open with its data policy.

I hope other countries hydrographic offices will follow up with open access bathemetry data e.g. in the framework of the international GEBCO Project https://portal.opentopography.org/datasetMetadata?otCollectionID=OT.122023.4326.1 . So far the GEBCO project seems good for open sea areas, but with 500m x 500 m resolution not jet good enough for coastal navigation and navigational obstructions.

I much hope we get the accassible information into a professional INT-1 like OSMAND display with nautical or marine rendering and the tools developed by quantenschaum.