search

OpenOrienteering / mapper

OpenOrienteering Mapper is a software for creating maps for the orienteering sport.
https://www.openorienteering.org/apps/mapper/
GNU General Public License v3.0
401 stars 106 forks source link

Vectorize raster template in Mapper #833

Closed ghost closed 4 years ago

ghost commented 7 years ago

Here is example code, that could be integrated inside Mapper (Qt5-based, licensed under GPLv3)

Expected behaviour

Add possibility for vectorizing raster templates directly in OOM and paste resulted curves in current map (map part).

Configuration

Mapper Version: 0.6.x and above Operating System: All supported

lpechacek commented 7 years ago

A prototype implementation is at https://github.com/lpechacek/mapper/commits/cove_rfc. Packages will be at http://download.opensuse.org/repositories/home:/LPechacek:/branches:/home:/dg0yt:/staging/ once I get the build working.

ghost commented 7 years ago

@lpechacek, good job!

Zerbembasqwibo commented 7 years ago

I tried the Windows version on Windows 7 64 bit.

It worked VERY well, I could quickly digitize an old map (let's not talk about the result though...).

It crashed once when I tried to change background map. It was a map I added after opening the program. I didn't try to reproduce it.

Otherwise:

I "tried this at home" before and marshes are very difficult since they destroy colors. I think anyway that with good knowledge of how to manipulate the picture before vecorizing you can get amizing good results! I am very happy for this branch since I am up to digitize old paper maps.

lpechacek commented 7 years ago

@Zerbembasqwibo, thanks for the encouraging feedback. I'd be happy to learn about the crash if you manage to replicate it.

Ad the progress bars, the line vectorization tool spawns QProgressDialog for every time consuming operation. However, Qt seems to withhold displaying it for a few seconds. Looking into the issue is on my list.

Thinning should not be slow. The algorithm is optimized and was reasonably fast even on the 166 MHz Pentium which ran the first version of this software. There are, however, problems with refreshing the B/W image window content after the operation completes. Moving the image or resizing the dialog window makes Qt repaint the face. This defect is on my list too.

Not sure about the "cancel button" note. Can you be more specific, please?

Zoom can be accessed via a context menu in the image view window. Also mouse wheel and Ctrl + mouse pointer drag work.

Regarding scanned maps with overprint, higher number of color classes (15) and repeated learning/color classification attempts help. Getting good results is usually subject to experimenting and I don't have enough experience in this area yet to provide solid guidance.

dg0yt commented 7 years ago

Regarding the progress dialog: You will need to setMinimumDuration(0), and you will have to call QApplication::processEvents()when updating the progress without return to the event loop. We have a similar situation with printing (https://github.com/OpenOrienteering/mapper/blob/master/src/gui/print_progress_dialog.cpp), including Cancel button.

Zerbembasqwibo commented 7 years ago

Thanks, zoom works fine.

15 colors and use 2 for vectorize, I used just default and used not even all buttons, useful result I would say. I guess even better will be easy to get if yuo spend time on it.

https://pbs.twimg.com/media/C55ffxtWUAAiYGc.jpg

Cancel: the box "Contour vectorizer" doesn't have a cancel / OK button, use only the upper right cross for closing the box.

Q: Areas possible to vectorize, the name of the box indivates otherwise?

2017-03-01 15:25 GMT+01:00 Libor Pecháček notifications@github.com:

@Zerbembasqwibo https://github.com/Zerbembasqwibo, thanks for the encouraging feedback. I'd be happy to learn about the crash if you manage to replicate it.

Ad the progress bars, the line vectorization tool spawns QProgressDialog for every time consuming operation. However, Qt seems to withhold displaying it for a few seconds. Looking into the issue is on my list.

Thinning should not be slow. The algorithm is optimized and was reasonably fast even on the 166 MHz Pentium which ran the first version of this software. There are, however, problems with refreshing the B/W image window content after the operation completes. Moving the image or resizing the dialog window makes Qt repaint the face. This defect is on my list too.

Not sure about the "cancel button" note. Can you be more specific, please?

Zoom can be accessed via a context menu in the image view window. Also mouse wheel and Ctrl + mouse pointer drag work.

Regarding scanned maps with overprint, higher number of color classes (15) and repeated learning/color classification attempts help. Getting good results is usually subject to experimenting and I don't have enough experience in this area yet to provide solid guidance.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenOrienteering/mapper/issues/833#issuecomment-283353563, or mute the thread https://github.com/notifications/unsubscribe-auth/APa4D0VMyBwoFXoWdttMgtJp_Bep-spQks5rhX_ygaJpZM4LyX1Q .

lpechacek commented 7 years ago

@dg0yt, thanks for the hint. The progress bars pop up unconditionally now.

@Zerbembasqwibo, is the image you are referencing a result of the line tracing? If that's the case, can you ahre the scanned template?

Ad area tracing, I deem it possible modulo patterned ones. However, areas in orienteering maps are not that numerous as lines and I currently don't consider implementing this functionality.

Zerbembasqwibo commented 7 years ago

@lpechacek Here is a picture of the scanned map.

https://pbs.twimg.com/media/C58p0vdXEAA6cxr.jpg:large

Area: OK, I understand.

dg0yt commented 7 years ago

I could merge/reimplement the build system parts which optionally add 3rd-party/cove (which I would add later). However, I would add the trigger for this feature to the per-template actions gear of raster templates, similar to the import-and-remove action for vector templates. In this way, there is no need to select a template in the cove code.

lpechacek commented 7 years ago

I like the plan.

Please note that I consider CoVe still alpha level. Among others I'll drop polygon transformation into Bezier curves (duplicates Mapper functionality) and mitigate the dependency on Qt XML module before requesting integration. We should also briefly discuss the code maintenance cooperation model.

dg0yt commented 7 years ago

The short term plan is to isolate your playground (3rd-party/cove in your fork) from the parts of Mapper which I am modifying at the moment (buildsystem in the first place). On OBS I could easily extract 3rd-party/cove from a separate tarball when we are ready to enable it in the unstable build.

In the long term, I would prefer to maintain at least the UI as part of Mapper, for consistency, translation etc. It could still be a quite separate component, similiar to how GDAL is handled. But I didn't study the code enough to know if this feasible, and I don't know your plans how much you want to maintain cove as a standalone tool.

ghost commented 7 years ago

@lpechacek, any news?

Are there working 'unstable' with this feature availiable for testing?

lpechacek commented 7 years ago

Unstable packages based on Mapper 0.7.0 are available at download.opensuse.org (see above). Progress is slow but constant. ;) CoVe is undergoing slimming as some features duplicate Mapper functionality. The goal is to trim it down to a stable core that does line tracing right. Of course usage documentation must be added unless someone develops a "wizzard" mode. Contributions are welcome.

ghost commented 6 years ago

@lpechacek, any news on CoVe development?

lpechacek commented 6 years ago

Yes. Code base modernization is progressing and you can watch it on https://github.com/lpechacek/mapper/commits/cove_rfc. However, I considered OCD file import and export more important topic so vectorization was lower priority in past months.

If there is enough interest in testing, I can provide updated packages and instructions how to use the tool.

ghost commented 6 years ago

@lpechacek, yes, I would be happy to test it!

If you can, provide package for Debian 9.x stretch.

Zerbembasqwibo commented 6 years ago

I use an old version at it works quite ok. Problem is that it's too many alternatives in the process: I cannot figure out how to receive best result. Probably lot's of testing can give suggestipns on how to improve the tool so it's easier for the user.

The mix of contours and open marshes is the most difficult I guess.

fredag 10 augusti 2018 skrev app4soft notifications@github.com:

@lpechacek https://github.com/lpechacek, yes, I would be happy to test it!

If you can, provide package for Debian 9.x stretch.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenOrienteering/mapper/issues/833#issuecomment-412076109, or mute the thread https://github.com/notifications/unsubscribe-auth/APa4DxhMERPMmxBgSPrqLxGlpoauLDykks5uPYT1gaJpZM4LyX1Q .

lpechacek commented 6 years ago

Updated packages are at https://download.opensuse.org/repositories/home:/LPechacek:/branches:/home:/dg0yt:/staging/

Quick how to: 1) Load a raster template into Mapper, make sure it is visible. It is recommended to create a new part for the vectorized lines as they need clean up and symbol assignment. 2) Under template Edit button (at the bottom of the template setup dialog) choose "Vectorize lines" 3) Vectorizer tool starts, presenting the image in 1:1 scale. You can pan and zoom the image. If there is color noise in the picture, due to scanning from paper, apply FIR filter (in the bottom right corner of the window). 4) Switch to Colors tab and perform color classification. The tool reduces image color space into predefined number of buckets to separate differently colored features. For single color images like contour lines from WMS (yes, in my country it is possible to get them), leave the default number of two color buckets. Full color maps digitally generated directly from source 12-15 buckets may be needed. Maps scanned from paper, especially those with overprint will require 15-20 buckets. 5) After color classification finishes, select buttons which correspond to the colors of the features you want to vectorize. The features will be highlighted in red. If the features "bleed" into surrounding ones, try running classification once more (it is a stochastic process) or try increasing the number of buckets. Pre-defining colors found in the picture in the Initial colors dialog may also help. 6) Once you are happy with the separation, proceed to Thinning. In this step there are modifications done to the final black-and-white image. If the lines you want to vectorize are too thin and occasionally broken with small gaps, you may consider applying Dilate operation on them. It will fix small gaps in lines making the continuous. However, if some of the lines are too close (steep hills), these lines will be melted together. 7) The final steps in vectorization are line thinning and the actual vectorization process. Without thinning any feature thicker than one pixel will not be considered for vectorization. Resulting vector lines appear in cyan color overlaid over the image. Clicking Save vectors transfers the resulting lines into currently selected map part. 8) After closing the vectorizer tool window you are back in Mapper. All vectorized lines are selected and can be removed with either Undo or Delete. All the lines carry tag "name=cove", are composed of straight lines and have no symbol assigned.

Further steps: Assign the lines their symbols. You can vectorize not only contour lines but also paths and any line object in general. Join/split lines as needed, remove artifacts. Convert lines to curves and possibly also simplify them. Vectorized lines can be looked up in the map using Find object tool with query "name=cove".

I'd be happy for any volunteer willing to morph the above how to into a proper manual with screenshots, recommendations and full functionality description.

ghost commented 6 years ago

Updated packages are at https://download.opensuse.org/repositories/home:/LPechacek:/branches:/home:/dg0yt:/staging/

Great! Will test it today ;-)

I'd be happy for any volunteer willing to morph the above how to into a proper manual with screenshots, recommendations and full functionality description.

I will try do it this week ;-)

NOTE:

Please, set custom package name, something like openorienteering-mapper-cove that will give possibility to install it side-by-side with official packages produced by dg0yt (i.e. openorienteering-mapper, openorienteering-mapper-master and openorienteering-mapper-unstable)

ghost commented 6 years ago

OK, I just tested openorienteering-mapper-cove_0.8.2.996 -- awesome work, @lpechacek!

cove test

Found few small issues on interface labels, but it mostly usable already. ;-)

Also, I started working on "morphing the above how to into a proper manual with screenshots, recommendations and full functionality description" (think, need week or two to finish it)

cove faq

lpechacek commented 6 years ago

Cool! Thanks for testing and starting work on the howto!

ghost commented 5 years ago

@lpechacek, any news about COVE?

FTR, here is in detail description of the OCAD TPI based contour generalization procedure:

lpechacek commented 5 years ago

@lpechacek, any news about COVE?

I'm maintaining CoVe enabled packages in the aforementioned repository. The packages are currently at v0.8.4 and "just work". @dg0yt what do you think about merging CoVe into Mapper in an "experimental/staging" manner. The code is not all beauty but it's obviously useful for some users.

FTR, here is in detail description of the OCAD TPI based contour generalization procedure: https://www.linkedin.com/pulse/way-fully-automatic-generation-contour-lines-highest-demands-gloor

The recent OCAD enhancement is unrelated to CoVe operation. OCAD applies smoothing to the digital elevation model. CoVe just extracts and interpolates selected lines from a bitmap.

BTW, I've recently generated similarly smooth contour lines from DEM using QGIS 3 with help of FFT Convolution filter plugin.

Contour lines from DEM before smoothing, interval 5m: image

After smoothing the DEM with kernel of 25m radius: image

OCAD's enhancement Gaussian filter with TPI index weighting is a very clever idea and surely gives better results than the above example.

I'm wondering what is @dg0yt's stance on enhancing Mapper with functionality like WMS or DEM processing. I know there is a ticket for WMS and GDAL can be persuaded to do the job for Mapper. On the other hand QGIS already has the necessary WMS and DEM processing functions and Mapper can leverage it by referencing the existing tool in its documentation.

ghost commented 5 years ago

@lpechacek , there was ContourTrace app as part of OpenOrienteering project, but it abandoned many years ago

As I remember, there was idea on adding DEM import into ContourTrace.

About WMS — it reqested since 2012

ghost commented 5 years ago

@lpechacek , near forget that OpenOrienteering also has Lasertool (Laserscan tool) for processing LiDAR data

It would be cool if Lasertool would be improved & implemented in Mapper too.

lpechacek commented 5 years ago

@Symbian9, good point about ContourTrace.

dg0yt commented 5 years ago

what do you think about merging CoVe into Mapper in an "experimental/staging" manner. The code is not all beauty but it's obviously useful for some users.

  • I would merge that in dev.
  • I'm a little bit reluctant with regard to duplicate identifiers such as MapColor
  • I would appreciate the possibility to use an external potrace, as provided by Linux distributions. I would put this package in the Superbuild repository.
  • Copyright and licensing must be clearly document, for all files. (I'm using and maintaining debian copyright files.)

I think the next steps somewhat depend on how much you want to continue to maintain it as a separate tool. (We might build it as a separate program within the Mapper package.) People will report bugs, and we may face build issues. So we have to know where this project has its home.

I'm wondering what is @dg0yt's stance on enhancing Mapper with functionality like WMS or DEM processing. I know there is a ticket for WMS and GDAL can be persuaded to do the job for Mapper. On the other hand QGIS already has the necessary WMS and DEM processing functions and Mapper can leverage it by referencing the existing tool in its documentation.

Mapper (desktop) appeals most to novice and occasional users ATM. From my impression, they need simple tools and good documentation. There is much room for improvement with our documentation (let alone translation), but QGIS is probably too hard for many of these users. With GDAL, we have many of the features already quite close.

However, I wonder where is the best place to put all these data tools in our desktop menu. (I don't even know where to place GPX recording.) A "data" menu? An "extra" menu?

dg0yt commented 5 years ago

@Symbian9 wrote:

Also, I started working on "morphing the above how to into a proper manual with screenshots, recommendations and full functionality description" (think, need week or two to finish it)

Any news?

Zerbembasqwibo commented 5 years ago

"Advanced" menu?

torsdag 31 januari 2019 skrev Kai Pastor notifications@github.com:

what do you think about merging CoVe into Mapper in an "experimental/staging" manner. The code is not all beauty but it's obviously useful for some users.

  • I would merge that in dev.
  • I'm a little bit reluctant with regard to duplicate identifiers such as MapColor
  • I would appreciate the possibility to use an external potrace, as provided by Linux distributions. I would put this package in the Superbuild repository.
  • Copyright and licensing must be clearly document, for all files. (I'm using and maintaining debian copyright files.)

I think the next steps somewhat depend on how much you want to continue to maintain it as a separate tool. (We might build it as a separate program within the Mapper package.) People will report bugs, and we may face build issues. So we have to know where this project has its home.

I'm wondering what is @dg0yt https://github.com/dg0yt's stance on enhancing Mapper with functionality like WMS or DEM processing. I know there is a ticket for WMS and GDAL can be persuaded to do the job for Mapper. On the other hand QGIS already has the necessary WMS and DEM processing functions and Mapper can leverage it by referencing the existing tool in its documentation.

Mapper (desktop) appeals most to novice and occasional users ATM. From my impression, they need simple tools and good documentation. There is much room for improvement with our documentation (let alone translation), but QGIS is probably too hard for many of these users. With GDAL, we have many of the features already quite close.

However, I wonder where is the best place to put all these data tools in our desktop menu. (I don't even know where to place GPX recording.) A "data" menu? An "extra" menu?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenOrienteering/mapper/issues/833#issuecomment-459251835, or mute the thread https://github.com/notifications/unsubscribe-auth/APa4Dzl3gRpX2TlJ4Bh7m1sIdgoed-35ks5vIqMXgaJpZM4LyX1Q .

ghost commented 5 years ago

"Advanced" menu?

Or "GIS Tools", as GIS -- is advanced mapping.

Zerbembasqwibo commented 5 years ago

"GIS tool" is good.

torsdag 31 januari 2019 skrev app4soft notifications@github.com:

"Advanced" menu?

Or "GIS Tools", as GIS -- is advanced mapping.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenOrienteering/mapper/issues/833#issuecomment-459266242, or mute the thread https://github.com/notifications/unsubscribe-auth/APa4D3s6QPk6r4i45czqCPyPmCg6LFOCks5vIq-TgaJpZM4LyX1Q .

jmacura commented 5 years ago

as GIS -- is advanced mapping.

It certainly isn't. And I don't believe that an unknown acronym will causes a "novice and occasional users" to use such tools, also unknown to them, and not to run away screamingly. I believe "Data tools" describes it quite well already.

ghost commented 5 years ago

I believe "Data tools" describes it quite well already.

Then -- "GIS data".

lpechacek commented 5 years ago

Gentlement, would you mind moving the extended GIS discussion to corresponding tickets? We do have a WMS one, there is another for laser scan tools and in addition there is the ideas incubator. Thanks for your understanding.

ghost commented 5 years ago

@lpechacek what you think about implementing PolyVectorization into cove?

lpechacek commented 4 years ago

@Symbian9, yes, it seems to make sense. I've created GH-1466 for tracking. Thanks for the suggestion!

lpechacek commented 4 years ago

I would appreciate the possibility to use an external potrace, as provided by Linux distributions. I would put this package in the Superbuild repository.

@dg0yt, potrace code had to be modified for tracing open paths. Refer to https://github.com/lpechacek/mapper/blob/cove/3rd-party/cove/potrace/potrace-1.6.diff for the modifications made. In my opinion, it makes more sense to rewrite Potrace's trace.c into C++ and merge it into CoVe, rather than push the above changes into mainline Potrace. Replacing Potrace with another implementation would be also an alternative.

dg0yt commented 4 years ago

potrace code had to be modified for tracing open paths. Refer to https://github.com/lpechacek/mapper/blob/cove/3rd-party/cove/potrace/potrace-1.6.diff for the modifications made. In my opinion, it makes more sense to rewrite Potrace's trace.c into C++ and merge it into CoVe, rather than push the above changes into mainline Potrace. Replacing Potrace with another implementation would be also an alternative.

I'm fine with local modifications. What I did for Qt printsupport (and recently started for QBezier), is adding the pristine code to a separate branch in this repository (for integration of future upstream changes), and do modifications in a separate branch (which may occassionally merge the pristine branch).

In this way, you have upstream and the local fork properly mapped to git branches, with all features of git for logs, diffs, etc.