Finish work on iOS .framework for libosmscout

frogg commented 6 years ago

As @vyskocil mentioned in #475 there is still some work to do for the iOS .framework project.

I also realized that the headers are missing from the framework and that indeed the sample code compiled because it look for the headers in the sources... I've to work more on this, I already look on it but not finished. The size might be explained because the frameworks contains also bitcode along the 64bits compiled code, some frameworks are not useful (maybe) on mobile like libosmscoutimport

And also:

The import library is something that is definitely not designed to be run on mobile. IO and memory requirements are too high for this.

Framstag commented 6 years ago

Which headers give you problems? I do not know anything about building frameworks under Mac OS, there are some special headers like Config.h and XXXFeatures.h that are generated on the fly by cmake/autoconf/meson and thus have to be handled especially.

Is there anything I can help with, without in deep knowledge of Mac OS frameworks?

vyskocil commented 6 years ago

I worked on that and found what still need to be done to get proper frameworks for Mac OSX and iOS :

Actually the headers are copied flat in the Headers frameworks directory without subdir structure like util/, system/, routing/... which break compilation
With Apple frameworks it is not possible to "merge" osmscout include dirs from several libraries (libosmscout, libosmscoutmap, libosmscoutmapiosx,...) ! The framework name (osmscout, osmscoutmap,...) is used as the first dir level when including a file for example to include Database.h from libosmscout : #include <osmscout/Database.h> works but for a .h file from libosmscoutmap you have to use osmscoutmap/ for example : #include <osmscoutmap/StyleConfig.h>.

I tried to apply this manually and could use the resulting frameworks in a project

Framstag commented 6 years ago

Hello @vyskocil: Some questions:

We are still speaking of building a OSX framwork using the cmake files?
Which part of the cmake files is the relevant code? How do you trigger the creation of the framework build? For which library did you test it?
Do you have relevant documentation from Apple and/or (better?) cmake so that I can take a look how things should work?
Regarding includes: Does that mean, that we need to the git repository directory structure below the include directories? That would be a rather big change :-/ Is there some "official" documentation from Apple that requires this or is this "just" a deficiency of cmake?
If you have/had time, what would thus be your next steps?
And....is the autoconf still required for OS X/iOS builds? Some changes would likely be easier, if we could drop one build system before fixing/improving things...

Framstag commented 6 years ago

I did some more analysis on my own:

Buildings frameworks seems to already been activated for the cmake build if under APPLE as described here: https://cmake.org/cmake/help/v3.10/prop_tgt/FRAMEWORK.html
As @vyskocil already described framework building is in principle running, but the resulting framework is (likely) disfunctional because header hierarchy is flattened.
Flattening is obviously the behaviour of XCode itself.
There are though workarounds for XCode: https://stackoverflow.com/questions/3807241/keeping-directory-structure-when-creating-frameworks-in-xcode http://www.cocoabuilder.com/archive/xcode/264731-subdirectories-in-framework-public-header.html
Other people have the problem with cmake, too: https://gitlab.kitware.com/cmake/cmake/issues/16739

Looks like we should try to fix the headers hierarchy by postprocessing the directory using install afterwards?

Framstag commented 6 years ago

One could also try to experiment with https://cmake.org/cmake/help/v3.4/prop_sf/MACOSX_PACKAGE_LOCATION.html

Framstag commented 6 years ago

http://www.kwwidgets.org/Bug/view.php?id=7941 points to https://cmake.org/pipermail/cmake/2008-August/023314.html which also suggest to use MACOSX_PACKAGE_LOCATION to move headers to correct subdirectories.

Framstag commented 6 years ago

I have no problem with splitting the HEADER_FILES definition into multiple parts (or make it a composition of multiple parts) to make this easier.

If this works, it will likely not fix all the mentioned problems, root directories still have to be unique, but perhaps this can be compensated in cmake at "framework usage as client location" again, too? One solution after the other :-)

Framstag commented 6 years ago

I tested the MACOSX_PACKAGE_LOCATION approach and while there are articles and mails that claim, that it works, it did not work. It looks like it is just ignored. Help!

I'll push the list of headers per directory soon anyway.

vyskocil commented 6 years ago

I tried something that works but seems a bit ugly : copy (manually) all the headers keeping the tree structure from libosmscout/include, libosmscout-map/include and libosmscout-map-iOSX/include to OSMScout.framework/Headers and then use the resulting OSMScout.framework along unmodified OSMScoutMap.framework and OSMScoutMapIOSX.framework.

Framstag commented 6 years ago

Yes, I understood this. But I would like a solution that does not need build magic. Question: Can frameworks share a common top level include directory? Or must not be there a common include directory?

E.g., hierarchical:

<osmscout>
<osmscout/import>
<osmscout/map>
<osmscout/map/agg>
...

vs. flat

<osmscout>
<osmscout-import>
<osmscout-map>
<osmscout-map-agg>
...

As far as I understood, Mac OS frameworks require the second approach. How do other libraries do this? Thinking of OpenGL, Gtk which consists of multiple libraries which should embed themselves into a common file structure.

I find a hierarchical solution nicer, but could live with a flt structured solution, too. But I want to definitely avoid a solution, where includes are different for different platforms.

vyskocil commented 6 years ago

Here is the OpenGL and GLUT frameworks structure :

$ ls /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/OpenGL.framework/Headers

CGLContext.h        CGLIOSurface.h      CGLTypes.h      gl.h            glext.h         gltypes.h       gluMacro.h
CGLCurrent.h        CGLMacro.h      OpenGL.h        gl3.h           gliContext.h        glu.h
CGLDevice.h     CGLRenderers.h      OpenGLAvailability.h    gl3ext.h        gliDispatch.h       gluContext.h

$ ls /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/GLUT.framework/Headers

copy.h      glsmap.h    glut.h      glutf90.h   gutil.h     port.h      segment.h   tube_gc.h
extrude.h   glsmapint.h glutbitmap.h    glutstroke.h    intersect.h rot.h       tube.h      vvector.h

vyskocil commented 6 years ago

By default XCode use a "header map" to resolve inclusion of headers and don't need subdirs in the Headers directory of a framework but I had difficulties to make this work because some osmscout include files conflicts with standard includes like Math.h

Framstag commented 6 years ago

Renaming headers that are in conflict should not be a problem. I also checked the existing frameworks on my system, while most have a flat header structure, not all have them, some seem to be hierarchical. A better example would be possibly Qt5. But while structured on disk, include paths are used in a way, that in result, there is no hierarchy in the actual includes.

Framstag commented 6 years ago

I now got MACOSX_PACKAGE_LOCATION working. Files with this property, must not be part of the list of headers passed to the framework directly via PUBLIC_HEADER.

I now have the problem that the absolute import of headers within the library does not seem to work.

Framstag commented 6 years ago

See mail to the mailing list for status update.

vyskocil commented 6 years ago

I wrote and use the following script (a dirty hack...) to copy the headers to the right locations in the frameworks :

#!/bin/sh                                                                                                                                                                                                                                               

if [ ! $# -eq 2 ]; then
   echo usage: cpHeaders fromDir toDir
fi

DIRFROM=$1
DIRTO=$2

DIRFROMOSMSCOUT="$DIRFROM/libosmscout/OSMScout.framework/Headers"
DIRTOOSMSCOUT="$DIRTO/OSMScout.framework/Headers"

declare -a ROUTING=(Route.h RouteData.h RouteNode.h RouteNodeDataFile.h RoutePostprocessor.h RoutingDB.h RoutingProfile.h RoutingService.h AbstractRoutingService.h SimpleRoutingService.h MultiDBRoutingService.h DBFileOffset.h TurnRestriction.h Mul\
tiDBRoutingState.h)

mkdir -p "$DIRTOOSMSCOUT/routing"
for file in "${ROUTING[@]}"; do
    echo copying "$DIRFROMOSMSCOUT/$file to $DIRTOOSMSCOUT/routing/$file"
    cp "$DIRFROMOSMSCOUT/$file" "$DIRTOOSMSCOUT/routing/$file"
done

declare -a UTIL=(Breaker.h Cache.h Color.h CmdLineParsing.h Distance.h Exception.h File.h FileScanner.h FileWriter.h HTMLWriter.h GeoBox.h Geometry.h Logger.h Magnification.h MemoryMonitor.h NodeUseMap.h Number.h NumberSet.h Parsing.h Progress.h P\
rojection.h StopClock.h String.h StringMatcher.h TagErrorReporter.h Tiling.h TileId.h Transformation.h WorkQueue.h)

mkdir -p "$DIRTOOSMSCOUT/util"
for file in "${UTIL[@]}"; do
    echo copying "$DIRFROMOSMSCOUT/$file to $DIRTOOSMSCOUT/util/$file"
    cp "$DIRFROMOSMSCOUT/$file" "$DIRTOOSMSCOUT/util/$file"
done

declare -a SYSTEM=(Assert.h Compiler.h Math.h OSMScoutTypes.h SSEMath.h SSEMathPublic.h)

mkdir -p "$DIRTOOSMSCOUT/system"
for file in "${SYSTEM[@]}"; do
    echo copying "$DIRFROMOSMSCOUT/$file to $DIRTOOSMSCOUT/system/$file"
    cp "$DIRFROMOSMSCOUT/$file" "$DIRTOOSMSCOUT/system/$file"
done

declare -a OST=(Parser.h Scanner.h)

mkdir -p "$DIRTOOSMSCOUT/ost"
for file in "${OST[@]}"; do
    echo copying "$DIRFROMOSMSCOUT/$file to $DIRTOOSMSCOUT/ost/$file"
    cp "$DIRFROMOSMSCOUT/$file" "$DIRTOOSMSCOUT/ost/$file"
done

declare -a ROOT=(CoreImportExport.h Area.h AreaAreaIndex.h AreaDataFile.h AreaNodeIndex.h AreaWayIndex.h Coord.h CoordDataFile.h CoverageIndex.h BoundingBoxDataFile.h TypeDistributionDataFile.h Database.h DataFile.h BasemapDatabase.h DebugDatabase\
.h GeoCoord.h GroundTile.h Intersection.h Location.h LocationIndex.h LocationService.h LocationDescriptionService.h Navigation.h Node.h NodeDataFile.h NumericIndex.h ObjectRef.h OptimizeAreasLowZoom.h OptimizeWaysLowZoom.h Path.h Pixel.h Point.h P\
OIService.h ObjectVariantDataFile.h SRTM.h Tag.h TypeConfig.h TypeFeature.h TypeFeatures.h TypeInfoSet.h FeatureReader.h OSMScoutTypes.h WaterIndex.h Way.h WayDataFile.h TextSearchIndex.h)

for file in "${ROOT[@]}"; do
    echo copying "$DIRFROMOSMSCOUT/$file to $DIRTOOSMSCOUT/$file"
    cp "$DIRFROMOSMSCOUT/$file" "$DIRTOOSMSCOUT/$file"
done

cp "$DIRFROM/libosmscout/include/osmscout/CoreFeatures.h" "$DIRTOOSMSCOUT/CoreFeatures.h"

#                                                                                                                                                                                                                                                       
# libosmscout-map                                                                                                                                                                                                                                       
#                                                                                                                                                                                                                                                       

DIRFROMOSMSCOUTMAP="$DIRFROM/libosmscout-map/OSMScoutMap.framework/Headers"
DIRTOOSMSCOUTMAP="$DIRTO/OSMScoutMap.framework/Headers"

declare -a OSSMAP=(Parser.h Scanner.h)

mkdir -p "$DIRTOOSMSCOUTMAP/oss"
for file in "${OSSMAP[@]}"; do
    echo copying "$DIRFROMOSMSCOUTMAP/$file to $DIRTOOSMSCOUTMAP/oss/$file"
    cp "$DIRFROMOSMSCOUTMAP/$file" "$DIRTOOSMSCOUTMAP/oss/$file"
done

declare -a ROOTMAP=(MapImportExport.h LabelLayouter.h MapPainter.h MapParameter.h MapService.h LabelProvider.h Styles.h StyleDescription.h StyleConfig.h StyleProcessor.h DataTileCache.h MapTileCache.h MapPainterNoOp.h)

for file in "${ROOTMAP[@]}"; do
    echo copying "$DIRFROMOSMSCOUTMAP/$file to $DIRTOOSMSCOUTMAP/$file"
    cp "$DIRFROMOSMSCOUTMAP/$file" "$DIRTOOSMSCOUTMAP/$file"
done

cp "$DIRFROM/libosmscout-map/include/osmscout/MapFeatures.h" "$DIRTOOSMSCOUT"

#                                                                                                                                                                                                                                                       
# Hack !!!                                                                                                                                                                                                                                              
#                                                                                                                                                                                                                                                       
cp "$DIRTOOSMSCOUTMAP"/*.h "$DIRTOOSMSCOUT"

#                                                                                                                                                                                                                                                       
# libosmscout-map-iOSX                                                                                                                                                                                                                                  
#                                                                                                                                                                                                                                                       

DIRFROMOSMSCOUTMAPIOSX="$DIRFROM/libosmscout-map-iOSX/OSMScoutMapIOSX.framework/Headers"
DIRTOOSMSCOUTMAPIOSX="$DIRTO/OSMScoutMapIOSX.framework/Headers"

echo "copying $DIRFROMOSMSCOUTMAPIOSX/MapPainterIOS.h to $DIRTOOSMSCOUTMAPIOSX/MapPainterIOS.h"
cp "$DIRFROMOSMSCOUTMAPIOSX/MapPainterIOS.h" "$DIRTOOSMSCOUTMAPIOSX/MapPainterIOS.h"

#                                                                                                                                                                                                                                                       
# Hack !!!                                                                                                                                                                                                                                              
#                                                                                                                                                                                                                                                       
cp "$DIRTOOSMSCOUTMAPIOSX"/*.h "$DIRTOOSMSCOUT"

What it does is mainly :

copy the public headers keeping the directory structure
copy the headers from libosmscout-map and libosmscout-map-iOSX to the Headers directory from libosmscout framework because they are not found elsewhere (because libosmscout-map define osmscoutmap/ and not osmscout and libosmscout-map-iOSX define osmscoutmapiosx)

I could find a way to copy files with directory structure using CMake...

Framstag commented 6 years ago

I di d cmake to copy the directory structure, but I did not get a test code to build against the library headers because of the way the includes were used.

I tested it using the osmscout-test library, I can check in my changes, but it will break framework builds (or builds against the framework) until someone with better XCode unterstanding fixes it. IMHO currently nobody uses framework builds so this should not be a problem. If things work out for the osmscout-test library we may find ways to modify the other libraries one by one, too.

Framstag commented 6 years ago

@vyskocil I now pushed my changes to the libosmsocut-test library. I did the changes a while ago, but as far as I remeber teh cmake build did create a frfamework with the right directory structure. Problem though was, that building against the library using XCode did not succeed. I did fiddle around using "..." included instead of <...>and also relative includes but that did not help. I'm not sure I have setup the OS X project. So this would be something you might take a look at. It is possible that we need an additional "collection" header that includes everything and possibly fixing the search path problem. If this is the case, I could live with that, too.

vyskocil commented 6 years ago

@Framstag I just looked at it and it seems that the headers structure in the OSMScoutTest framework is still flat :

$ ls -alR libosmscout-test/OSMScoutTest.framework/Headers/
total 56
drwxr-xr-x  7 vyskocil  staff   224 Aug  2 02:00 .
drwxr-xr-x  5 vyskocil  staff   160 Aug  2 02:00 ..
-rw-r--r--  1 vyskocil  staff  2807 Aug  2 02:00 Parser.h
-rw-r--r--  1 vyskocil  staff  3034 Aug  2 02:00 PreprocessOLT.h
-rw-r--r--  1 vyskocil  staff  5485 Aug  2 02:00 RegionList.h
-rw-r--r--  1 vyskocil  staff  4869 Aug  2 02:00 Scanner.h
-rw-r--r--  1 vyskocil  staff  2272 Aug  2 02:00 TestImportExport.h

Framstag commented 6 years ago

I'll check.

Framstag commented 6 years ago

@vyskocil

Works for me:

Dose:build tim$ ls -alR libosmscout-test/osmscout-test.framework/Headers/
total 32
drwxr-xr-x  6 tim  staff   192 16 Aug 20:05 .
drwxr-xr-x  5 tim  staff   160 16 Aug 20:05 ..
-rw-r--r--  1 tim  staff  3016 16 Aug 20:05 PreprocessOLT.h
-rw-r--r--  1 tim  staff  5476 16 Aug 20:05 RegionList.h
-rw-r--r--  1 tim  staff  2277 16 Aug 20:05 TestImportExport.h
drwxr-xr-x  4 tim  staff   128 16 Aug 20:05 ost

libosmscout-test/osmscout-test.framework/Headers//ost:
total 24
drwxr-xr-x  4 tim  staff   128 16 Aug 20:05 .
drwxr-xr-x  6 tim  staff   192 16 Aug 20:05 ..
-rw-r--r--  1 tim  staff  2787 16 Aug 20:05 Parser.h
-rw-r--r--  1 tim  staff  4869 16 Aug 20:05 Scanner.h

Since your files are rather old and the name of the framework is different, I assume that you are possibly not on current master (and yes, I do not not have local changes ;-))?

vyskocil commented 6 years ago

@Framstag you are right (as almost always ;-) I forgot the cleanup the build directory and update to the latest sources... Is there also a test application that use osmscout-test.framework ?

Framstag commented 6 years ago

Hello Vladimir,

@Framstag https://github.com/Framstag you are right (as almost always ;-) I forgot the cleanup the build directory and update to the latest sources... Is there also a test application that use osmscout-test.framework ?

I generate a OS X App stub using XCode and just included one of the headrs for testing purposes. That already failed. Since your knowledge of OS X is better than my, you should create such application yourself. It is very likely, that I might have done something wrong in the setup, so it should not get checked in.

The CLI test LocationServiceTest.cpp also makes use of the test library.

-- Gruß... Tim

Framstag commented 5 years ago

@vyskocil The issue is open for a while now. While I have a Mac I do not actually use it for Mac specific work. So how to continue?

vyskocil commented 5 years ago

I restarted working on this, for now I updated the CMakeLists.txt to build apple frameworks keeping the directory structure for the include files. I also renamed the frameworks to be all lowercases and add '-' between words, ex: OSMScoutMap become osmscout-map. The last thing to do is to "merge" the include files from the frameworks under osmscout/ There are two options :

try to use a "umbrella framework" but I don't know much about this and this seems to not be supported by CMake ?
rename the include directories of the osmscout-map, osmscout-map-*, osmscout-import,... modules ie instead of include/osmscout/... use include/osmscout-modulename/ for example: libosmscout-map/include/osmscout/ became libosmscout-map/include/osmscout-map/

I don't know what is the best solution ?

vyskocil commented 3 years ago

I just made a pull request https://github.com/Framstag/libosmscout/pull/1025 for the first step : keeping the directory structure for osmscout headers

frogg commented 3 years ago

Hello! Just wanted to ask what currently the suggested way is to compile libosmscout for iOS. I‘ve cloned the project but I‘m a bit stuck what the next steps would be. I ran cmake -G "Xcode" .. Then, I noticed there's a Xcode project available under /Apple/OSMScoutOSX. However, the resulting project is not buildable and contains many errors, mostly complaining about the C++ syntax. So I assume there must be a problem with either the C++ version or I am missing some important steps.

Examples for error messages I see:

/libosmscout/include/osmscout/Database.h:431:75: Invalid suffix on literal; C++11 requires a space between literal and identifier

/libosmscout/include/osmscout/util/ObjectPool.h:102:12: No template named 'scoped_lock' in namespace 'std'

/libosmscout/include/osmscout/Database.h:456:78: Expected ')'

vyskocil commented 3 years ago

Hello @frogg ! To compile libosmscout for iOS I use the following steps :

cd libosmscout
mkdir build.ios
cd build.ios
~/Bin/cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/iOS.cmake -DMARISA_INCLUDE_DIRS=~/Dev/marisa/lib  ..
make

frogg commented 3 years ago

Hello @frogg ! To compile libosmscout for iOS I use the following steps :
cd libosmscout
mkdir build.ios
cd build.ios
~/Bin/cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/iOS.cmake -DMARISA_INCLUDE_DIRS=~/Dev/marisa/lib  ..
make

Thanks for the quick reply! When I execute the cmake command similar to how you provided it, the whole thing fails with this issue:

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
UIKIT
    linked by target "OSMScoutMapIOSX" in directory /Users/…/libosmscout/libosmscout-map-iosx

Any idea what that could be?

vyskocil commented 3 years ago

Thanks for the quick reply! When I execute the cmake command similar to how you provided it, the whole thing fails with this issue:
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
UIKIT
    linked by target "OSMScoutMapIOSX" in directory /Users/…/libosmscout/libosmscout-map-iosx
Any idea what that could be?

It seems that CMake don't find the UIKIT iOS framework which is standard... Are you sure that XCode command line tools are properly installed ? Or maybe CMake is not properly configured ?

frogg commented 3 years ago

It seems that CMake don't find the UIKIT iOS framework which is standard... Are you sure that XCode command line tools are properly installed ? Or maybe CMake is not properly configured ?

Thanks, that was actually it! I had multiple versions of Xcode + the new Xcode betas installed, so I guess cmake was a bit confused which one it should pick 😬

That brought me one step further, great! Now I am able to run cmake without any problems, however when I try to make the project, I run into the next problem:

[ 36%] Building CXX object libosmscout-import/CMakeFiles/OSMScoutImport.dir/src/osmscout/import/Preprocess.cpp.o
In file included from /Users/…/Developer/libosmscout/libosmscout-import/src/osmscout/import/Preprocess.cpp:44:
/Users/…/Developer/libosmscout/libosmscout-import/include/osmscout/import/PreprocessPBF.h:38:12: fatal error: 'osmscout/import/pbf/fileformat.pb.h' file not found
  #include <osmscout/import/pbf/fileformat.pb.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

I assume it‘s trying to find the protobuf headers here, I have protobuf installed via brew if that‘s important.

vyskocil commented 3 years ago

To compile libosmscout-import lib and tools that are needed to import OpenstreetMap data in PBF format and generate OSMScout database used by the lib you need to install the Protobuf library from google (https://gist.github.com/madelinegannon/393fe7b1a24664abd0976467162f5f14). This is only needed when you compile OSMScout for the Mac because building OSMScout databases on iPhone would be very difficult for medium to large territories !

frogg commented 3 years ago

ok, I see! Right now, I am only trying to get the iOS framework to run, do I still need it then?

vyskocil commented 3 years ago

No you don't have to. In my case the cmake command for iOS target display this :

...
-- import library:                  ON
--  - libxml2 support:              TRUE
--  - protobuf support:             FALSE
...

It still compile the import lib and tools but without protobuf support. Maybe you should deactivate the compilation of the whole import package but I didn't have to do that before and don't know exactly how to do it.

frogg commented 3 years ago

Interestingly, mine has it disabled as well, however still complaining about the #include <osmscout/import/pbf/fileformat.pb.h>:

…
-- import library:                  ON
--  - libxml2 support:              TRUE
--  - protobuf support:             FALSE
…

vyskocil commented 3 years ago

It seems to be a dependency detection bug... In my case I have the protobuf package installed for OSX, so the includes are there also for iOS... Maybe you should open a ticket.

frogg commented 3 years ago

Thanks for the suggestion, I‘ve opened a new issue for that. As a workaround I habe now installed protobuf according to the instruction provided by you earlier and I was able to go one step further:

-- Found Protobuf: /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.5.sdk/usr/lib/libprotobuf.tbd (found version "3.6.1")
-- Protobuf library found, but protoc compiler is missing

As you can see, it is still complaining that the protoc compiler is missing.

However, the build files are successfully generated: -- Build files have been written to: /Users/…/Developer/libosmscout/build.ios

When I now try to run make in the iOS build folder, the whole thing terminates with this problem:

[ 47%] Building CXX object libosmscout-import/CMakeFiles/OSMScoutImport.dir/src/osmscout/import/Preprocess.cpp.o
In file included from /Users/…/Developer/libosmscout/libosmscout-import/src/osmscout/import/Preprocess.cpp:44:
/Users/…/Developer/libosmscout/libosmscout-import/include/osmscout/import/PreprocessPBF.h:38:12: fatal error: 'osmscout/import/pbf/fileformat.pb.h' file not found
  #include <osmscout/import/pbf/fileformat.pb.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [libosmscout-import/CMakeFiles/OSMScoutImport.dir/src/osmscout/import/Preprocess.cpp.o] Error 1
make[1]: *** [libosmscout-import/CMakeFiles/OSMScoutImport.dir/all] Error 2
make: *** [all] Error 2

Framstag commented 3 years ago

Hello Everybody,

just a node that we have an "official" iOS build, see https://github.com/Framstag/libosmscout/runs/3261167254?check_suite_focus=true

See https://github.com/Framstag/libosmscout/blob/master/.github/workflows/build_and%20test_on_ios.yml for the basic github actions build script.

As one can see, we do install protobuf and protobuf-c. As @vyskocil already pointed out, this should not really be needed, since there is no point to use the import library on iOS devices.

If one looks at the android build (https://github.com/Framstag/libosmscout/blob/master/.github/workflows/build_on_ubuntu_18_04_qt_android.yml), it should be possible to drop building the library by passing "-DOSMSCOUT_BUILD_IMPORT=OFF". Nevertheless, the bug is correct, as we should be consistent in our dependency detection and handling. The Android build also shows other options that might simplify the build and make is smaller and more tailed.

frogg commented 3 years ago

Thanks, @Framstag! With the help of the Github Action I was able to run the cmake build. In my build folder I now have different frameworks, such as OSMScout.framework, OSMScoutMap.framework, OSMScoutMapIOSX.framework. Where do I go from here? Is there any documentation on how to initialize a Map View or even a sample app? (I couldn't get the OSMScoutOSX Xcode project in the Apple directory to run…).

I know that I was able to do all of this back in 2017 and 2018 when I first played around with this library, but now it seems a lot has changed, so please excuse all my basic questions. I just want to get this up and running on my device 😬

vyskocil commented 3 years ago

You should now copy OSMScout.framework, OSMScoutMap.framework and OSMScoutMapIOSX.framework to the iOS Framework directory of the OSMScoutOSX sample app (OSMScoutOSX/OSMScoutiOS/Frameworks), open the project with XCode, select OSMScoutIOS as target and build it ! But it should fail because the work on the iOS frameworks is not finished but there is a merge request that is awaiting : https://github.com/Framstag/libosmscout/pull/1106

vyskocil commented 3 years ago

@frogg you could also build the frameworks for the iOS simulator using the following cmake command line :

~/Bin/cmake -DIOS_PLATFORM=SIMULATOR64 -DCMAKE_TOOLCHAIN_FILE=../cmake/iOS.cmake ..

Then you could test the OSMScoutIOS app on the iPhone simulator

frogg commented 3 years ago

Perfect, manually moving over the frameworks to the Xcode Project did the trick :) Thanks for your ongoing help @Framstag and @vyskocil! I now have it up and running 🎉

IMG_42BE181C84D8-1

Framstag commented 3 years ago

Tiled rendering and auto-sizing labels do not work well together. Labels are only rendered in a tile, if the data for the label is part of the tile. If the label is much bigger than the actual object, the label data will not appear in a tile. To reduce this effect, you need to offer the tile a data area that is bigger than the rendered tile. This is a common problem of tile rendering and not a problem of libosmscout itself.

I'm sure the client-qt library does handle this, and likely @Karry can give some general hints ho to do similar in other backends.

Having a quick looker at the Tiler demo, it looks like it adds some data (nodes and areas with labels) of the surrounding 8 tiles to the dataset of the current "to render tile" to avoid the above effect.

frogg commented 3 years ago

Having a quick looker at the Tiler demo, it looks like it adds some data (nodes and areas with labels) of the surrounding 8 tiles to the dataset of the current "to render tile" to avoid the above effect.

this actually sounds like a reasonable improvement that I could implement for the iOS demo in a PR. I just need some context where to start.

vyskocil commented 3 years ago

@frogg maybe you should start porting the tiler demo concepts into the iOS/OSX sample app. I have some thoughts about this project :

I think you should have some tile cache to store rendered tiles because the use case when providing custom tiles to Apple MapKit component is the following :
- MapKit ask you to render a tile at (x,y) for a given zoomLevel
- You use OSMScout to render a meta-tile centered at (x,y), for example the tile at (x,y) and 8 tiles around it at (x-1, y-1), (x,y-1),....(x+1,y+1)
- Then Mapkit may ask you to render tile (x+1,y) or another of the already previously rendered tile, so if you cached it you could simply return the cached tile
when rendering a meta-tile you should extend it with a border around the tiles to allow some overlapping of the meta tiles and reduce cut labels. This is how OpenStreetMap Mapnik renderer draw maps. The border width should be adjustable and be adapted with the max size of the labels.

vyskocil commented 3 years ago

I already tried to implement that but didn't finished it, I couldn't iron out the bugs introduced by the relation between the zoomlevel and the size of the meta-tile, tiles and border...

frogg commented 3 years ago

Thanks for the insights so far, @vyskocil! I have one more addition that might change how we approach this. I figured out that caching the rendered tiles of the 3×3 matrix could lead to additional problems for the tiles directly surrounding (x, y), e.g. for (x-2, y) The main problem is that cut labels propagate to those tiles as well, of course (see attached ad extended graphic, based on yours).

Is this what you have introduced the additional border of meta-rendering for? If yes, I don't see the point anymore to render the surrounding tiles if they are not requested. We could just always add an additional border/padding around (x, y) for meta-rendering and then cut the center piece out which is then our tile for (x, y).

If no, please explain what the border is for.

In general, this definitely looks like something that I could build, I just need to clarify all my assumptions and questions (and maybe I‘m completely pointing in the wrong direction with my questions and annotations, please let me know 🙃).

130322755-498743d0-bce6-47b2-8f87-5804e1bff9a5

vyskocil commented 3 years ago

@frogg Yes the additional border is to avoid cutting most of the labels that are near the meta-tile edges, the area of the padding border is the drawn surface that is thrown away. The point of drawing several tiles (9, 25,…) in one meta tile is performance, it is quicker to draw a large meta-tiles than separated tiles due to initializations of the osm data buffer, etc… and the ratio of kept drawing over thrown away (the border area) is better.

vyskocil commented 3 years ago

A after thought about one of the point I wrote earlier, when the map component ask for a tile, it may not be the center of a meta-tile, but any of the n x n tiles composing the meta-tile, we should calculate the right meta-tile from which the requested tile belong to avoid recalculations. The world map should be subdivided into meta-tiles.

Framstag commented 3 years ago

This github issue at least handles two problems. Building and improving the rendering approach for iOS. For the build problem there is currently the pull request #1106. Should me make a new issue for the rendering improvements?

Framstag / libosmscout

Finish work on iOS .framework for libosmscout #526