GIScience / oshdb

OpenStreetMap History Data Analysis Framework
https://ohsome.org
GNU Lesser General Public License v3.0
110 stars 18 forks source link

Enable simple code sharing through properties file #208

Closed SlowMo24 closed 4 years ago

SlowMo24 commented 4 years ago

Working with multiples users on an application (e.g. via git) is hampered by the fact that the setup of the OSHDBDatabase class might be very personal. This results in broken code when run on a different machine as H2 files might not be present (and to big to share over git) or cluster access might not be available. Basic settings like BBX also frequently change during development possibly resulting in unwanted git commits.

The problem may be overcome by the possibility to read in a properties file not synchronised over git each team member may set up individually. Thereby users may concentrate on coding the map-reduce procedure ignoring OSHDBDatabase set up and tear down as well as detailed View definitions.

For an example please see the beginning of the main class of https://gitlab.gistools.geog.uni-heidelberg.de/giscience/big-data/ohsome/apps/event-identification/blob/ca15925931edd34bfecf8f797e87ec5e4b50d691/src/main/java/org/heigit/bigspatialdata/eventfinder/EventFinder.java

SlowMo24 commented 4 years ago

@tyrasd who is help wanted directed to? do you need more details?

SlowMo24 commented 4 years ago
tyrasd commented 4 years ago

who is help wanted directed to?

everyone :blush:

from http://api.ohsome.org/v0.9/metadata.

well, not everyone is using the same database backend. imho it could be confusing if we added such specific functionality.

SlowMo24 commented 4 years ago

well, not everyone is using the same database backend. imho it could be confusing if we added such specific functionality.

yes of course this is a HeiGIT-GIScience internal issue. What about a HD-specific customisation-package one could add to her installation?

rtroilo commented 4 years ago

So I'm not sure what is the actual task here, should we provide a helper class? I did something similar for an other project, like

OSHDBUtil.connect(properties, (oshdb) -> {
   oshdb.getContributionView()
      .areaOfInterest((Geometry & Polygonal) areaOfInterest)
      .timestamps(tstamps)
      .osmTag(key)
      .map(...)
      ...
});

with props like:

prefix=global
oshdb=ignite:./ignite.xml  or jdbc:h2:/PATH_TO.oshdb;user=sa;access_mode_data=r
keytable=jdbc:postgresql://URL/keytables-%PREFIX;user=ohsome;password=pass
SlowMo24 commented 4 years ago

yes something like that would be helpful. you can see I did a similar thing in the project linked

rtroilo commented 4 years ago

I would upload the OSHDBUtil+ classes somewhere. Do we have a nice place for that? What about there? https://gitlab.gistools.geog.uni-heidelberg.de/giscience/big-data/ohsome/helpers Name suggestions for the subproject are welcome!

SlowMo24 commented 4 years ago

can you make the class available over our maven repository?

rtroilo commented 4 years ago

sure we can, but we still need a place and a name for such "things". I'm not sure if this is something we want to have in github.

TODO to us, decide where and name.

rtroilo commented 4 years ago

I just upload a helper for connecting to OSHDB ignite/h2

https://gitlab.gistools.geog.uni-heidelberg.de/giscience/big-data/ohsome/helpers/oshdb-database-driver

feel free to edit and improve

tyrasd commented 4 years ago

can you make the class available over our maven repository?

-> http://repo.heigit.org/artifactory/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/org/heigit/ohsome/oshdb/helpers/OSHDBDriver

tyrasd commented 4 years ago

here's also the source code for the database driver: https://gitlab.gistools.geog.uni-heidelberg.de/giscience/big-data/ohsome/helpers/oshdb-database-driver

I think the issue can be closed now, right?