TimSC / pycrocosm

OSM API v0.6 implemented in Django/Python
7 stars 4 forks source link

diff files incorrect #18

Closed 4x4falcon closed 6 years ago

4x4falcon commented 6 years ago

minutely, hourly and daily diff files do not have create, modify or delete tags.

http://fosm.org/replication/minute/200/001/356.osc

Should have "\<create>\</create>" around the node, this was created in this changeset.

It is the same for 356.osc.gz

This is needed for updating the map and any future update from a server rebuild.

4x4falcon commented 6 years ago

I have done three test diffs. 1 - create two nodes and a way http://fosm.org/replication/minute/200/007/627.osc 2 - modify both nodes and way http://fosm.org/replication/minute/200/007/628.osc 3 - delete both nodes and way http://fosm.org/replication/minute/200/007/630.osc

TimSC commented 6 years ago

@4x4falcon Right I've added a fix for this, as well as unit tests. Probably safe to roll it out and do more testing.

4x4falcon commented 6 years ago

In pgmap

When I run this:

swig -python -c++ -DPYTHON_AWARE -DSWIGWORDSIZE64 pgmap.i

I get the following errors:

running build_ext building '_pgmap' extension swigging pgmap.i to pgmap_wrap.cpp swig -python -c++ -DPYTHON_AWARE -DSWIGWORDSIZE64 -o pgmap_wrap.cpp pgmap.i pgmap.i:69: Warning 362: operator= ignored pgmap.i:207: Warning 362: operator= ignored pgmap.i:279: Warning 362: operator= ignored pgmap.i:386: Warning 362: operator= ignored streambuf.i:4: Warning 401: Nothing known about base class 'std::streambuf'. Ignored. creating build creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/cppo5m creating build/temp.linux-x86_64-2.7/cppo5m/iso8601lib creating build/temp.linux-x86_64-2.7/cppGzip x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPYTHON_AWARE=1 -I/usr/include/python2.7 -c pgmap_wrap.cpp -o build/temp.linux-x86_64-2.7/pgmap_wrap.o -std=c++11 cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ pgmap_wrap.cpp: In function ‘PyObject _wrap_LoadFromO5m__SWIG_1(PyObject, PyObject)’: pgmap_wrap.cpp:51450:50: error: invalid initialization of reference of type ‘std::streambuf& {aka std::basic_streambuf&}’ from expression of type ‘const string {aka const std::__cxx11::basic_string}’ LoadFromO5m((std::string const &)arg1,arg2); ^ In file included from cppo5m/osmxml.h:6:0, from pgmap.h:9, from pgmap_wrap.cpp:3585: cppo5m/OsmData.h:103:6: note: in passing argument 1 of ‘void LoadFromO5m(std::streambuf&, std::shared_ptr)’ void LoadFromO5m(std::streambuf &fi, std::shared_ptr output); ^ pgmap_wrap.cpp: In function ‘PyObject _wrap_LoadFromOsmXml__SWIG_1(PyObject, PyObject)’: pgmap_wrap.cpp:51550:53: error: invalid initialization of reference of type ‘std::streambuf& {aka std::basic_streambuf&}’ from expression of type ‘const string {aka const std::__cxx11::basic_string}’ LoadFromOsmXml((std::string const &)arg1,arg2); ^ In file included from cppo5m/osmxml.h:6:0, from pgmap.h:9, from pgmap_wrap.cpp:3585: cppo5m/OsmData.h:104:6: note: in passing argument 1 of ‘void LoadFromOsmXml(std::streambuf&, std::shared_ptr)’ void LoadFromOsmXml(std::streambuf &fi, std::shared_ptr output); ^ pgmap_wrap.cpp: In function ‘PyObject _wrap_LoadFromOsmChangeXml__SWIG_1(PyObject, PyObject)’: pgmap_wrap.cpp:51650:59: error: invalid initialization of reference of type ‘std::streambuf& {aka std::basic_streambuf&}’ from expression of type ‘const string {aka const std::__cxx11::basic_string}’ LoadFromOsmChangeXml((std::string const &)arg1,arg2); ^ In file included from cppo5m/osmxml.h:6:0, from pgmap.h:9, from pgmap_wrap.cpp:3585: cppo5m/OsmData.h:105:6: note: in passing argument 1 of ‘void LoadFromOsmChangeXml(std::streambuf&, std::shared_ptr)’ void LoadFromOsmChangeXml(std::streambuf &fi, std::shared_ptr output); ^ error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

This is a fresh clone of pgmap from git.

TimSC commented 6 years ago

@4x4falcon I messed up the commits for the project's submodules. Please try again with newer pgmap and cppo5m :)

4x4falcon commented 6 years ago

I'm getting this error now attempting to access any page including the frontpage and any data request.

Request Method: GET

http://fosm.org/ 1.11.8 NotImplementedError Wrong number or type of arguments for overloaded function 'new_mapstringstring'. Possible C/C++ prototypes are: std::map< std::string,std::string >::map(std::less< std::string > const &) std::map< std::string,std::string >::map() std::map< std::string,std::string >::map(std::map< std::string,std::string > const &) /usr/local/lib/python2.7/dist-packages/pgmap.py in init, line 426 /usr/bin/uwsgi-core 2.7.12 ['.', '', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages/django_oauth_plus-2.2.9-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/oauth2-1.9.0.post1-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/httplib2-0.10.3-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/django_oauth_plus_mod-2.2.9-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/oauth10a-1.9.1-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/mapnik-3.1.0-py2.7-linux-x86_64.egg', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat'] Mon, 25 Jun 2018 09:21:24 +0000

TimSC commented 6 years ago

I updated a second server to the latest version and things seem ok.

Can you double check the commit IDs of the various modules?

pycrocosm 27db480 pgmap b1292cb cppo5m a8aa1e6

Tim

PS are you running pgmap in a virtualenv?

4x4falcon commented 6 years ago

I have all those.

I think I need to leave it for now and start again fresh in the morning. After driving 800kms today in 10 hours brain is a little slow.

pgmap is running in the virtualenv same as previous.

4x4falcon commented 6 years ago

I've sorted this now. Took all of 5 minutes with a fresh brain. A complete fresh install and then add the website in one file at a time. Worked until I put in the replacement settings.py and it all crashed. Put the original settings.py back in and it works. So I added the necessary parts to the settings.py rather than replacing it and it all works. Must have been a spurious character in the file causing the problem.

4x4falcon commented 6 years ago

This can be closed as the diffs are all working correctly.

TimSC commented 6 years ago

That's great!