search

erikbroo / gmapcatcher

Automatically exported from code.google.com/p/gmapcatcher
0 stars 0 forks source link

GMapCatcher does not work or hangs with SQLite. #258

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
 1. Open gmapcatcher.
 2. Make gmapcatcher go online to download maps.
 3. It will either hang or complain about the following (see below).

What is the expected output? What do you see instead?
Screen also tends to flicker between the normal blank tiles before map tiles 
have been downloaded and a grey colour (which is the background window colour 
from my theme) when scrolling.

What version of the product are you using? On what operating system?
gmapcatcher 0.7.5, on Ubuntu Karmic. Python 2.6.4.

Please provide any additional information below.
It will keep cycling through messages like the following in the scenario where 
it does not crash:
argument 1 must be string or buffer, not None

argument 1 must be string or buffer, not None

argument 1 must be string or buffer, not None

File corrupted: /home/advise/.googlemaps/sat_tiles/9/0/138/0/102.png
File corrupted: /home/advise/.googlemaps/sat_tiles/9/0/138/0/101.png
File corrupted: /home/advise/.googlemaps/sat_tiles/9/0/138/0/100.png
Cannot parse result
argument 1 must be string or buffer, not None

argument 1 must be string or buffer, not None

In the scenario where it does crash (seems to consistently happen if I use 
SQLite3), I get this:
Exception in thread Thread-6:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 97, in run
    self.process_sqlrequest()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 124, in process_sqlrequest
    self.get_tile_row(req[0], req[1], req[2], req[3])
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 153, in get_tile_row
    qry = "SELECT  x,y,zoom,layer, tstamp, img FROM tiles WHERE zoom=%i AND x=%i AND y=%i AND layer=%i" % (zoom_level, coord[0], coord[1], layer)
TypeError: %d format: a number is required, not type

Killed

or this:

Cannot parse result
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 97, in run
    self.process_sqlrequest()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 121, in process_sqlrequest
    self.store_tile(req[0], req[1], req[2], req[3], req[4])
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 166, in store_tile
    dbcursor.execute( "INSERT INTO tiles (x,y,zoom,layer,tstamp,img)  VALUES(?,?,?,?,?,?)", (coord[0], coord[1],zoom_level,layer,tstamp,sqlite3.Binary(data)) )
TypeError: buffer object expected

Cannot parse result
Cannot parse result
Killed

A couple of days ago, everything seemed to be working fine.

Original issue reported on code.google.com by cobal...@gmail.com on 11 Feb 2011 at 4:47

GoogleCodeExporter commented 8 years ago 
Interesting, what if you change the map provider to OpenStreetMap, Do you still 
get the same errors?

Original comment by heldersepu on 11 Feb 2011 at 6:28

GoogleCodeExporter commented 8 years ago 
so far I'm not getting any errors in windows...

Original comment by heldersepu on 11 Feb 2011 at 6:29

GoogleCodeExporter commented 8 years ago 
Also I see that you are using SQLITE can you reset that to the default, and try 
again?

the easiest way to reset GMapCatcher is to delete the file "gmapcatcher.conf"

Original comment by heldersepu on 11 Feb 2011 at 6:33

GoogleCodeExporter commented 8 years ago 
Thanks for the fast reply.

My apologies, I should have clarified slightly.

I was running on a clean reinstall of gmapcatcher (had deleted the whole of the 
.googlemaps directory). I have a .deb file, I'm pretty sure I got from the 
official website. I take it it was the regular file I got: 
mapcatcher_0.7.5.0-1_all.deb and reinstalled from that.

For the results above, I was running with 'files' when it failed to work, but 
did not crash the program (and it was complaining about the corrupt files).

Where it actually hangs the program and throws those exceptions, I was using 
SQLite.

Having said that, I tried it again, here's some more results, first with 
SQLite, then with files. I only tried Google and Openstreetmap with files.

It was working whilst I was abroad in the UK (with SQLite). I got back home, 
and it stopped working. I also don't appear to have installed anything that 
would have messed with my Python binaries. Also, I haven't been getting any 
other errors in other applications that would suggest I've got a failing hard 
disk.

Hope this helps.

OpenStreetmap with SQLite works.

Cloudmade with SQLite fails with those (and nothing else more specific - the 
program does not hang):
    download failed - <type 'exceptions.NameError'>

Yahoo (map, satellite and hybrid) works with SQLite.

Information freeway fails with the following on SQLite and hangs gmapcatcher:
Cannot parse result
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 97, in run
    self.process_sqlrequest()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 121, in process_sqlrequest
    self.store_tile(req[0], req[1], req[2], req[3], req[4])
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 166, in store_tile
    dbcursor.execute( "INSERT INTO tiles (x,y,zoom,layer,tstamp,img)  VALUES(?,?,?,?,?,?)", (coord[0], coord[1],zoom_level,layer,tstamp,sqlite3.Binary(data)) )
TypeError: buffer object expected

Cannot parse result

OpenCyclemap works with SQLite.

Google Mapmaker works with SQLite.

Virtual Earth works with SQLite (map, satellite). It crashed when I got to 
terrain with the following again:
Cannot parse result
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 97, in run
    self.process_sqlrequest()
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 121, in process_sqlrequest
    self.store_tile(req[0], req[1], req[2], req[3], req[4])
  File "/usr/lib/pymodules/python2.6/gmapcatcher/tilesRepoSQLite3.py", line 166, in store_tile
    dbcursor.execute( "INSERT INTO tiles (x,y,zoom,layer,tstamp,img)  VALUES(?,?,?,?,?,?)", (coord[0], coord[1],zoom_level,layer,tstamp,sqlite3.Binary(data)) )
TypeError: buffer object expected

Cannot parse result

With settings back to the default 'File', I get the following:
Google fails but does not hang the program:
Cannot parse result
argument 1 must be string or buffer, not None
File corrupted: /home/advise/.googlemaps/sat_tiles/16/0/1/0/1.png
Cannot parse result
argument 1 must be string or buffer, not None
File corrupted: /home/advise/.googlemaps/sat_tiles/16/0/0/0/1.png
Cannot parse result
argument 1 must be string or buffer, not None
File corrupted: /home/advise/.googlemaps/sat_tiles/16/0/0/0/0.png
Cannot parse result
argument 1 must be string or buffer, not None
File corrupted: /home/advise/.googlemaps/sat_tiles/16/0/1/0/0.png

If I change this to Openstreetmap straightaway, I get the flickering.
If I quiet and restart with Openstreetmap, it works, although it complains 
about some corrupt files although I suspect, if I force a refresh, those would 
go away:
File corrupted: /home/advise/.googlemaps/tiles/14/0/3/0/2.png
File corrupted: /home/advise/.googlemaps/tiles/14/0/4/0/3.png

I haven't tried the other ones. Got a bit lazy in the meantime.

Original comment by cobal...@gmail.com on 11 Feb 2011 at 8:54

GoogleCodeExporter commented 8 years ago 
The legacy .deb binaries give me the same as well.

Original comment by cobal...@gmail.com on 11 Feb 2011 at 9:01

GoogleCodeExporter commented 8 years ago 
No need to try the rest, that is plenty; this is definitely not an issue 
accessing Google Maps, I just tested in an old VMWare with Ubuntu with the 
latest source files and it looks good, I don't get those errors.

Helder

Original comment by heldersepu on 11 Feb 2011 at 9:02

GoogleCodeExporter commented 8 years ago 
I've tried this on two different Ubuntu machines, one running Karmic and one 
running Maverick, and the problem seems to be the same. I'm not sure why I'm 
the only one getting this. The baffling thing is that it used to work initially 
on the Karmic netbook. I get the 'cannot parse' stuff with 'corrupted' files, 
which turn out not to exist in the first place.

Could this be a missing library issue that got skipped in the dependency list?

Original comment by cobal...@gmail.com on 18 Feb 2011 at 9:48

GoogleCodeExporter commented 8 years ago

Original comment by heldersepu on 18 Feb 2011 at 9:53

GoogleCodeExporter commented 8 years ago 
I guess this is an issue that happens only with SQLite, I will rename this 
issue accordingly.

Original comment by heldersepu on 18 Feb 2011 at 9:57

GoogleCodeExporter commented 8 years ago 
That is not the case, if you read what I wrote again, you'll find that there 
are two different sets of errors, with Google Maps, which works just fine in a 
browser, and was working fine in Mapcatcher on my Karmic box too prior to 
February.

The first issue is the 'cannot parse' lot of errors associated with the 
'corrupt' files, which happens when using Google Maps whiilst using _files_. 
The program does not crash, and remains responsive, but there are no maps.

When I DO use SQLite3, what I get is an actual unhandled exception which 
crashes the whole program. I've also included that error in the pastes above.

I've looked at my dpkg logs to see if I'd uninstalled anything Pythony, but 
there appears to be almost nothing there, I've installed what I had 
uninstalled, and reinstalled all my other Python and SQL packages, to no avail.

Original comment by cobal...@gmail.com on 18 Feb 2011 at 10:23

GoogleCodeExporter commented 8 years ago 
I think I got the error while trying to access Google Terrain Maps.

IMO problem arises in mapServers/googleMaps.py
on lines:

    # List of patterns add more as needed
    paList = ['http://([a-z]{2,3})[0-9].google.com/([a-z]+)[?/]v=([a-z0-9.]+)&',
              'http://([a-z]{2,3})[0-9].google.com/([a-z]+)[?/]lyrs=([a-z@0-9.]+)&',
              'http://([a-z]{2,3})[0-9].google.com/([a-z]+)[?/]v\\\\x3d([a-z0-9.]+)\\\\x26']
    for srtPattern in paList:
        p = re.compile(srtPattern)
        match = p.search(html)
        if match: break
70:    if not match:
        print "Cannot parse result"
        return None

74:    return 'http://%s%%d.google.com/%s/v=%s' % tuple(match.groups()) + 
end_str
pattern is not matched on line 70 and this error situation is not checked so 
from the point not URL is returned (line 74) but None. 

Correct situation:
- handle error situation correctly. This should be preferred method
- add URL to paList.

Since google maps are no more supported I don't know how to deal with it.

Original comment by standa31...@gmail.com on 20 Feb 2011 at 1:16

GoogleCodeExporter commented 8 years ago

Original comment by heldersepu on 13 Oct 2011 at 8:46