Black rectangles in radar maps.

[feh123]

In the last week I have started to see black areas (or various sizes) appear in both my maps. I am using dark sky and mapbox on a Raspberry Pi 3B (running Stretch). No sensors just a basic setup. I have tried burning a stored image to another SD card and I have also updated PiClock via git pull. The black areas are brief and are not present throughout the radar transition - probably around 10% of the transiton time. I have a transition time of 500 and zooms of 4 and 7. The higher zoom has more black areas.

Any thoughts on fixing this appreciated.

[Knackster]

I'm getting the same blocks on my maps for the past week as well. I rebooted my pi and still have then.

On Thu, Jul 18, 2019, 7:38 AM feh123 notifications@github.com wrote:

In the last week I have started to see black areas (or various sizes) appear in both my maps. I am using dark sky and mapbox on a Raspberry Pi 3B (running Stretch). No sensors just a basic setup. I have tried burning a stored image to another SD card and I have also updated PiClock via git pull. The black areas are brief and are not present throughout the radar transition - probably around 10% of the transiton time. I have a transition time of 500 and zooms of 4 and 7. The higher zoom has more black areas.

Any thoughts on fixing this appreciated.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/163?email_source=notifications&email_token=AFIERVEW6SZS5MZIG3LSACDQABP6BA5CNFSM4IE2JSGKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G77YKNQ, or mute the thread https://github.com/notifications/unsubscribe-auth/AFIERVHIYWCW2HJTH4T7K3LQABP6BANCNFSM4IE2JSGA .

[ScottChapman]

Ditto

[ckinsler]

Same - I thought I had a flash card going bad. Glad to hear it's not just me!

[vanhalen26505]

Same here.

On Thu, Jul 18, 2019 at 8:38 AM feh123 notifications@github.com wrote:

In the last week I have started to see black areas (or various sizes) appear in both my maps. I am using dark sky and mapbox on a Raspberry Pi 3B (running Stretch). No sensors just a basic setup. I have tried burning a stored image to another SD card and I have also updated PiClock via git pull. The black areas are brief and are not present throughout the radar transition - probably around 10% of the transiton time. I have a transition time of 500 and zooms of 4 and 7. The higher zoom has more black areas.

Any thoughts on fixing this appreciated.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/163?email_source=notifications&email_token=AK4ASA6IZWQBUH3CCEBYCYLQABP6DA5CNFSM4IE2JSGKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G77YKNQ, or mute the thread https://github.com/notifications/unsubscribe-auth/AK4ASA4CRUE4SCNBMQOADJLQABP6DANCNFSM4IE2JSGA .

-- Lee O. Silverman, M.A., Owner & Voice Talent Leecast Professional Voice-Over Services http://leecast.com

[ScottChapman]

FYI - i am reaching out to the rainviewer team to see if they are aware of anything.

[n0bel]

Its coming from rainvewer api. Not much we can do about it. The only fix might be to recognize the bad map tile and render it transparent. This would result in missing radar data rather than black squares

[Knackster]

So looking at my display this morning before leaving for work, I no longer am seeing black tiles. I guess the blip is gone.

Chris

On Thu, Jul 18, 2019 at 11:24 AM Kevin Uhlir notifications@github.com wrote:

Its coming from rainvewer api. Not much we can do about it. The only fix might be to recognize the bad map tile and render it transparent. This would result in missing radar data rather than black squares

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/163?email_source=notifications&email_token=AFIERVC2IHRE2EML3BFVRX3QACKNBA5CNFSM4IE2JSGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2JA4CI#issuecomment-512888329, or mute the thread https://github.com/notifications/unsubscribe-auth/AFIERVHLWJLDDK2XQZDUYWLQACKNBANCNFSM4IE2JSGA .

[Techfun49]

FYI ... I was still getting black rectangles occasionally as of this afternoon.

[ckinsler]

same - was working fine for a few hours, then the issue came back

[Knackster]

Me too. Fine this morning. Got home from work and they are back.

Chris

On Fri, Jul 19, 2019 at 8:08 PM Chris Kinsler notifications@github.com wrote:

same - was working fine for a few hours, then the issue came back

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/163?email_source=notifications&email_token=AFIERVE4UXLHGGRLBBPIEVDQAJQPTA5CNFSM4IE2JSGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NDE3A#issuecomment-513421932, or mute the thread https://github.com/notifications/unsubscribe-auth/AFIERVHE2KIKQFDBWV6JRMDQAJQPTANCNFSM4IE2JSGA .

[feh123]

I am still getting black areas - these are missed tiles? For a hour this morning UK it looked good but now the high zoom map is badly affected.

[BertLindeman]

Same here (Netherlands) Saw some black tiles.

Took all radar images from the log and made an html page of it to verify if any black parts in it. Looks like it is impossible to upload a picture :-( Something went really wrong, and we can’t process that file.

html looks like:

<br />  radar1 0 <img src="https://tilecache.rainviewer.com/v2/radar/1563599400//256/2/1/1/1/1_1.png"  class="ImageBorder" border="1"  />
        radar1 1 <img src="https://tilecache.rainviewer.com/v2/radar/1563599400//256/2/2/1/1/1_1.png" class="ImageBorder" border="1"  />
<br />  radar1 0 <img src="https://tilecache.rainviewer.com/v2/radar/1563600000//256/2/1/1/1/1_1.png" class="ImageBorder" border="1"  />
        radar1 1 <img src="https://tilecache.rainviewer.com/v2/radar/1563600000//256/2/2/1/1/1_1.png" class="ImageBorder" border="1"  />

But all are OKE

Might there be a problem processing transparency in the clock?

[togatown]

I don't think it's rainviewer, I have two clocks side by side, one running python2/PyQt4 and the other is python3/PyQt5. The PyQt5 machine is not exhibiting this behavior.

I am not responsible for you being a bonehead! Back up your files before proceeding!

To convert yours: Install PyQt5 for python3

sudo apt-get install python3-pyqt5

Then change the imports in Config.py header from:

from PyQt4.QtGui import QColor

to:

from PyQt5.QtGui import QColor

and the imports in PyQtPiClock.py header from:

from PyQt4 import QtGui, QtCore, QtNetwork
from PyQt4.QtGui import QPixmap, QBrush, QColor
from PyQt4.QtGui import QPainter, QImage, QFont
from PyQt4.QtCore import QUrl
from PyQt4.QtCore import Qt
from PyQt4.QtNetwork import QNetworkReply
from PyQt4.QtNetwork import QNetworkRequest

to:

from PyQt5 import QtGui, QtCore, QtNetwork, QtWidgets
from PyQt5.QtGui import QPixmap, QBrush, QColor
from PyQt5.QtGui import QPainter, QImage, QFont
from PyQt5.QtCore import QUrl, QObject
from PyQt5.QtCore import Qt
from PyQt5.QtNetwork import QNetworkReply
from PyQt5.QtNetwork import QNetworkRequest

Next do a Find and Replace all in PyQtPiClock.py for:

QtGui.QMatrix get replaced with QtGui.QTransform
QtGui.QLabel get replaced with QtWidgets.QLabel
QtGui.QApplication get replaced with QtWidgets.QApplication
QtGui.QWidget get replaced with QtWidgets.QWidget
 QtGui.QFrame get replaced with QtWidgets.QFrame

Next, find all the print statements and make sure the parameters and enclosed to parens:

print whatever stuff here

becomes:

print(whatever stuff here)

do a find for the following lines:

wxstr = str(wxreply.readAll())
wxdata = json.loads(wxstr)

and replace them with:

wxstr = bytes(wxreply.readAll()).decode("utf-8")
wxdata = json.loads(str(wxstr))

and the final edit to PyQtPiClock.py is to find and replace these line:

QtCore.QObject.connect(self.tilereply, QtCore.SIGNAL(
             "finished()"), self.getTilesReply)

is replaced with:

self.tilereply.finished.connect(self.getTilesReply)

and:

QtCore.QObject.connect(self.basereply, QtCore.SIGNAL(
            "finished()"), self.basefinished)

is replaced with:

self.basereply.finished.connect(self.basefinished)

finally, edit the startup.sh file. Find:

python -u PyQtPiClock.py

and replace it with:

python3 -u PyQtPiClock.py

and:

python -u PyQtPiClock.py >Logs/PyQtPiClock.1.log 2>&1

get replaced with:

python3 -u PyQtPiClock.py >Logs/PyQtPiClock.1.log 2>&1

[ScottChapman]

Interesting. Why are we seeing this all or a sudden? What's the issue exactly?

[n0bel]

I have not done any research yet.. My initial reaction to @togatown report would be some issue with a png file that can't be read by pyqt4 properly, but is still valid, like a bug fix in qt5. But this is just a guess.

@BertLindeman report would lead me back to a problem at rainviewer.

So I have no idea.

We have to capture some bad tiles and proceed from there.

Kevin

[feh123]

Just a question - why do I lose both the map and the radar?

On my PiClock I only see the black area in the high zoom where the map covers around 100 x 100 miles. The black areas can be up to 60 x 30 miles - there is a large range of sizes - some nearly square, others rectangular. The lower zoom map covers 1500 x 1500 miles - I never/rarely see black areas in this map.

[n0bel]

The bad radar image tile is not transparent, so it blots out the map. The radar is composed if map tiles.. Different zoom levels use different map tiles. Each tile is a separate image requested from the rainviewer api separately. So each image has the opportunity to be bad independently of others.

[n0bel]

I captured some of the black frames. They are a very small PNG file that PyQt4 interprets as a monochrome (format=1) black image. Any other graphics program interprets it as a fully transparent image. I'm betting that Rainviewer did some work to make a simple transparent png file that is nice and small to conserve bandwidth. The lurking bug in PyQt4 then hit us.

So it does appear to be a Qt4 bug as it interprets a png file of this kind. This bug also occurs in the windows version. It makes sense that @togatown fix would work.

It seems to me that I saw these black blocks on areas of the radar map that should have had radar data. If we're actually seeing missing radar tiles, then this is not a total fix, and the real fix must come from upstream (Rainviewer)

The to to interpret the improperly interpreted png files correctly, the fix is pretty simple:

Insert one line and indent the next on line 716 of PyQtPiClock.py From this:

        for y in range(0, self.totalHeight, 256):
            for x in range(0, self.totalWidth, 256):
                painter.drawImage(x, y, self.tileQimages[i])
                # painter.drawRect(x, y, 255, 255)
                # painter.drawText(x+3, y+12, self.tiletails[i])
                i += 1

To This

        for y in range(0, self.totalHeight, 256):
            for x in range(0, self.totalWidth, 256):
                if self.tileQimages[i].format() == 5:
                    painter.drawImage(x, y, self.tileQimages[i])
                # painter.drawRect(x, y, 255, 255)
                # painter.drawText(x+3, y+12, self.tiletails[i])
                i += 1

What this does is simply not stitch any QImages that are not format=5 (RGBA) into the final displayed image. The black blobs are effectively transparent, i.e. not rendered.

I'll be testing this here for a few hours. We have no rain atm, so I won't see any missing bloc issues for a while but my black blocks are gone.

I tried some radar map testing using lat/lon from areas that are having storms. There does seem to be some missing radar maptiles at times, but not having black blobs does seem to be a better result.

[togatown]

@n0bel I have 2 Qt4 clocks set to different coords with current precipitation and this has fixed both.

[feh123]

@n0bel - thanks for the explanation re the bad radar image tile. I have a saved image file of my RPI so I will try the fixes suggested tomorrow.

[n0bel]

Looks like the change works fine. Still I see missing radar tiles occasionally. Perhaps the data has always been that way.

I've committed the changes: https://github.com/n0bel/PiClock/commit/b8040b7c1f549214b3bd0c3e8fd1ff11daceaa54

Users should be able to do a git pull, as outlined in https://github.com/n0bel/PiClock/blob/master/Documentation/Install.md#updating-to-newerupdated-versions

[feh123]

Tried the git pull. Seemed to go okay but no change - black areas still present. Could Rainviewer in UK/EU be more prone to having bad tiles?

[feh123]

Update - it looks good now - no black areas for one hour. Not sure why the black areas were still present after git pull.

[Knackster]

Thanks for the commit Kevin. Just updated mine.

Chris

On Sun, Jul 21, 2019 at 8:08 AM feh123 notifications@github.com wrote:

Update - it looks good now - no black areas for one hour. Not sure why the black areas were still present after git pull.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/163?email_source=notifications&email_token=AFIERVFTKPCMOUDOYFUMMQTQARNTBA5CNFSM4IE2JSGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2ODEJQ#issuecomment-513552934, or mute the thread https://github.com/notifications/unsubscribe-auth/AFIERVCBAOVOXRBO64J7NKDQARNTBANCNFSM4IE2JSGA .

[ghost]

I had this issue when I updated pi-clock. I had to go back to accuweather api's this fixed it for me.

[feh123]

The black areas are still occurring. Its not a major issue but I would like to fix it. I tried @togatown installation of pyqt5 but I am not a coder and copy/paste did not work for me - I can easily introduce a coding error and not see it. I did have backups so no problem to revert. Can I just add accuweather api's? I thought it was more complicated and needs other changes as I recall from moving to dark sky and rainviewer. Any advice appreciated.