search

ham-radio-software / D-Rats

D-Rats program for D-Star Ham Radios
https://iz2lxi.jimdofree.com/
Other
44 stars 13 forks source link

Large amount of map data being left in /tmp #79

Open dranch opened 4 years ago

dranch commented 4 years ago

On D-rats 0.3.10Beta2, I'm seeing large amounts of map data being left over in /tmp. These left over files seem to go in triples created day consuming about 163MB. I feel that one new files are downloaded, the old files should be deleted.

Today's example: -rw------- 1 dranch dranch 29392896 Jul 29 11:34 tmpXRENJd
-rw------- 1 dranch dranch 35205 Jul 29 11:05 tmpYqavPZ
-rw------- 1 dranch dranch 133724837 Jul 29 11:05 tmp1PzF37

The file -rw------- 1 dranch dranch 29392896 Jul 29 11:34 tmpXRENJd has the top contents:

---------------------------------- WARNING ----------------------------------------

Some of the data that you have obtained from this U.S. Geological Survey database

may not have received Director's approval. Any such data values are qualified

as provisional and are subject to revision. Provisional data are released on the

condition that neither the USGS nor the United States Government may be held liable

for any damages resulting from its use.

#

Additional info: https://help.waterdata.usgs.gov/policies/provisional-data-statement

#

File-format description: https://help.waterdata.usgs.gov/faq/about-tab-delimited-output

Automated-retrieval info: https://help.waterdata.usgs.gov/faq/automated-retrievals

#

Contact: gs-w_support_nwisweb@usgs.gov

retrieved: 2020-07-29 14:05:09 EDT (sdww02)

#

Data for the following 21914 site(s) are contained in this file

USGS 01010000 St. John River at Ninemile Bridge, Maine

USGS 01010070 Big Black River near Depot Mtn, Maine

USGS 01010500 St. John River at Dickey, Maine

USGS 01011000 Allagash River near Allagash, Maine

USGS 01011500 St. Francis River near Connors, New Brunswick

. . .

The file -rw------- 1 dranch dranch 35205 Jul 29 11:05 tmpYqavPZ
has the contents

<!DOCTYPE html>

USGS User Input Message . . . -- The file -rw------- 1 dranch dranch 133724837 Jul 29 11:05 tmp1PzF37 has the contents: -- # ---------------------------------- WARNING ---------------------------------------- # Some of the data that you have obtained from this U.S. Geological Survey database # may not have received Director's approval. Any such data values are qualified # as provisional and are subject to revision. Provisional data are released on the # condition that neither the USGS nor the United States Government may be held liable # for any damages resulting from its use. # # Additional info: https://help.waterdata.usgs.gov/policies/provisional-data-statement # # File-format description: https://help.waterdata.usgs.gov/faq/about-tab-delimited-output # Automated-retrieval info: https://help.waterdata.usgs.gov/faq/automated-retrievals # # Contact: gs-w_support_nwisweb@usgs.gov # retrieved: 2020-07-29 11:02:09 EDT (sdww01) # # Data for the following 21914 site(s) are contained in this file # USGS 01010000 St. John River at Ninemile Bridge, Maine # USGS 01010070 Big Black River near Depot Mtn, Maine # USGS 01010500 St. John River at Dickey, Maine # USGS 01011000 Allagash River near Allagash, Maine # USGS 01011500 St. Francis River near Connors, New Brunswick # USGS 01012960 Fish River at Soldier Pond, Maine # USGS 01013500 Fish River near Fort Kent, Maine . . . -- My configuration in D-rats is the following: Preferences --> Maps: - Map to use: Outdoors - Outdoors server URL: https://tile.thunderforest.com/outdoors/ - Key string for Outdoors URL: - Freshen map after: 720
maurizioandreotti commented 3 years ago

from the reported text i dont really get what data is that- is it about jpg tiles of map download or something else? what happens in deleting the files with r,-f? does it regrow?

wb8tyw commented 2 years ago

The USGS has totally changed their URLs and download API from what D-RATS can use.

So instead of downloading MAP data from pre-configured maps, you are getting an HTML error messages instead. Something is probably not cleaning that up.

The new API appears to require a LAT/LONG box coordinates passed to it just to get a list of the available maps. And then the downloaded data has a UTF-8 tag, so is refusing to parse it.

I have not started looking at this yet or at Thunderforest.com. Thunderforest terms of use want a copyright notice to be displayed when D-Rats starts and is configured to use their maps.

wb8tyw commented 1 year ago

We need a QA checklist for things to look for in D-Rats. Normally /tmp is managed by the operating system, but we should be making sure that we do not over fill it.