reverse_geocoding

[iamqq]

Thanks for the app! I made a mistake several times with the disk size in the virtual machine, but loaded 849683 points. However as a result, I was probably banned from nominatim.openstreetmap.org :

in 5 minutes    7   reverse_geocoding   ReverseGeocodingJob 
7261
Errno::EADDRNOTAVAIL: Failed to open TCP connection to nominatim.openstreetmap.org:443 (Address not available - connect(2) for "nominatim.openstreetmap.org" port 443)

Perhaps because of this the number of countries and cities in the statistics is empty. As a result, I deleted tasks in the reverse_geocoding queue several more times. I don’t know how everything is implemented, but maybe it’s possible to re-calculate countries and cities using existing points? And if possible, get statistics by states(USA)/regiones(Russia). And optimize the appeal to nominatim.openstreetmap.org or replace it with something else so as not to get banned.

[TychoWerner]

I'm scared for my import now... still in progress, Reports file that is 1,8GB with 71 million lines of JSON. Not sure what amount of points I will get as it's still processing; currently at 6 million in queue I noticed you can check at :3000/sidekiq

[Freika]

Hi @iamqq! I got about 700k of points and didn't get banned on OSM, so, bad luck, I guess. The library I use to implement reverse geocoding supports different mapping services, so I'll make sure to allow a user to use one of their choice in future. Your points won't be left un-reverse-geocoded!

@TychoWerner did you import Records.json, the huge file? :O

[TychoWerner]

Hi @iamqq! I got about 700k of points and didn't get banned on OSM, so, bad luck, I guess. The library I use to implement reverse geocoding supports different mapping services, so I'll make sure to allow a user to use one of their choice in future. Your points won't be left un-reverse-geocoded!

@TychoWerner did you import Records.json, the huge file? :O

Still importing haha, currently at 8.5GB RAM, 50% CPU of 4 core, 13 million in queue So about 3.5 million per hour, this is going to take a while I guess 16 hours to import into the queue and then whatever it takes to do the stats calculation

[TychoWerner]

Loading loading loading, added some SWAP in case (it will... at this rate) run out of RAM. Have 'only' 32GB of RAM available total so it will have to do.

[Freika]

Keep in mind, logs may clog your disk space at this rate

Sent from Proton Mail for iOS

On Fri, Jun 7, 2024 at 23:44, Tycho Werner @.***(mailto:On Fri, Jun 7, 2024 at 23:44, Tycho Werner < wrote:

Scherm.afbeelding.2024-06-07.om.23.43.51.png (view on web) Loading loading loading, added some SWAP in case (it will... at this rate) run out of RAM. Have 'only' 32GB of RAM available total so it will have to do.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[Freika]

Also, @iamqq

I don’t know how everything is implemented, but maybe it’s possible to re-calculate countries and cities using existing points? And if possible, get statistics by states(USA)/regiones(Russia). And optimize the appeal to nominatim.openstreetmap.org or replace it with something else so as not to get banned.

I'll implement a couple of buttons: to re-start reverse geocoding from scratch for all points and to start if only for the points that are not yet reverse geocoded.

[Freika]

Keep in mind, logs may clog your disk space at this rate

This was fixed in 0.8.7

I'll implement a couple of buttons: to re-start reverse geocoding from scratch for all points and to start if only for the points that are not yet reverse geocoded.

It's done in release 0.9.1, feel free to update :)