search

unige-geohealth / accessmod

accessmod 5 : anisotropic accessibility analysis.
GNU Lesser General Public License v3.0
42 stars 14 forks source link

Error when creating an archive for exporting referral vectors #388

Open loicbaecher opened 2 years ago

loicbaecher commented 2 years ago

Current Behavior

When trying to create an archive for exporting referral vectors, an error occurs. The error occurs even if the maximum data size limit is increased. The error occurs not at the exporting step but at the archive creation step. The referral vectors are the only component in the archive.

Detailed Description

The error message is the following :

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

  | time | type | msg -- | -- | -- | -- 4999 | 2022-08-11@10_00_18 | log | The command:v.out.ogr --overwrite input=vReferralNetwork__ZMBref output=/tmp/RtmpkDhpNM/qiftotjcwqrgktmswtc/vector_referral_network_ZMBref output_layer=vector_referral_network_ZMBref format=ESRI_Shapefileproduced an error (1) during execution:Warning 1: 2GB file size limit reached for /tmp/RtmpkDhpNM/qiftotjcwqrgktmswtc/vector_referral_network_ZMBref/vector_referral_network_ZMBref.shp. Going on, but might cause compatibility issues with third party softwareERROR 1: Failed to write shape object. The maximum file size of 4294924780 has been reached. The current record of size 101944 cannot be added.ERROR: Failed to create OGR featureCallexecGRASS_orig(v.out.ogr, input = vReferralNetwork__ZMBref,output = /tmp/RtmpkDhpNM/qiftotjcwqrgktmswtc/vector_referral_network_ZMBref,output_layer = vector_referral_network_ZMBref, flags = overwrite,format = ESRI_Shapefile)

fxi commented 2 years ago

Thanks for posting this. This is an underlying limitation of shapefile export and OGR. [ I don't remember having ] reached this before. The solution would be to export as sqlite or gpkg. We could let the user decides the format. It's about 1/2 day of work + testing. @nicolasray Meanwhile, the data will be.. locked inside the DB. That's a bummer. I can try to manually recover it, if required.

loicbaecher commented 2 years ago

Understood. Thank you for the proposition. Yes please, I would very much like to see this shapefile (as well as another one for which I have a similar problem), if you have time to recover it (them).

loicbaecher commented 2 years ago

Thanks for recovering the data. I wanted to mention that I have indeed noticed that some route vectors were not complete (even when exported as gpkg), as you mentionned. More especially, the vector is not drawn in its entirety but the travel time and distance values have been correctly calculated. I have tried relaunching a referral analysis to calculate an incomplete route which, this time, was drawn in its entirety.

fxi commented 2 years ago

Ok. Did the analysis that produced the routes finish without error / warnings ?

loicbaecher commented 2 years ago

As far as I remember, they were no errors or warnings.