search

GeoEra-GIP / Project-Support-WP8

Science Project Data provider support
https://geoera-gip.github.io/support/
7 stars 2 forks source link

error uploading shapefile #545

Closed fire-bot closed 3 years ago

fire-bot commented 3 years ago

Sent by Leticia Baena Ruíz (l.baena@igme.es). Created by fire.

Dear GeoEra support team,

I am trying to upload a shapefile in the test platform but it returns an error:

Failed to connect to destination PG

Pleaase find attached the shapefile that I want to upload and a screenshot of the steps to make that.

I uploaded it in GeoTIFF format (in order to make a test) and it works correctly, but I need to upload a shapefile in order to add a link to a document in the attribute table.

I look forward to hearing from you soon.

Best regards,

Leticia

Attachments:

  1. Pilots_WP5_Del5.shp.xml
  2. Pilots_WP5_Del5.dbf
  3. Pilots_WP5_Del5.shx
  4. EGDI_-Upload.pdf
  5. Pilots_WP5_Del5.prj
  6. Pilots_WP5_Del5.cpg
  7. Pilots_WP5_Del5.sbn
  8. Imagen1.jpg
  9. Pilots_WP5_Del5.sbx
  10. Pilots_WP5_Del5.shp
nmtoken commented 3 years ago

As a workaround, try converting the shapefile to GeoPackage and uploading that

nmtoken commented 3 years ago

Try with the attached zipped GeoPackage.

I added a unique feature identifier column, but otherwise the same data. I can't see a link to any documents though, so you would need to add those, something like https://repository.europe-geology.eu/egdidocs/...

Pilots_WP5_Del5.zip

lbaenar commented 3 years ago

Thank you for your quick reply. I have not added the link to the document yet, I only was testing how the platform works. I thought to add a new column including this link https://egditest01.geus.dk/egdidocs/tactic/tactic_del_5_3.pdf. I have tried the zipped GeoPackage but it retuns the following error. Could you give me any other advise?

Thank you in advance.

Upload failed Exception : Error importing layer 'poligons' java.lang.Exception: Error importing layer 'poligons' at eu.egdiadmin.dataimport.geopackage.GeoPackageDataImporter.importLayer(GeoPackageDataImporter.java:79) at org.apache.jsp.geopack3_jsp._jspService(geopack3_jsp.java:693) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:476) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:386) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:330) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.tomee.catalina.OpenEJBValve.invoke(OpenEJBValve.java:44) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.tomee.catalina.OpenEJBSecurityListener$RequestCapturer.invoke(OpenEJBSecurityListener.java:97) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.Exception: DB Datasource error at eu.egdiadmin.dataimport.geopackage.GeoPackageDataImporter.copyLayerData(GeoPackageDataImporter.java:178) at eu.egdiadmin.dataimport.geopackage.GeoPackageDataImporter.copyLayerDataToDb(GeoPackageDataImporter.java:199) at eu.egdiadmin.dataimport.geopackage.GeoPackageDataImporter.importData(GeoPackageDataImporter.java:105) at eu.egdiadmin.dataimport.geopackage.GeoPackageDataImporter.importLayer(GeoPackageDataImporter.java:53) ... 31 more

nmtoken commented 3 years ago

To clarify did you upload the zip file, or did you upload the geopackage?

You need to do the unzipped file; sorry if I wasn't clear, I had to zip it to attach it to this issue but you'll need to unzip it

lbaenar commented 3 years ago

Yes, of course, I unzipped the file and I uploaded the Geopackage. When I tried to upload the GeoPackage that I created it returned the same error.

nmtoken commented 3 years ago

odd, it works for me:

image

lbaenar commented 3 years ago

I dont know what happens, I have tried several options, I also copied the GeoPackage in my pc in a folder without " ", but it does not work... Find attached the information for each field in the upload process. I have tried several options in the field "Show field" and the three options in the field"Add to map".

image

nmtoken commented 3 years ago

I just drag the file into the box, rather than browse to it.

nmtoken commented 3 years ago

try import without selecting either an attribute field or map

lbaenar commented 3 years ago

I dont get to upload the GeoPackage but I copied the mapserver definition from the dataset that you created and now it works. https://egditest01.geus.dk/egdi/?mapname=wp5_del5_3_tester#baslay=baseMapGEUS&extent=-79.00422135416667,11.363713541666662,106.08522135416666,85.61628645833332 Anyway I need to solve the problem in order to upload the pilots from other work packages.

On the other hand, I dont know how to show an attribute table in "details" when I click into a pilot. I have seen a layer from HOVER WP6 and it is just what I want to do for my WP. Find attached a screenshot from HOVER WP6.

image

Thank you very much.

nmtoken commented 3 years ago

For a vector dataset you should get attribute details on click (a WMS GetFeatureInfo) out of the box; you can edit which attributes appear and even whether you allow a response in the config, but as it doesn't work at all and you haven't edited the config something else is wrong...

Back to the data.

I checked the shapefile data in QGIS and see that there is an invalid geometry reported for the Malta MSLA polygon (self intersection), I've run the geometry fix tool and saved the data as GeoPackage (zip attached)

TACTIC_Pilots_WP5_Del5-fixed.zip

can you see if you can upload that dataset.

lbaenar commented 3 years ago

The same error continues.

Thank you very much for your help.

nmtoken commented 3 years ago

I don't understand why you can't upload, or why the dataset doesn't give attributes on query. something for the platform developers to debug @mh-geus-dk ?

mh-geus-dk commented 3 years ago

When we helped HOVER with linking their documents to polygons defining their pilot areas we wrote how to that in the EGDI FAQ: http://egdi-public.gitlabpages.geus.dk/egdi-documentation/#/main-content/GeoPackageExamples?id=how-to-link-articles-to-a-spatial-table-with-pilot-areas If this doesn't help please let me know. Cheers Martin

lbaenar commented 3 years ago

Dear Martin, I saw this link but I use ArcGIS, not QGIS, and the steps are not exactly the same.

Thank you for your comment.

Best regards, Leticia

nmtoken commented 3 years ago

@mh-geus-dk The link to the report of how to link to documents is useful, but the reported issue is not how to create a geopackage with links to reports, (though this is mentioned in passing) the reported issue is cannot upload the example shapefile (then geopackage created from this shapefile), by Leticia and a second issue is that no attributes show in a copy of the data that I uploaded.

Do you know why Leticia might not be able to upload any data?

and for the data I uploaded can you see any reason of data issue that prevents this data from being queried, other data I've uploaded works

Many thanks

lbaenar commented 3 years ago

Dear Martin and James, I was in contact with the database manager of my organization and we think that the problem may be that I do not have permission to write the database. Please find below the error message that it retuns when I try to upload a shapefile. I think the problem to upload the geopackage must be similar to this.

Thank you in advance. Leticia

OGR conversion

gdal version = 2020300

sourceDriverTxt = ESRI Shapefile

dbname = egdi host = localhost port = 5432 username = l.baena@igme.es password = **

destDriverTxt = PostgreSQL destTablename = project_tactic.pilotswp5del5shx

INPUT LAYER GetDescription : pilots_wp5_del5 GetGeomType : 3 (Polygon) GetName : pilots_wp5_del5 GetExtent : -8.082640862820195,35.80641370270993,21.087064907293026,56.57111558399389 GetFeatureCount : 9 GetSpatialRef (epsg): 4326

TEMP LAYER GetDescription : project_tactic.pilotswp5del5shx GetGeomType : 3 (Polygon) GetName : project_tactic.pilotswp5del5shx GetExtent : -8.082640862820195,35.80641370270993,21.087064907293026,56.57111558399389 GetFeatureCount : 9 GetSpatialRef (epsg): 4326 Failed to connect to destination PG:dbname=egdi host=localhost port=5432 user=l.baena@igme.es password=***

Job failed...

mh-geus-dk commented 3 years ago

Dear Leticia,

Are you trying to write directly to the EGDI database? That is not possible. You have to upload your data through the administration module.

You can upload your shape file from here:

Production: https://data.geus.dk/egdiadmin/ Test: https://egditest01.geus.dk/egdiadmin/

Cheers, Martin

Fra: lbaenar @.> Sendt: 13. juli 2021 10:11 Til: GeoEra-GIP/Project-Support-WP8 @.> Cc: Martin Hansen @.>; Mention @.> Emne: Re: [GeoEra-GIP/Project-Support-WP8] error uploading shapefile (#545)

Dear Martin and James, I was in contact with the database manager of my organization and we think that the problem may be that I do not have permission to write the database. Please find below the error message that it retuns when I try to upload a shapefile. I think the problem to upload the geopackage must be similar to this.

Thank you in advance. Leticia

OGR conversion

gdal version = 2020300

sourceDriverTxt = ESRI Shapefile

dbname = egdi host = localhost port = 5432 username = @.**@.> password = **

destDriverTxt = PostgreSQL destTablename = project_tactic.pilotswp5del5shx

INPUT LAYER GetDescription : pilots_wp5_del5 GetGeomType : 3 (Polygon) GetName : pilots_wp5_del5 GetExtent : -8.082640862820195,35.80641370270993,21.087064907293026,56.57111558399389 GetFeatureCount : 9 GetSpatialRef (epsg): 4326

TEMP LAYER GetDescription : project_tactic.pilotswp5del5shx GetGeomType : 3 (Polygon) GetName : project_tactic.pilotswp5del5shx GetExtent : -8.082640862820195,35.80641370270993,21.087064907293026,56.57111558399389 GetFeatureCount : 9 GetSpatialRef (epsg): 4326 Failed to connect to destination PG:dbname=egdi host=localhost port=5432 @.**@.> password=e;CL=6x#x\

Job failed...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/GeoEra-GIP/Project-Support-WP8/issues/545#issuecomment-878874640, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQB5EULW3NAZCRAKHZRHDPLTXPYKLANCNFSM476MI5EA.

lbaenar commented 3 years ago

I am trying to upload data from Test environment: https://egditest01.geus.dk/egdiadmin/

lbaenar commented 3 years ago

I mean that maybe my user do not have permission to connect to egdi database: Failed to connect to destination PG:dbname=egdi host=localhost port=5432 user=l.baena@igme.es password=***

mh-geus-dk commented 3 years ago

When you log on to EGDI it is actually the database you log on to, so that cannot be the problem. Are you still getting the same error? If you do could you send me the data you are trying to upload I can do some testing.

lbaenar commented 3 years ago

The error continues. I tried to upload by using another pc but it does not work. @nmtoken uploaded the same data and he had not problem, so I think it must be related to my user.

TACTIC_Pilots_WP5_Del5-fixed.zip

mh-geus-dk commented 3 years ago

Please try once more ;-) I noticed that you were added as a tactic user in production. Cheers Martin

lbaenar commented 3 years ago

I always log in in the test environment. I have tried it again and the error continues.

mh-geus-dk commented 3 years ago

I think I finally found the problem, your username most probably contained a character that caused the problem. I have deleted your user account and created a new one for you in test. I have tried to upload a shapefile and it worked. Sorry for all the troubles :-)

lbaenar commented 3 years ago

Thank you very much, Martin. Now it works! I was going crazy :))

I will continue with the tests. Best regards, Leticia

mh-geus-dk commented 3 years ago

metoo

Don't hesitate asking questions ;-) Best regards, Martin

nmtoken commented 3 years ago

Closing this issue as the main point of uploading has been sorted. @lbaenar, please raise a separate issue if you have trouble with the attributes showing