search

qgis / QGIS

QGIS is a free, open source, cross platform (lin/win/mac) geographical information system (GIS)
https://qgis.org
GNU General Public License v2.0
10.37k stars 2.98k forks source link

Strange problems exporting a layer to file, restoring it to map #27318

Closed qgib closed 5 years ago

qgib commented 6 years ago

Author Name: Loren Amelang (@LorenAmelang) Original Redmine Issue: 19490 Affected QGIS version: 3.2 Redmine category:project_loading/saving

After yesterday's file corruption debacle, I'm trying to make backups of complex creations.

Right-clicked a layer XSCutLines, Export -> Save Features As... (with load to map checked)

After Exporting a copy to shapefile...

Attribute Table data looked good:

wkt_geom XsecID ReachID Station RiverCode ReachCode LeftBank RightBank LLength ChLength RLength NodeName DtmID MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715)) 69 10 4253.06211053154 WALKER GULCH 2
...

Attribute Tables and data match, copy and original, include stationing Original draws stationing Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back...

Saved project, closed and restarted QGIS Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files Now it can be Added or drug into QGIS, shows shapefile as source and it draws the lines, but not the station numbers Numbers are clearly in the table...

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile? Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager...

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote? I'd assumed it was only active when I asked it to do something. Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original? (See #27317 for the reason I'm paranoid...)

qgib commented 6 years ago

Author Name: Jürgen Fischer (@jef-n)

Right-clicked a layer XSCutLines, Export -> Save Features As... (with load to map checked)

After Exporting a copy to shapefile...

Attribute Table data looked good:

wkt_geom XsecID ReachID Station RiverCode ReachCode LeftBank RightBank LLength ChLength RLength NodeName DtmID MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715)) 69 10 4253.06211053154 WALKER GULCH 2
...

Attribute Tables and data match, copy and original, include stationing Original draws stationing Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back...

Saved project, closed and restarted QGIS Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files Now it can be Added or drug into QGIS, shows shapefile as source and it draws the lines, but not the station numbers Numbers are clearly in the table...

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile? Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager...

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote? I'd assumed it was only active when I asked it to do something. Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original? (See [[https://issues.qgis.org/issues/19489]] for the reason I'm paranoid...) to After yesterday's file corruption debacle, I'm trying to make backups of complex creations.

Right-clicked a layer XSCutLines, Export -> Save Features As… (with load to map checked)

After Exporting a copy to shapefile…

Attribute Table data looked good: 

wkt_geom    XsecID  ReachID Station RiverCode   ReachCode   LeftBank    RightBank   LLength ChLength    RLength NodeName    DtmID
MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715))    69  10  4253.06211053154    WALKER GULCH    2   
…

Attribute Tables and data match, copy and original, include stationing Original draws stationing Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back…

Saved project, closed and restarted QGIS Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files Now it can be Added or drug into QGIS, shows shapefile as source and it draws the lines, but not the station numbers Numbers are clearly in the table…

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile? Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager…

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote? I'd assumed it was only active when I asked it to do something. Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original? (See #27317 for the reason I'm paranoid…)

qgib commented 6 years ago

Author Name: Giovanni Manghi (@gioman)

I cannot replicate (trying also from PostGIS to Shapefile), seems likely a local issue of your QGIS installation or maybe something with your database.

There are tons of ways you can make copies (in different formats) of your PostGIS tables: "save features as..." is one, the DB manager had an export (and import) functionality. In Processing you can use "convert format" that makes use of ogr2ogr under the hood. And I'm sure there are probably other ways.

Right-clicked a layer XSCutLines, Export -> Save Features As… (with load to map checked)

After Exporting a copy to shapefile…

Attribute Table data looked good: 

wkt_geom    XsecID  ReachID Station RiverCode   ReachCode   LeftBank    RightBank   LLength ChLength    RLength NodeName    DtmID
MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715))    69  10  4253.06211053154    WALKER GULCH    2   
…

Attribute Tables and data match, copy and original, include stationing Original draws stationing Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back…

Saved project, closed and restarted QGIS Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files Now it can be Added or drug into QGIS, shows shapefile as source and it draws the lines, but not the station numbers Numbers are clearly in the table…

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile? Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager…

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote? I'd assumed it was only active when I asked it to do something. Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original? (See #27317 for the reason I'm paranoid…) to After yesterday's file corruption debacle, I'm trying to make backups of complex creations.

Right-clicked a layer XSCutLines, Export -> Save Features As... (with load to map checked)

After Exporting a copy to shapefile...

Attribute Table data looked good:

wkt_geom XsecID ReachID Station RiverCode ReachCode LeftBank RightBank LLength ChLength RLength NodeName DtmID MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715)) 69 10 4253.06211053154 WALKER GULCH 2
...

Attribute Tables and data match, copy and original, include stationing Original draws stationing Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back...

Saved project, closed and restarted QGIS Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files Now it can be Added or drug into QGIS, shows shapefile as source and it draws the lines, but not the station numbers Numbers are clearly in the table...

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile? Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager...

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote? I'd assumed it was only active when I asked it to do something. Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original? (See [[https://issues.qgis.org/issues/19489]] for the reason I'm paranoid...)

qgib commented 6 years ago

Author Name: Loren Amelang (@LorenAmelang)

I have not duplicated the early problems with saving only .shp and .dbf, instead of all six files. But I have noticed several problems with my later efforts. The Attribute Table views are overlaid in the screenshot, with matching data rows highlighted and copied below.

Comparing the copied data from the Attribute Tables in the screenshot to the original file:

XSCutlines33 multiline:

(Original did not have "Station" when this was saved)

wkt_geom    XsecID  ReachID Station RiverCode   ReachCode   LeftBank    RightBank   LLength ChLength    RLength NodeName    DtmID
MultiLineString ((6182445.5257692439481616 2245399.48681368958204985, 6182721.15499151404947042 2245816.65408449433743954)) 2   10      WALKER GULCH    2                           
MultiLineString ((6182464.27876121643930674 2245387.57281608274206519, 6182762.44966983143240213 2245788.93812257470563054))    1   10      WALKER GULCH    2

Changed to MultiLineString! Rows re-ordered Remaining data blank, not zero

Gulch multiline:

wkt_geom    XsecID  ReachID Station RiverCode   ReachCode   LeftBank    RightBank   LLength ChLength    RLength NodeName    DtmID
MultiLineString ((6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)) 1   10  6998.78176718143    WALKER GULCH    2                           
MultiLineString ((6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848))  2   10  7144.87310829749    WALKER GULCH    2

Changed to MultiLineString! Rows re-ordered Remaining data blank, not zero

Gulch gpkg:

wkt_geom    fid XsecID  ReachID Station RiverCode   ReachCode   LeftBank    RightBank   LLength ChLength    RLength NodeName    DtmID
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    1   10  6999    0   2                               
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)   3   10  7378    0   2                               
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848) 2   10  7145    0   2

Added the "fid" label, didn't seem to add a data column Reduced float "Station" to integer "RiverCode" text changed to zeros Remaining data blank, not zero

Original: (I spaced out and edited labels here to fit columns - but your "pre" is different from my TextPad!)

wkt_geom                                                                                                            XsecID  ReachID Station             RiverCode   RchCode LftBank RgtBank LLength ChLngth RLength NdeName DtmID
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    1   10  6998.78176718143    WALKER GULCH    2   0   0   0   0   0       0
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)   3   10  7378.2609221161     WALKER GULCH    2   0   0   0   0   0       0
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848) 2   10  7144.87310829749    WALKER GULCH    2   0   0   0   0   0       0

I'm not positive changes in file details are the explanation, but I get errors from RiverGIS when I try to use the changed backup files:

https://postgis.net/docs/manual-2.4/ST_LineLocatePoint.html @ST_LineLocatePoint — Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point, as a fraction of total 2d line length. Synopsis float8 ST_LineLocatePoint(geometry a_linestring, geometry a_point); @

Here's the error message:

ST_LineLocatePoint(path.geom, ST_Intersection(r.geom, path.geom))

2st arg isn't a point

So the first path.geom didn't error, thus ST_Intersection(r.geom... must be the problem

https://postgis.net/docs/manual-2.4/ST_Intersection.html

@ST_Intersection — (T) Returns a geometry that represents the shared portion of geomA and geomB. SELECT ST_AsText(ST_Intersection('POINT(0 0)'::geometry, 'LINESTRING ( 2 0, 0 2 )'::geometry)); st_astext @ --> So it can return a point, but also a point set, or other geometry...

I'm way out of my depth here, but it seems like sending it a MultiLineString triggers the error, even if the actual string seems to have only two points in the data. (A lot of other quirks in the source file also trigger the same error, so it is hard to be sure...)

So I just installed QGIS 3.2.1. Let's see what it makes...

Select Original file Export -> Save Features As Make new folder, open it, enter file name, ok Saved six files, shows on map, but no station numbers drawn, changed to MultiLineString Same data as before

Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Shows on map, still LineString, but station numbers all zero in Table, integer in actual data Numbers not drawn Same data as before Maybe the function that draws Station numbers insists on a float value, not integer?

Processing -> Toolbox -> GDAL -> Vector conversion -> Convert format -> .gpkg Hmmm... It shows three listings for my XSCutLines source file... They have identical popup descriptions, in the Postgre database. Picked first one, chose .gpkg output file, ok... Yet a different row order Hey - it saved Station as a float, in a gpkg! And they show in the Attribute Table! But the numbers still are not drawn on the screen:

wkt_geom    fid XsecID  ReachID Station RiverCode   ReachCode   LeftBank    RightBank   LLength ChLength    RLength NodeName    DtmID
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848) 49  2   10  7144.87310829749    WALKER GULCH    2                           
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)   48  3   10  7378.2609221161 WALKER GULCH    2                           
LineString (6182872.53668023273348808 2245141.03621458914130926, 6182984.4070006487891078 2245638.22226869175210595)    51  14  10  8625.57504443754    WALKER GULCH    2                           
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    50  1   10  6998.78176718143    WALKER GULCH    2

Same but export to .sqlite: Much smaller file, column names all lowercase, but Station still float, shows in Table Same data order as gpkg export. Still no numbers drawn. 

wkt_geom    ogc_fid xsecid  reachid station rivercode   reachcode   leftbank    rightbank   llength chlength    rlength nodename    dtmid
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848) 49  2   10  7144.87310829749    WALKER GULCH    2                           
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)   48  3   10  7378.2609221161 WALKER GULCH    2                           
LineString (6182872.53668023273348808 2245141.03621458914130926, 6182984.4070006487891078 2245638.22226869175210595)    51  14  10  8625.57504443754    WALKER GULCH    2                           
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    50  1   10  6998.78176718143    WALKER GULCH    2

So it looks like Processing -> Toolbox -> GDAL -> Vector conversion -> Convert format -> .gpkg is the clear winner, avoiding most of the problems all the other routes create.

But the Station numbers still don't get drawn on the map. How do I discover why?

Does nobody else care about all the other "little glitches" in reproducing the original file? Is preparing input files for HEC-RAS via RiverGIS really so much more picky than what other people use QGIS for?

qgib commented 6 years ago

Author Name: Jürgen Fischer (@jef-n)

Right-clicked a layer XSCutLines, Export -> Save Features As... (with load to map checked)

After Exporting a copy to shapefile...

Attribute Table data looked good:

wkt_geom XsecID ReachID Station RiverCode ReachCode LeftBank RightBank LLength ChLength RLength NodeName DtmID MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715)) 69 10 4253.06211053154 WALKER GULCH 2
...

Attribute Tables and data match, copy and original, include stationing Original draws stationing Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back...

Saved project, closed and restarted QGIS Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files Now it can be Added or drug into QGIS, shows shapefile as source and it draws the lines, but not the station numbers Numbers are clearly in the table...

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile? Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager...

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote? I'd assumed it was only active when I asked it to do something. Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original? (See [[https://issues.qgis.org/issues/19489]] for the reason I'm paranoid...) to After yesterday's file corruption debacle, I'm trying to make backups of complex creations.

Right-clicked a layer XSCutLines, Export -> Save Features As... (with load to map checked)

After Exporting a copy to shapefile...

Attribute Table data looked good:

wkt_geom XsecID ReachID Station RiverCode ReachCode LeftBank RightBank LLength ChLength RLength NodeName DtmID MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715)) 69 10 4253.06211053154 WALKER GULCH 2
...

Attribute Tables and data match, copy and original, include stationing Original draws stationing Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back...

Saved project, closed and restarted QGIS Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files Now it can be Added or drug into QGIS, shows shapefile as source and it draws the lines, but not the station numbers Numbers are clearly in the table...

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile? Database -> DB Manager -> Table -> Export to vector file -> GeoPackage Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager...

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote? I'd assumed it was only active when I asked it to do something. Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original? (See #27317 for the reason I'm paranoid...)

qgib commented 5 years ago

Author Name: Jürgen Fischer (@jef-n)

Bulk closing 82 tickets in feedback state for more than 90 days affecting an old version. Feel free to reopen if it still applies to a current version and you have more information that clarify the issue.