Saving CSV edits fails on Windows

sigmoe commented 4 years ago

Describe the bug

When you have a CSV layer in the project, the edit mode is possible, so you can modify the values of the objects of the CSV layer, but when you try to save the layer (to save the modification), an error message appears (OGR Error) .

How to Reproduce

1) Add a CSV file to the QGIS project (by drag and drop from the Windows Explorer) 2) Edit the layer 3) Modify some values 4) Save the layer 5) The OGR Error appears

QGIS and OS versions

Windows 10 - 64 bits

Version de QGIS 3.10.3-A Coruña Révision du code 0e1f846438 Compilé avec Qt 5.11.2 Utilisant Qt 5.11.2 Compilé avec GDAL/OGR 3.0.4 Utilisé avec GDAL/OGR 3.0.4 Compilé avec GEOS 3.8.0-CAPI-1.13.1 Utilisé avec GEOS 3.8.0-CAPI-1.13.1 Compiled against SQLite 3.29.0 Running against SQLite 3.29.0 Version du client PostgreSQL 11.5 Version de SpatiaLite 4.3.0 Version de QWT 6.1.3 Version de QScintilla2 2.10.8 Compilé avec PROJ 6.3.1 Fonctionne avec PROJ Rel. 6.3.1, February 10th, 2020 OS Version Windows 10 (10.0) Extensions Python actives cadastre; ClipMultipleLayers; gban; GeofoncierEditeurRFU; gml_application_schema_toolbox; loadthemall; localiserparcelle; mmqgis; OSMDownloader; qgis2web; QgisLayerBoardPlugin-master; qProf; quick_map_services; refFunctions; SgmCdmTransfo; SgmComparePmAvvApv; SgmDxfImportPro; SgmExpressFunctions; SgmLinesToPoints; SgmVertex2NodeByAtt; StreetView; db_manager; MetaSearch; processing

Additional context Using a python program (little extension), it's totally possible to modify a CSV layer.

gioman commented 4 years ago

Confirmed on Windows. Works on 3.12 on Linux with old dependencies (GDAL 2.4).

jgrocha commented 4 years ago

@sigmoe Which error message? Just "OGR Error"?

Works on 3.13.0-Master, with GDAL 3.1.0dev (on Linux)

sigmoe commented 4 years ago

@jgrocha The Error message: CRITICAL Couche t_baie : Erreur OGR lors la synchronisation vers le disque : Cannot rename files

gioman commented 4 years ago

@sigmoe did it worked ok on 3.4?

sigmoe commented 4 years ago

@gioman same problem with QGIS 3.4

gioman commented 4 years ago

As also on 2.18 so it seems this is long standing issue.

roya0045 commented 4 years ago

see #28703

gioman commented 4 years ago

see #28703

this ticket is not completely accurate. You can edit CSVs on Linux (I can't test macOS now), but not on Windows. So the functionality is there.

stev-0 commented 4 years ago

This is a duplicate of #25421 - can we close that older one?

samillar94 commented 3 years ago

This issue is still here on QGIS 3.16.0. Why is it not a higher priority? To quote Gio from the old forum, "To add insult to injury if the table covers the message bar the user don't see the error and when toggling editing the unsaved values are sticking in the table giving the sensation that edits went well".

bjornharrtell commented 3 years ago

As far as I've been able to verify fix done in https://github.com/qgis/QGIS/pull/39919 is effective.

gioman commented 3 years ago

As far as I've been able to verify fix done in #39919 is effective.

@bjornharrtell so I guess we can close this?

samillar94 commented 3 years ago

Thanks all for your hard work, much appreciated

bjornharrtell commented 3 years ago

@gioman as far as I'm concerned yes, but as disclaimer I will say that I have not been able to reproduce this issue even before the fix.

roya0045 commented 3 years ago

Just let someone test the MXE to see if it truly solves the issue in this case.

stev-0 commented 3 years ago

I did experience the problem and it seems to be fixed in master. However this is only when loading via drag and drop.

When using the Delimited Text widget, I no longer seem to be able to load CSVs at all. Not sure if this is related to this or something else. Can anyone else confirm?

Message is Layer is not valid : The layer file:///C:/Users/Stephen.Knox/Documents/test.csv?type=csv&maxFields=10000&detectTypes=yes&xyDms=yes&geomType=none&subsetIndex=no&watchFile=no is not a valid layer and can not be added to the map. Reason:

Possibly related to #38348?

stev-0 commented 3 years ago

Actually, it isn't present in 3.16, so it could be this that the PR to fix this issue has introduced the problem as the PR I linked above is pre-3.16

gioman commented 3 years ago

When using the Delimited Text widget, I no longer seem to be able to load CSVs at all. Not sure if this is related to this or something else. Can anyone else confirm?

Message is Layer is not valid : The layer file:///C:/Users/Stephen.Knox/Documents/test.csv?type=csv&maxFields=10000&detectTypes=yes&xyDms=yes&geomType=none&subsetIndex=no&watchFile=no is not a valid layer and can not be added to the map. Reason:

@stev-0 qgis version? sample data?

bjornharrtell commented 3 years ago

@stev-0 I also noticed the problem you describe on some CSV content however AFAIK completely unrelated to this issue.

stev-0 commented 3 years ago

@gioman I am using the master (fe05aad7e1) osgeo4w build which has the bugfix for this issue in.

To be more precise, with the this file, it loads fine in both master and 3.16 when dragging and dropping.

However, when loading via Add Layer -> Delimited Text, with the defaults it loads correctly in 3.16 but not in master

@bjornharrtell - guess I should open another ticket.

bjornharrtell commented 3 years ago

@stev-0 please do, and please be more careful when wording the description - I can't make sense where it works and fails for you with what you write above ("loads correctly in 3.16 but not in master" vs "loads fine in both master and 3.16").

stev-0 commented 3 years ago

@stev-0 please do, and please be more careful when wording the description - I can't make sense where it works and fails for you with what you write above ("loads correctly in 3.16 but not in master" vs "loads fine in both master and 3.16").

whoops - yes added that in to the reply

gioman commented 3 years ago

However, when loading via Add Layer -> Delimited Text, with the defaults it loads correctly in 3.16 but not in master

@stev-0 confirmed, please file a new ticket.

gioman commented 3 years ago

Just let someone test the MXE to see if it truly solves the issue in this case.

@roya0045 I tested https://github.com/qgis/QGIS/suites/1486274517/artifacts/25792318 and works ok while on master/ccdff23dbe is not, but the latter probably does not have the fix. Closing?

gioman commented 3 years ago

@stev-0 confirmed, please file a new ticket.

@stev-0 I did it https://github.com/qgis/QGIS/issues/39968

roya0045 commented 3 years ago

@gioman Since the commits reference this issue it should close when merged.

stev-0 commented 3 years ago

@roya0045 it should have closed but hasn't so I think someone needs to close it manually. It's fixed from my point of view.

roya0045 commented 3 years ago

@stev-0 you're right, when I gave a quick look I saw that the backport was not merged and though it was still pending...

djagar commented 2 years ago

The error is still present in the actual LTR version 3.22, exactly as described in the first post, for both Windows 10 Home 64bit and Ubuntu LTS 20.04 64bit:

Error: Layer xyz: OGR error syncing to disk: Error while creating xyz.csv_ogr_tmp.csv

Ubuntu LTS 20.04.4 LTS: QGIS version 3.22.4-Białowieża QGIS code revision ce8e65e95e Qt version 5.12.8 Python version 3.8.10 GDAL/OGR version 3.0.4 PROJ version 6.3.1 EPSG Registry database version v9.8.6 (2020-01-22) Compiled against GEOS 3.8.0-CAPI-1.13.1 Running against GEOS 3.8.0-CAPI-1.13.1 SQLite version 3.31.1 PDAL version 2.0.1 PostgreSQL client version 12.9 (Ubuntu 12.9-0ubuntu0.20.04.1) SpatiaLite version 4.3.0a QWT version 6.1.4 QScintilla2 version 2.11.2 OS version Ubuntu 20.04.4 LTS Active Python plugins processing 2.12.99 grassprovider 2.12.99 MetaSearch 0.3.5 db_manager 0.1.20 sagaprovider 2.12.99

qgis / QGIS

Saving CSV edits fails on Windows #35144