search

olivierkes / manuskript

A open-source tool for writers
http://www.theologeek.ch/manuskript
GNU General Public License v3.0
1.77k stars 235 forks source link

outliner folder gets renamed and contents replaced when dragged over other folder, only in upward direction #719

Open bartart3d opened 4 years ago

bartart3d commented 4 years ago

Hi all,

I've got a weird, but serious issue with the latest manuskript edition, 0.11, but I can't reproduce it with a newly created document.

In the outliner, when I drag a folder up, so, more to the beginning of the document tree, the first folder it is being dragged over ends up being renamed to the same name as the folder that I am dragging. The contents is being replaced too. Wen dragging a folder down the document tree, further away from the beginning, this does not happen. Perhaps this info can pin down the cause of it.

First I had automatic saving on when exiting. Luckily I had a very recent backup. From now on I uncheck 'save on project close' for every new file I make. Over the years I have encountered way too many bugs causing data loss. I strongly suggest this to become the default setting for the time being.

bartart3d commented 4 years ago

I just reïnstalled 0.10.0-1, and there it happens too.

gedakc commented 4 years ago

Did you expand the destination folder and look inside?

For example when I drag the Chapter 5 folder on top of the Chapter 1 folder then the Chapter 5 folder becomes a child of the Chapter 1 folder.

Manuskript-Drag-Folder-Up-Example

When I tried the steps you mentioned, the screen shot above is what occurred. Hence I am unable to reproduce the issue you described.

bartart3d commented 4 years ago

Hi, @gedakc : thanks for looking into it. I couldn't reproduce it with a new file either, but no, it's not what you describe. The end result of the bug are two different folders having te same name, with the same contents, not nested.

gedakc commented 4 years ago

I also tried dragging a lower folder and dropping on top of a higher folder with an existing project and was unable to reproduce the error you experienced.

If you are able to determine the steps with a new project that cause the issue then that would go a long way towards identifying the problem.

bartart3d commented 4 years ago

I'm starting to think it is because of a slow, or bad disk. I already mentioned in earlier reports the slow rendering of a folder full of text files as one long text. It could take minutes to appear on the screen.

I've been looking inside the manuskript file at what happens when dragging folders and moving them around, and if I understand it correctly, files and folders are actually being renamed or copied, or overwritten, to change the order since there is no database or xml tree involved in the actual document. I only saw an xml file for the plots.

I have copied to file to another drive, and tested dragging the folders around, and nothing went wrong. This does sound like a bad drive to me, but I can't be really sure of course.

bartart3d commented 4 years ago

I have this issue again today, not with folders but with files. Yesterday, while writing I decided to move a series of scenes into a new chapter (folder), so I created the new folder and dragged the scenes into it. Then I read everything through and saved the file.

Today I reopen the file and I notice that the files back in the original chapter (folder), but not in the right order. In the new chapter (folder), only a part of the files I yesterday have moved are also present (so they've been duplicated, present in both folders). These are in the right order.

Trying to fix things, I dragged one of the files in the original folder up the scene tree to fix the ordering, which caused another file to be renamed, now having two of the same text files, renamed and contents overwritten. Again, when dragging down the tree, nothing went wrong.

In a previous comment I thought it was because of a faulty disk, but since I'm now working on an entirely different machine, it is very unlikely. I'm certain there is something fundamental going wrong in the saving process. It certainly is not secure, that is obvious to me.

the manuskript file is not saved as a single file, revisions are disabled and autosave is luckily disabled too. I'm using 0.11.0 now, on Ubuntu.

I absolutely enjoy this piece of software, but it the project I'm working on is way to valuable for me. I'll move to something else for the time being, but I'll keep the manuskript file for testing purposes if anyone asks me tot try something out to get it finally fixed.

bartart3d commented 4 years ago

Because it's sometimes difficult to describe issues in words (even when learning to write a novel ;-) ), I thought about gif's :-). I managed to make some screencasts to illustrate the issue. Dragging a scene up the document tree, as you can see 'ontbijt 1' becomes 'ontbijt 2' too. The contents is overwritten too (not in the gif of course): vokoscreen-2020-06-08_21-34-33

When dragging down the tree, nothing goes wrong: vokoscreen-2020-06-08_21-37-15

Biep commented 4 years ago

I had the same thing, with a new file I added and then moved up. https://github.com/olivierkes/manuskript/issues/796#issuecomment-680263157

Biep commented 4 years ago

I see you are a contributor (and Dutch - 👋). Is there anything you can do yourself to repair this?

bartart3d commented 4 years ago

I'm hardly a contributor in terms of coding. I did work on some translation to Dutch though. I have coded in the past, in php and I wish I had the time for it, but I have too much to write to code :-).

bartart3d commented 3 years ago

After a new version being released, 0.12.0 I thought of trying it out again, but the issue remains.

Good news though! I think I found a way to reproduce the problem for others to check! Maybe finally a solution can be found!

Step by step guide on how I encountered the issue on a new file. I had started writing a new project in Joplin, so I used that text to copy paste. Of course, choose you own text:

  1. Created new file, I chose 'novel' there were 20 chapters, 5 scene's each
  2. In chapter one I deleted scene 2 to 5.
  3. In scene 1, I pasted all the text I had already written of chapter 1 (+/- 2500 words)
  4. At about 1000 words I split the scene (menu organise, split at cursor)
  5. Now two scenes are present in chapter one: Scene 1 and Scene 1_2.
  6. Drag Scene 1_2 up and release when it's in the first position.
  7. You now might see (here it is the case) that Scene 1_2 is present twice, both containing the text of Scene 1_2

I hope others can reproduce it this way

TheJackiMonster commented 3 years ago

@bartart3d Thanks for the detailed description. I will try to reproduce but I already have an idea how the code could look like causing the issue. So this can probably be solved.

TheJackiMonster commented 3 years ago

Hmm... nevermind I can't reproduce the issue. At the last point I just get Scene 1_2 before Scene 1 and the text in both scenes is different and as expected from the regarding split.

TheJackiMonster commented 3 years ago

@bartart3d Which operating system do you use? Maybe it's related to the filesystem or something... do you use the single-file mode for a project?

bartart3d commented 3 years ago

Hey,

This is a really weird and annoying bug. Now I´m running ubuntustudio 20.10, filesystem is ext4. It happens both in single file mode and without single file mode.

This is going on for years now and happens on both of my computers, both linux. So weird it´s so hard to reproduce by others while it happens here all the time on two computers.

Op di 4 mei 2021 15:40 schreef Tobias Frisch @.***>:

@bartart3d https://github.com/bartart3d Which operating system do you use? Maybe it's related to the filesystem or something... do you use the single-file mode for a project?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/olivierkes/manuskript/issues/719#issuecomment-831949837, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVYKTW25J2LUDJOQMPYCZLTL72LNANCNFSM4KLEG3QA .

TheJackiMonster commented 3 years ago

Now I´m running ubuntustudio 20.10, filesystem is ext4. It happens both in single file mode and without single file mode.

Mhmm, okay I am using Archlinux with ext4 on my system. I'll try a VM to reproduce, maybe it's caused by some inconsistent versions of dependencies since Arch uses newest stable releases and Ubuntu does usually wait longer to update these.

Biep commented 3 years ago

I am on Ubuntu 20.04 - and the corresponding older version of Manuskript - still 0.10.0. Here it happens too, but not consistently. But then I haven't been experimenting with it.

TheJackiMonster commented 3 years ago

Okay, so I tried to reproduce it in UbuntuStudio 20.04 now because I didn't find a download for 20.10 on the website. But I still can't reproduce the issue with 0.10.0 which can be installed from repository.

Actually I wanted to reproduce the issue with 0.12.0 but I can't run manuskript from source because I get an unhandled exception from Qt somehow:

qt5ct: using qt5ct plugin
WARNING> Qt WebEngine seems to be initialized from a plugin. Please set Qt::AA_ShareOpenGLContexts using QCoreApplication::setAttribute before constructing QGuiApplication.
CRITICAL> An unhandled exception has occurred!
Traceback (most recent call last):
  File "./bin/manuskript", line 13, in <module>
    main.run()
  File "/home/thejackimonster/github/manuskript/bin/../manuskript/main.py", line 291, in run
    app, MW = prepare(arguments)
  File "/home/thejackimonster/github/manuskript/bin/../manuskript/main.py", line 170, in prepare
    from manuskript.mainWindow import MainWindow
  File "/home/thejackimonster/github/manuskript/bin/../manuskript/mainWindow.py", line 29, in <module>
    from manuskript.ui.exporters.exporter import exporterDialog
  File "/home/thejackimonster/github/manuskript/bin/../manuskript/ui/exporters/exporter.py", line 12, in <module>
    from manuskript.ui.exporters.exporter_ui import Ui_exporter
ImportError: cannot import name 'Ui_exporter' from 'manuskript.ui.exporters.exporter_ui' (/home/thejackimonster/github/manuskript/bin/../manuskript/ui/exporters/exporter_ui.py)

It's very weird that the package from repository just works though. Also I can install and run the build .deb package from 0.12.0 release and don't get an issue while startup. I can not reproduce this issue here as well.

So I'm really clueless what's the actual problem. I recommend updating your system overall maybe it will fix this issue but I'm unsure (I assume you already tried that?).

bartart3d commented 3 years ago

Hi, yes, indeed, I run my daily updates. The first time I reported this bug was januari 2020. The issue doesn't happen consistently here either, it just happened again at first test Trying to reproduce it again it did not happen here either. So, it's an unreliable bug, but a bug it is, it does happen to others too and it keeps happening, even though not consistently.

It was the reason why I strongly recommended to disable automatic saving by default, and I see it's enabled by default again This bug, combined with automatic saving caused me to loose al of data in the past, which was for me a reason to abandon Manuskript altogether. Again, I strongly advise to disable automatic saving because of this bug.

Speaking of which, single file mode was enabled by default, I had to disable it when the new file was created. I guess you asked about it because it is thought to cause problems, why is it on by default then? Shouldn't it be safer to be off? People can't disable it without a file open.

I read this latest version has a new logging system, can it be used to track the cause of a problem like this? If so, I'm willing to use manuskript on this project, backing up constantly, logging what's happening and see what shows up. I only need an explanation on how to use this logging system and how to read it.

grtz, Bart

TheJackiMonster commented 3 years ago

Speaking of which, single file mode was enabled by default, I had to disable it when the new file was created. I guess you asked about it because it is thought to cause problems, why is it on by default then? Shouldn't it be safer to be off? People can't disable it without a file open.

The single-file-mode is mostly for compatibility to older files, I think. Storing a project in multiple files is much better for versioning, editing with other tools, arranging your project, getting an overview and it's less probable to get issues though to no need to use any compression or extraction.

I am currently working on refactoring the whole IO system to make loading faster, safer and reduce the codebase as well. I will probably also change the default to opt in for single-file-mode. Also I plan to practically not use the single-file-mode at all (the new system will just extract all files from the one of the project to a new temporary directory... everything else will act the same way as using multiple files in the background but saving will still overwrite the one archive).

I read this latest version has a new logging system, can it be used to track the cause of a problem like this? If so, I'm willing to use manuskript on this project, backing up constantly, logging what's happening and see what shows up. I only need an explanation on how to use this logging system and how to read it.

You can get to your log file by pressing Shift+F1 or via the Help-menu. After closing Manuskript the log-file should be complete and consist all logged commands but it's probably still a problem that many functions in the application don't get logged. Once to not flood the system with log-files and second because it's rather new functionality.

So maybe something can be read from those but I can't promise anything. It's definitely better to post your log if the issue appears as long as I can't reproduce it. I mean technically the inconsistency is a hint how it could get caused. I fixed some multi-threading related bugs in the past which only appear randomly. I suspect this here is something similar or maybe related to the auto-saving or another timed functionality... so whenever the timing fits, the issue becomes possible. I just can only speculate which function is the problem. ^^'

bartart3d commented 3 years ago

Hey,

the log is still present from the test I ran for my first post today. Getting too tired to see if it shows where it goes wrong, I pinpointed the actions I described though, copy pasting them here in case someone here can see what goes wrong.

2021-05-04 11:09:55,990 - manuskript.mainWindow - INFO - Project The Letters.msk saved.
2021-05-04 11:10:23,028 - qt.qpa.xcb - WARNING - QXcbConnection: XCB error: 3 (BadWindow), sequence: 21504, resource id: 25287335, major code: 40 (TranslateCoords), minor code: 0
2021-05-04 11:10:29,962 - manuskript.load_save.version_1 - INFO - Saving to: folder
2021-05-04 11:10:29,979 - manuskript.load_save.version_1 - DEBUG - Saving to folder The Letters
2021-05-04 11:10:29,988 - manuskript.load_save.version_1 - DEBUG - * Removing outline/00-Chapter_1/3-Scene_4.md
2021-05-04 11:10:29,988 - manuskript.load_save.version_1 - DEBUG - * Removing outline/00-Chapter_1/2-Scene_3.md
2021-05-04 11:10:29,989 - manuskript.load_save.version_1 - DEBUG - * Removing outline/00-Chapter_1/1-Scene_2.md
2021-05-04 11:10:29,989 - manuskript.load_save.version_1 - DEBUG - * Removing outline/00-Chapter_1/4-Scene_5.md
2021-05-04 11:10:30,013 - manuskript.mainWindow - INFO - Project The Letters.msk saved.
2021-05-04 11:10:45,827 - manuskript.load_save.version_1 - INFO - Saving to: folder
2021-05-04 11:10:45,876 - manuskript.load_save.version_1 - DEBUG - Saving to folder The Letters
2021-05-04 11:10:45,894 - manuskript.mainWindow - INFO - Project The Letters.msk saved.
2021-05-04 11:10:57,962 - manuskript.load_save.version_1 - INFO - Saving to: folder
2021-05-04 11:10:57,963 - manuskript.load_save.version_1 - DEBUG - Scene 1_2 has been renamed (outline/00-Chapter_1/0-Scene_1.md → outline/00-Chapter_1/1-Scene_1-2.md)
2021-05-04 11:10:57,963 - manuskript.load_save.version_1 - DEBUG -  → We mark for moving: outline/00-Chapter_1/0-Scene_1.md
2021-05-04 11:10:57,982 - manuskript.load_save.version_1 - DEBUG - Saving to folder The Letters
2021-05-04 11:10:57,982 - manuskript.load_save.version_1 - DEBUG - * Renaming/moving outline/00-Chapter_1/0-Scene_1.md to outline/00-Chapter_1/1-Scene_1-2.md
2021-05-04 11:10:57,983 - manuskript.load_save.version_1 - DEBUG -   * Updating cache: outline/00-Chapter_1/0-Scene_1.md, outline/00-Chapter_1/1-Scene_1-2.md
2021-05-04 11:10:57,983 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/folder.txt (different)
2021-05-04 11:10:58,043 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/0-Scene_1.md (not in cache)
2021-05-04 11:10:58,044 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/1-Scene_1-2.md (different)
2021-05-04 11:10:58,138 - manuskript.mainWindow - INFO - Project The Letters.msk saved.
2021-05-04 11:11:06,961 - manuskript.load_save.version_1 - INFO - Saving to: folder
2021-05-04 11:11:06,962 - manuskript.load_save.version_1 - DEBUG - Scene 1_2 has been renamed (outline/00-Chapter_1/1-Scene_1-2.md → outline/00-Chapter_1/0-Scene_1-2-121.md)
2021-05-04 11:11:06,962 - manuskript.load_save.version_1 - DEBUG -  → We mark for moving: outline/00-Chapter_1/1-Scene_1-2.md
2021-05-04 11:11:06,963 - manuskript.load_save.version_1 - DEBUG - Scene 1_2 has been renamed (outline/00-Chapter_1/1-Scene_1-2.md → outline/00-Chapter_1/1-Scene_1-2-121.md)
2021-05-04 11:11:06,963 - manuskript.load_save.version_1 - DEBUG -  → We mark for moving: outline/00-Chapter_1/1-Scene_1-2.md
2021-05-04 11:11:06,986 - manuskript.load_save.version_1 - DEBUG - Saving to folder The Letters
2021-05-04 11:11:06,986 - manuskript.load_save.version_1 - DEBUG - * Renaming/moving outline/00-Chapter_1/1-Scene_1-2.md to outline/00-Chapter_1/0-Scene_1-2-121.md
2021-05-04 11:11:06,987 - manuskript.load_save.version_1 - DEBUG -   * Updating cache: outline/00-Chapter_1/1-Scene_1-2.md, outline/00-Chapter_1/0-Scene_1-2-121.md
2021-05-04 11:11:06,988 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/folder.txt (different)
2021-05-04 11:11:06,988 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/1-Scene_1-2-121.md (not in cache)
2021-05-04 11:11:07,000 - manuskript.load_save.version_1 - DEBUG - * Removing outline/00-Chapter_1/0-Scene_1.md
2021-05-04 11:11:07,006 - manuskript.mainWindow - INFO - Project The Letters.msk saved.
2021-05-04 11:19:24,122 - manuskript.loadSave - INFO - Loading: /media/bart/DATA/The Letters.msk
2021-05-04 11:19:24,123 - manuskript.loadSave - INFO - Detected file format version: 1. Zip: False.
2021-05-04 11:19:24,123 - manuskript.load_save.version_1 - DEBUG - Loading /media/bart/DATA/The Letters.msk (folder)
TheJackiMonster commented 3 years ago

Okay, I think the error is caught on the log because "Scene 1_2 has been renamed" appears two times in a row which marks and re-marks the same scene for moving. I assume this causes the removal of "0-Scene-1.md" in the end which is the critical data loss.

In comparison this is a log from me trying to reproduce but not getting an issue:

2021-05-04 20:05:17,037 - manuskript.mainWindow - INFO - Project Bug.msk saved.
2021-05-04 20:05:53,734 - manuskript.load_save.version_1 - INFO - Saving to: folder
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - Scene 1_2 has been renamed (outline/00-Chapter_1/0-Scene_1.md → outline/00-Chapter_1/0-Scene_1-2.md)
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG -  → We mark for moving: outline/00-Chapter_1/0-Scene_1.md
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - Scene 1 has been renamed (outline/00-Chapter_1/0-Scene_1.md → outline/00-Chapter_1/1-Scene_1.md)
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG -  → We mark for moving: outline/00-Chapter_1/0-Scene_1.md
2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - Saving to folder Bug
2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - * Renaming/moving outline/00-Chapter_1/0-Scene_1.md to outline/00-Chapter_1/0-Scene_1-2.md
2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG -   * Updating cache: outline/00-Chapter_1/0-Scene_1.md, outline/00-Chapter_1/0-Scene_1-2.md
2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/folder.txt (different)
2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/0-Scene_1-2.md (different)
2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/1-Scene_1.md (not in cache)
2021-05-04 20:05:53,742 - manuskript.mainWindow - INFO - Project Bug.msk saved.

So we probably have to adjust Manuskript that it either cancels renaming if it causes overwrite or that it cancels the first renaming if a file gets renamed twice in a row. From looking at the log this is pretty much inconsistent asynchronous IO handling which causes the issue... fortunately this is something I wanted to refactor anyway. So I think it makes more sense to get the whole refactoring done than trying to fix this issue inside of the old code.

bartart3d commented 3 years ago

Hey,

that is great news! Now we're getting somewhere :-D !

Thank you for looking into it, I'm really looking forward to the refactored version :) If you need someone for testing.. just shout!

grtz, Bart

https://esmiltania.be On Twitter https://twitter.com/Bart_Issimo On Google+ https://plus.google.com/u/0/b/116379400376517483499/

Op di 4 mei 2021 om 20:22 schreef Tobias Frisch @.***>:

Okay, I think the error is caught on the log because "Scene 1_2 has been renamed" appears two times in a row which marks and re-marks the same scene for moving. I assume this causes the removal of "0-Scene-1.md" in the end which is the critical data loss.

In comparison this is a log from me trying to reproduce but not getting an issue:

2021-05-04 20:05:17,037 - manuskript.mainWindow - INFO - Project Bug.msk saved.

2021-05-04 20:05:53,734 - manuskript.load_save.version_1 - INFO - Saving to: folder

2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - Scene 1_2 has been renamed (outline/00-Chapter_1/0-Scene_1.md → outline/00-Chapter_1/0-Scene_1-2.md)

2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/00-Chapter_1/0-Scene_1.md

2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - Scene 1 has been renamed (outline/00-Chapter_1/0-Scene_1.md → outline/00-Chapter_1/1-Scene_1.md)

2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/00-Chapter_1/0-Scene_1.md

2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - Saving to folder Bug

2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - * Renaming/moving outline/00-Chapter_1/0-Scene_1.md to outline/00-Chapter_1/0-Scene_1-2.md

2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - * Updating cache: outline/00-Chapter_1/0-Scene_1.md, outline/00-Chapter_1/0-Scene_1-2.md

2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/folder.txt (different)

2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/0-Scene_1-2.md (different)

2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/1-Scene_1.md (not in cache)

2021-05-04 20:05:53,742 - manuskript.mainWindow - INFO - Project Bug.msk saved.

So we probably have to adjust Manuskript that it either cancels renaming if it causes overwrite or that it cancels the first renaming if a file gets renamed twice in a row. From looking at the log this is pretty much inconsistent asynchronous IO handling which causes the issue... fortunately this is something I wanted to refactor anyway. So I think it makes more sense to get the whole refactoring done than trying to fix this issue inside of the old code.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/olivierkes/manuskript/issues/719#issuecomment-832147916, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVYKTX5HHTLO34AATWNNUDTMA3MXANCNFSM4KLEG3QA .

RedOctober11 commented 3 years ago

I'm not sure if this would be helpful, but I'm experiencing the same issue on both 0.11 and 0.12, both set for single file saving. I'm running Manjaro 5.10, and the version of Manuskript was built from the AUR through pamac. I'm seeing it moving individual text pages, not just folders, but only in larger/older files. I couldn't recreate it with a new file. I'll include what looks like the relavent lines from the log, I can include more if requested. I moved a single text page around a few times until it happened, and it did seem to only occur when moving the page upwards in the tree.

2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - 3-6 has been renamed (outline/2-3/5-3-6.md → outline/2-3/1-3-6-12.md) 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/2-3/5-3-6.md 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - 3-6 has been renamed (outline/2-3/5-3-6.md → outline/2-3/2-3-6-12.md) 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/2-3/5-3-6.md 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - 3-3 has been renamed (outline/2-3/2-3-3.md → outline/2-3/3-3-3.md) 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/2-3/2-3-3.md 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - 3-4 has been renamed (outline/2-3/3-3-4.md → outline/2-3/4-3-4.md) 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/2-3/3-3-4.md 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - 3-5 has been renamed (outline/2-3/4-3-5.md → outline/2-3/5-3-5.md) 2021-05-21 15:39:39,357 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/2-3/4-3-5.md

Please let me know if I can help by providing any more information. I really enjoy using Manuskript so I am happy to help improve it for others any way I can.

bartart3d commented 2 years ago

Hello,

just to let know that this issue is not solved yet in 0.13.1! I just tested it, the same issue continues to happen.

Any ideas on when it can be solved? It seems like quite a critical bug though.... I'd love to start using manuskript again, but as long as this is not solved, I cannot use it, way too risky for me.

I guess I'll have to stick with novelWriter, also a great creative writing application!

gedakc commented 2 years ago

For a possible work around this issue did not occur for me if I always drag chapters/scenes down never up.

bartart3d commented 2 years ago

@gedakc : This is at most 'avoiding the bug to happen' and your suggestion is already suggested by myself in the title.

I don't work with software that harms my data, period.

olivierkes commented 2 years ago

@bartart3d, can you run manuskript from source, and if so could you check https://github.com/olivierkes/manuskript/pull/1002? It might be a fix.

bartart3d commented 2 years ago

@olivierkes, hey, I tried it as soon as I could :-) In terminal I can indeed see the id's being renamed as I open the document.

The issue is not solved though, as soon as I dragged up, the file got renamed and contents overwritten.

olivierkes commented 2 years ago

Thanks for the feedback @bartart3d .

Could you try to:

bartart3d commented 2 years ago

@olivierkes I assume you want me to close the file to save it, because of the autosave function and then try again. I had autosave disabled, not to destroy the file accidentally.

So, these were my steps now:

I looked up which files had which ID's, so I looked inside the folders. The ID's that had been fixed, were not those of the documents I dragged up. The fixed ID's were in documents of later chapters (folders), empty documents.

olivierkes commented 2 years ago

Thanks @bartart3d. In my tests, I was able to reproduce the bug only on items with duplicate IDs. But clearly that's not the issue here.

Could you send me your file to see if I can reproduce the bug with it on my end?

bartart3d commented 2 years ago

@olivierkes : I've created a reduced version of the file, the problem is still happening here, consistently when I drag Scene 1_2 up, to above Scene 1 in Chapter 2. I tried the same thing in chapter 3, it happens there too when dragging Scene 3 up to above Scene 2. Those don't even contain any text. I haven't tried any further, since it already happens on two scenes.

I zipped the project, because I did not use a single-file option, revisions should be disabled, just extract and you should have the project in the state as it is on my drive.

bugfixVersion.zip

bartart3d commented 2 years ago

@olivierkes : it has been a while since I sent the file in the comment above. Maybe you didn't get a notification because I had forgotten to mention you in it, changed that in an edit. Curious if it has been of help.

TheJackiMonster commented 2 years ago

I've now merged the PR to potentially fix it because I think it should address a huge portion. If there's still a problem, I will try to fix it. But I want to make a new release in the next week which will hopefully get rid of the issue since we have multiple people reporting issues with ID conflicts.

bartart3d commented 2 years ago

@TheJackiMonster : Thanks for merging, but the issue I reported does not seem fixed here... too bad.

I installed the newest version, 0.14.0 from .deb, having python 3.9.7, PyQT 5.15.4 Qt 5.15.2 .

Using the zipped document in attachment here , Scene 1 in Chapter 2 ends up being renamed as Scene 1_2 when dragged up to before Scene 1, and it's contents is also replaced.

If fixed, this would not be the case. I'd suggest to reopen the report.

TheJackiMonster commented 2 years ago

Hmm, that's unfortunate. My thinking was that the partial fix would at least help with multiple issues and I currently don't have enough time to check into it myself. I think we still need to replace the back-end in a way that we can be sure at all times there's no state with ID collisions. Also manual changes in files from outside the application should be handled properly and we probably need a versioning/revision tool for that like GIT or SVN.

I don't think it makes sense to work-around single issues when the whole data storage is still not as reliable as it should be and we don't even have the option to process partial changes in files to gain better performance.

Anyway the issue is reopened then.