search

vkbo / novelWriter

novelWriter is an open source plain text editor designed for writing novels. It supports a minimal markdown-like syntax for formatting text. It is written with Python 3 (3.9+) and Qt 5 (5.15) for cross-platform support.
https://novelwriter.io
GNU General Public License v3.0
2.05k stars 106 forks source link

novelWriter lacks an import capability #63

Closed johnblommers closed 4 years ago

johnblommers commented 5 years ago

Enhancement Request

novelWriter has a very nice set of tools to allow the novelist to organize and analyze their manuscript. That capability is only available as material is manually added to novelWriter. Once the novel is exported to Markdown, Text or HTML5 for editing using a grammar checker or a human editor, there is no way back. That's limiting.

How wonderful it would be if novelWriter could import a Markdown file and assign chapter folders and scenes automatically guided by the H1 and H2 headers in the file.

IMHO only Markdown needs to be importable. The novelist can readily convert an ODT or a DOCX or even an HTML file into Markdown using Pandoc.

vkbo commented 5 years ago

Hi, and thanks for the feedback.

novelWriter is of course still in pre-release, so not complete, so a few more features are planned.

Something similar to what you propose is already on my list. What I want to add is the option to split a single file up into multiple files by a selectable heading level, so H1 to H4. This is actually what the file type "book" is intended for. In principle, you could copy/paste the markdown into such a file and then split it. Adding a feature to load a markdown file from disk would be a trivial addition of course, so I can certainly add this too.

Since my markdown format allows for comments and commands, that regular markdown does not, I'm not sure how to preserve this in case of an export/import of the same project.

johnblommers commented 5 years ago

novelWriter is of course still in pre-release, so not complete, so a few more features are planned.

Absolutely I understand completely. I'm developing a lot of enthusiasm for novelWriter mostly because it's feature set is going in the right direction as seen from my little perch.

Since my markdown format allows for comments and commands, that regular markdown does not, I'm not sure how to preserve this in case of an export/import of the same project.

I really like the current implementation of comments. If memory serves Markdown processors generally pass non-Markdown text to the output so HTML comments might be a good fit. Such comments will survive the export to Markdown directly and on export to HTML they're be preserved as HTML comments.

One use case I envision is that, at some point in its development, I want to export a novel to Markdown and run the text over a good grammar checker such as ProWritingAid. I'd want to keep all of the special tags and Markdown comments intact during this edit. Then I'll want to import the manuscript back into novelWriter and have it understand the tags and commands already in it. Indeed IMHO it would be a disaster if all the tags and commands vanished on import.

Another use case I envision is that I'd export a novel to Markdown and then to DOCX format using Pandoc so that I can pass it on for human review. I'd still want all of the tags and commands left intact and command the human editors to leave them alone. After making changes based on review comments I'd convert the DOCX back to Markdown with Pandoc. Then I'd import the Markdown file with all its tags and commands intact back into novelWriter.

Note that I'm not a fan of DOCX per se. But the file format is useful because most people running Linux, macOS or Windows have one tool or another that can read and write it. At an rate, what happens after the user exports to Markdown isn't novelWriter's issue. It's up to the user to keep the tags, commands and comments intact as the manuscript traverses the treacherous DOCX badlands.

vkbo commented 5 years ago

novelWriter is of course still in pre-release, so not complete, so a few more features are planned.

Absolutely I understand completely. I'm developing a lot of enthusiasm for novelWriter mostly because it's feature set is going in the right direction as seen from my little perch.

I'm happy to implement suggestions that fit within the scope of the application. That is, I want to keep it a markdown type plain text editor with files I can track with versioning tools like git.

Since my markdown format allows for comments and commands, that regular markdown does not, I'm not sure how to preserve this in case of an export/import of the same project.

I really like the current implementation of comments. If memory serves Markdown processors generally pass non-Markdown text to the output so HTML comments might be a good fit. Such comments will survive the export to Markdown directly and on export to HTML they're be preserved as HTML comments.

Not a bad idea. I'll add it as an option to the export tool. It could wrap both meta data and comments in <!--, --> tags, and then parse those in reverse on import.

One use case I envision is that, at some point in its development, I want to export a novel to Markdown and run the text over a good grammar checker such as ProWritingAid. I'd want to keep all of the special tags and Markdown comments intact during this edit. Then I'll want to import the manuscript back into novelWriter and have it understand the tags and commands already in it. Indeed IMHO it would be a disaster if all the tags and commands vanished on import.

I'll add an option to export the novelWriter comments and tags as-is for markdown export. Which means the file can be read back in again with the import from file option I added yesterday.

Another use case I envision is that I'd export a novel to Markdown and then to DOCX format using Pandoc so that I can pass it on for human review. I'd still want all of the tags and commands left intact and command the human editors to leave them alone. After making changes based on review comments I'd convert the DOCX back to Markdown with Pandoc. Then I'd import the Markdown file with all its tags and commands intact back into novelWriter.

I will add export to open document files, as there is a package available to handle such files for Python. Alternatively, I can export to .fodt files, which are just flat XML files. This is what FocusWriter does.

Note that I'm not a fan of DOCX per se. But the file format is useful because most people running Linux, macOS or Windows have one tool or another that can read and write it. At an rate, what happens after the user exports to Markdown isn't novelWriter's issue. It's up to the user to keep the tags, commands and comments intact as the manuscript traverses the treacherous DOCX badlands.

A lot of applications I've used offer no good way back after an export, but I'd like to have the option to do this at least to a limited extent. Editors (the human kind) I've worked with in the past have used docx comments to provide feedback, so I may look into supporting these to, at least via odt format. But that will imply I need a full import tool similar to the current export.

I got a lot of time on my hands early next year, so I will try and reach the end of my list of features for a 1.0 release by February, give or take.

johnblommers commented 4 years ago

Should release 1.0 happen as early as February 2020 it will be in plenty of time for Camp NaNoWriMo. You might consider reaching out to this organization at that time to introduce novelWriter.

johnblommers commented 4 years ago

May I ask for an indication of when you might implement an import feature? I fully respect your schedule and your priorities so no pressure.

The import feature as discussed above would be huge, in my opinion, because it lets a writer import a draft Markdown file and at once use the organizing power of the file tree, tags and timeline view to wrestle the plot arcs into order.

vkbo commented 4 years ago

I can bump it on my list. Trying to sort out issues with spell checking at the moment. I guess you're referring to the merge/split part? It probably isn't a lot of work since the import itself is in place, and so is the code to parse a file into sections (it's part of the indexer class). Some of the code needed lives in the outline branch though. This is a much bigger one to complete. I can probably merge a part of it, but that implies killing the timeline feature ahead of time.

I've injured a tendon in my hand, so I can only use a keyboard for short periods at a time. I'm on sick leave until the 9th, so it's not going to be very soon.

johnblommers commented 4 years ago

Yes I'm referring to the merge/split part that lets you import a whole Markdown file and let novelWriter break it apart into files and folders.

IMHO you might consider leaving the keyboard alone until the 9th. Take care of number one.

It's too bad we don't have good voice control over our computers in 2019.

johnblommers commented 4 years ago

Happy New Year! May I ask for an indication of when this feature sits in the schedule? No pressure.

vkbo commented 4 years ago

Happy New Year :)

I've started working on it, but I'm travelling this week, so probably not until I'm back. I just managed last week to re-assemble my PC after moving country.

vkbo commented 4 years ago

So, getting the PC ready took a lot longer than I expected. Needed to order new parts. I'm now back in business and should be able to finish this one.

johnblommers commented 4 years ago

Thanks for checking in. It's good to know you're back.

vkbo commented 4 years ago

Ok, I just merged the split and merge functionality. Please give it a try, and make a new issue if there are any bugs.

The split feature works on a single file, and creates a folder with all the split files in it. The merge feature works the other way around, and merges all files in a folder into a single file.

The original file/folder is left in both cases and the user can delete them if they want to. I've also added the option to permanently delete a file from the trash folder.