search

doorstop-dev / doorstop

Requirements management using version control.
https://doorstop.readthedocs.io
Other
465 stars 126 forks source link

HTML Publisher is not working #628

Closed villanella closed 6 months ago

villanella commented 6 months ago

Hello! There are some problems in Doorstop v3.0b12 with refactored HTML publisher :

  1. Publish error 
    $ doorstop publish all output -H
building tree...
loading documents...
publishing tree to '/home/username/doorstop/doorstop-rtm/output'...
Traceback (most recent call last):
File "/usr/bin/doorstop", line 8, in <module>
sys.exit(main())
         ^^^^^^
File "/usr/lib/python3.11/site-packages/doorstop/cli/main.py", line 177, in main
success = function(args, os.getcwd(), parser.error)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/doorstop/cli/commands.py", line 567, in run_publish
published_path = publisher.publish(
                 ^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/doorstop/core/publisher.py", line 85, in publish
common.write_lines(
File "/usr/lib/python3.11/site-packages/doorstop/common.py", line 189, in write_lines
for line in lines:
File "/usr/lib/python3.11/site-packages/doorstop/core/publisher.py", line 121, in publish_lines
yield from gen(obj, **kwargs)
File "/usr/lib/python3.11/site-packages/doorstop/core/publishers/html.py", line 293, in lines
doc_attributes = get_document_attributes(
                 ^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/doorstop/core/publishers/base.py", line 323, in get_document_attributes
if "name" in attribute_defaults["doc"]:
             ~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'doc'
  2. Bad formatting in doorstop-server - it seems there is no templates applied:

pic-selected-240211-0156-36

neerdoc commented 6 months ago

Hi @villanella,

  1. Can you supply the .doorstop.yml files for the document that fails? And also the printout with the --verbose flag set.
  2. This looks correct to me as the new default. As I wrote in the release information, the whole html output was remade, including the templating. The reasons being: a) consolidating the html export with the doorstop-server so that they use the same templating, and b), a lot of the information in the template output was hard coded and not changeable by a template. I know the documentation is a bit bad on this part, but please try to do a default html export and all the template files will be included in the exported folder and give you at good starting point on how to create your own template in the new way.
villanella commented 6 months ago

Hello @neerdoc, thank you for helping me with debug!

  1. This config file contains custom attribute:

    settings:
digits: 3
itemformat: markdown
parent: ROOT
prefix: REQ
sep: '.'
attributes:
defaults:
component:

    Verbose log:

    ...
loading document ADH-2122-UC's items...
loading document ADH-2122-REQ's items...
loading document ADH-2122-TST's items...
Deleting contents of template directory /home/username/doorstop/doorstop-rtm/output/template
Copying /usr/lib/python3.11/site-packages/doorstop/core/files/templates/html to /home/username/doorstop/doorstop-rtm/template
Copying /usr/lib/python3.11/site-packages/doorstop/core/../views to /home/username/doorstop/doorstop-rtm/output/template/views
Template = doorstop
publishing to /home/username/doorstop/doorstop-rtm/output/documents/ROOT.html...
publishing to /home/username/doorstop/doorstop-rtm/output/documents/OVERVIEW.html...
publishing to /home/username/doorstop/doorstop-rtm/output/documents/REQ.html...
Traceback (most recent call last):
File "/usr/bin/doorstop", line 8, in <module>
........ [error message here] ....
KeyError: 'doc'

    If I removes attributes, the publish result of this doc is successful. All docs at once I can't publish because of my RAM (according the error zsh: killed doorstop publish all output -H --verbose). I have specs with a lot of items.

  2. Default template can't render markdown tables. How can I fix it? Maybe I apply the templates in incorrect way. My templates are located at the next path:

    Deleting contents of template directory /home/username/doorstop/doorstop-rtm/output/template
Copying /usr/lib/python3.11/site-packages/doorstop/core/files/templates/html to /home/username/doorstop/doorstop-rtm/template
Copying /usr/lib/python3.11/site-packages/doorstop/core/../views to /home/username/doorstop/doorstop-rtm/output/template/views
Template = doorstop
fwuehr1995 commented 6 months ago

I am facing the same issue, it really seems to be related to the available RAM. Is there any way of sequencing the publishing in order to limit the necessary available RAM ?

neerdoc commented 6 months ago
  1. I realise that I've been cheating a bit on the checking of this information in the .doorstop.yml file. I will fix as soon as I can. In the meantime, please test to add the doc attribute with a "dummy" text. It should help. Like this: 
    attributes:
defaults:
component:
doc:
  dummy
  2. For the second problem with the template: The template folder should be placed alongside the documents. So for a document tree you would place the files like this: 
    reqs
├── REQ001.yml
├── assets
│   └── logo-black-white.png -> ../../docs/images/logo-black-white.png
├── template
│   ├── bootstrap.bundle.min.js
│   ├── bootstrap.min.css
│   ├── doorstop.css
│   ├── general.css
│   ├── jquery.min.js
│   ├── logo-black-white.png
│   ├── output
│   │   ├── chtml
│   │   │   └── fonts
│   │   │       ├── tex.js
│   │   │       └── woff-v2
│   │   │           ├── MathJax_AMS-Regular.woff
│   │   │           ├── MathJax_Calligraphic-Bold.woff
│   │   │           ├── MathJax_Calligraphic-Regular.woff
│   │   │           ├── MathJax_Fraktur-Bold.woff
│   │   │           ├── MathJax_Fraktur-Regular.woff
│   │   │           ├── MathJax_Main-Bold.woff
│   │   │           ├── MathJax_Main-Italic.woff
│   │   │           ├── MathJax_Main-Regular.woff
│   │   │           ├── MathJax_Math-BoldItalic.woff
│   │   │           ├── MathJax_Math-Italic.woff
│   │   │           ├── MathJax_Math-Regular.woff
│   │   │           ├── MathJax_SansSerif-Bold.woff
│   │   │           ├── MathJax_SansSerif-Italic.woff
│   │   │           ├── MathJax_SansSerif-Regular.woff
│   │   │           ├── MathJax_Script-Regular.woff
│   │   │           ├── MathJax_Size1-Regular.woff
│   │   │           ├── MathJax_Size2-Regular.woff
│   │   │           ├── MathJax_Size3-Regular.woff
│   │   │           ├── MathJax_Size4-Regular.woff
│   │   │           ├── MathJax_Typewriter-Regular.woff
│   │   │           ├── MathJax_Vector-Bold.woff
│   │   │           ├── MathJax_Vector-Regular.woff
│   │   │           └── MathJax_Zero.woff
│   │   ├── chtml.js
│   │   ├── svg
│   │   │   └── fonts
│   │   │       └── tex.js
│   │   └── svg.js
│   ├── tex-mml-chtml.js
│   └── views
│       ├── base.tpl
│       ├── document_list.tpl
│       ├── my_template.tpl
│       └── item_list.tpl
└── tutorial
├── TUT001.yml

    Then you can publish like this: doorstop --html --template my_template all output

For the RAM issue: Can either of you contact my directly so that we can discuss if there is a way that you can share your document temporarily for testing as this is not something that I have encountered. It is on the 3.2 milestone to test very large setups, but I have not had the time yet.

fwuehr1995 commented 6 months ago

Hi @neerdoc, I tried to send you my files via mail, "gustav@neer.se" is correct ?

fwuehr1995 commented 6 months ago

@neerdoc I have used your fix from https://github.com/neerdoc/doorstop/commit/3c57c445768895c8fe5d065f743b7e4e51e3febf This fixes the situation for me, so please disregard my mail and my previous comments regarding a potential RAM issue, it works fine for me now !

neerdoc commented 6 months ago

@fwuehr1995 Very good! I'll disregard the email.

villanella commented 6 months ago
  1. I realise that I've been cheating a bit on the checking of this information in the .doorstop.yml file. I will fix as soon as I can. In the meantime, please test to add the doc attribute with a "dummy" text. It should help. Like this:
attributes:
  defaults:
    component:
    doc:
      dummy
2. For the second problem with the template: The template folder should be placed alongside the documents. So for a document tree you would place the files like this:
reqs
├── REQ001.yml
├── assets
│   └── logo-black-white.png -> ../../docs/images/logo-black-white.png
├── template
│   ├── bootstrap.bundle.min.js
│   ├── bootstrap.min.css
│   ├── doorstop.css
│   ├── general.css
│   ├── jquery.min.js
│   ├── logo-black-white.png
│   ├── output
│   │   ├── chtml
│   │   │   └── fonts
│   │   │       ├── tex.js
│   │   │       └── woff-v2
│   │   │           ├── MathJax_AMS-Regular.woff
│   │   │           ├── MathJax_Calligraphic-Bold.woff
│   │   │           ├── MathJax_Calligraphic-Regular.woff
│   │   │           ├── MathJax_Fraktur-Bold.woff
│   │   │           ├── MathJax_Fraktur-Regular.woff
│   │   │           ├── MathJax_Main-Bold.woff
│   │   │           ├── MathJax_Main-Italic.woff
│   │   │           ├── MathJax_Main-Regular.woff
│   │   │           ├── MathJax_Math-BoldItalic.woff
│   │   │           ├── MathJax_Math-Italic.woff
│   │   │           ├── MathJax_Math-Regular.woff
│   │   │           ├── MathJax_SansSerif-Bold.woff
│   │   │           ├── MathJax_SansSerif-Italic.woff
│   │   │           ├── MathJax_SansSerif-Regular.woff
│   │   │           ├── MathJax_Script-Regular.woff
│   │   │           ├── MathJax_Size1-Regular.woff
│   │   │           ├── MathJax_Size2-Regular.woff
│   │   │           ├── MathJax_Size3-Regular.woff
│   │   │           ├── MathJax_Size4-Regular.woff
│   │   │           ├── MathJax_Typewriter-Regular.woff
│   │   │           ├── MathJax_Vector-Bold.woff
│   │   │           ├── MathJax_Vector-Regular.woff
│   │   │           └── MathJax_Zero.woff
│   │   ├── chtml.js
│   │   ├── svg
│   │   │   └── fonts
│   │   │       └── tex.js
│   │   └── svg.js
│   ├── tex-mml-chtml.js
│   └── views
│       ├── base.tpl
│       ├── document_list.tpl
│       ├── my_template.tpl
│       └── item_list.tpl
└── tutorial
    ├── TUT001.yml

Then you can publish like this: doorstop --html --template my_template all output

For the RAM issue: Can either of you contact my directly so that we can discuss if there is a way that you can share your document temporarily for testing as this is not something that I have encountered. It is on the 3.2 milestone to test very large setups, but I have not had the time yet.

This dummy code is working, thank you:

    doc:
      dummy

Only the error with publishing tables is still actual. My publish template structure is the same as yours, but no tables are rendered in html output and in doorstop-server.

villanella commented 6 months ago

@neerdoc This is an example how tables are rendered in doorstop-server and doorstop publish output

Screen 1: pic-selected-240215-2224-41

Screen 2: pic-selected-240215-2224-10