metanorma / metanorma-docker

Docker container for running the Metanorma toolchain
https://www.metanorma.com
5 stars 3 forks source link

metanorma compile fails with sassc error #119

Closed archaeogeek closed 2 years ago

archaeogeek commented 2 years ago

Hello, this is my first foray into metanorma and I've hit a problem. I'm running the docker container on Ubuntu 20.04, and am running the following command:

docker run -it -v "$(pwd)":/metanorma/ -w /metanorma metanorma/mn:latest metanorma compile -t generic -x html myadoc.adoc

This fails with the following error:

Traceback (most recent call last):
        36: from /usr/local/bundle/bin/metanorma:23:in `<main>'
        35: from /usr/local/bundle/bin/metanorma:23:in `load'
        34: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/exe/metanorma:40:in `<top (required)>'
        33: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/exe/metanorma:22:in `block in <top (required)>'
        32: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli.rb:34:in `start'
        31: from /usr/local/bundle/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
        30: from /usr/local/bundle/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
        29: from /usr/local/bundle/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
        28: from /usr/local/bundle/gems/thor-hollaback-0.2.0/lib/thor/hollaback.rb:62:in `run'
        27: from /usr/local/bundle/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
        26: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli/command.rb:45:in `compile'
        25: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli/command.rb:45:in `each'
        24: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli/command.rb:45:in `block in compile'
        23: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli/command.rb:210:in `compile_document'
        22: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli/compiler.rb:35:in `compile'
        21: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli/compiler.rb:30:in `compile'
        20: from /usr/local/bundle/bundler/gems/metanorma-cli-dc3c6c044cf2/lib/metanorma/cli/compiler.rb:45:in `compile_file'
        19: from /usr/local/bundle/gems/metanorma-1.4.1/lib/metanorma/compile.rb:35:in `compile'
        18: from /usr/local/bundle/gems/metanorma-1.4.1/lib/metanorma/compile.rb:206:in `process_extensions'
        17: from /usr/local/bundle/gems/metanorma-1.4.1/lib/metanorma/compile.rb:206:in `each'
        16: from /usr/local/bundle/gems/metanorma-1.4.1/lib/metanorma/compile.rb:225:in `block in process_extensions'
        15: from /usr/local/bundle/gems/metanorma-generic-1.11.0/lib/metanorma/generic/processor.rb:43:in `output'
        14: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/html_convert.rb:26:in `convert'
        13: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/convert.rb:175:in `convert'
        12: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/html_function/html.rb:9:in `convert1'
        11: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/function/utils.rb:33:in `noko'
        10: from /usr/local/bundle/gems/nokogiri-1.12.5-x86_64-linux/lib/nokogiri/xml/builder.rb:292:in `with'
         9: from /usr/local/bundle/gems/nokogiri-1.12.5-x86_64-linux/lib/nokogiri/xml/builder.rb:292:in `new'
         8: from /usr/local/bundle/gems/nokogiri-1.12.5-x86_64-linux/lib/nokogiri/xml/builder.rb:334:in `initialize'
         7: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/html_function/html.rb:10:in `block in convert1'
         6: from /usr/local/bundle/gems/nokogiri-1.12.5-x86_64-linux/lib/nokogiri/xml/builder.rb:416:in `method_missing'
         5: from /usr/local/bundle/gems/nokogiri-1.12.5-x86_64-linux/lib/nokogiri/xml/builder.rb:433:in `insert'
         4: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/html_function/html.rb:12:in `block (2 levels) in convert1'
         3: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/css.rb:28:in `populate_css'
         2: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/css.rb:100:in `generate_css'
         1: from /usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/css.rb:78:in `convert_scss'
/usr/local/bundle/gems/isodoc-1.8.1/lib/isodoc/css.rb:78:in `require': cannot load such file -- sassc (LoadError)

However if I run the following instead: docker run -it -v "$(pwd)":/metanorma/ -w /metanorma metanorma/mn:latest metanorma compile -t iso -x html myadoc.adoc Then the command finishes.

Note that I get the same results regardless of which metanorma container I use.

I'm very unfamiliar with ruby, but OK with command line debugging and generally poking around in docker containers, so I dropped into a running container but couldn't find sassc amongst the gems.

I have attempted to manually install the sassc gem using gem install sassc and while that seemed to complete without error it didn't solve the problem. Variations on bundle install also failed but I really don't know the best approach here so it's possible I didn't do this correctly.

I'd be very grateful for any assistance people can provide- step by step instructions would be really nice if there's any further debugging I can do to help resolve the issue.

Thanks in advance

Jo

ronaldtse commented 2 years ago

Thanks @archaeogeek for raising this issue.

By any chance you have a "Gemfile" in the document folder? Typically a user should not need to do anything with the "sassc" gem.

@CAMOBAP can you take a look at this?

CAMOBAP commented 2 years ago

Last ci looks good, @ronaldtse you are right, it's critical to check Gemfile here

@archaeogeek could you please share your Gemfile?

archaeogeek commented 2 years ago

Thanks for the prompt response. I should have said initially that I am following the "simple adoption" approach from https://www.metanorma.org/builder/topics/simple-adoption/ and all I have is a metanorma.yml file, which is based almost exactly on the default one listed:

metanorma_name: generic
organization_name_short: myorg
organization_name_long: My Org Name
document_namespace: https://www.metanorma.org/standards/generic
xml_root_tag: 'generic-standard'
logo_path: /config/mylogo.jpg
logo_paths: []
validate_rng_file: lib/asciidoctor/generic/generic.rng
htmlcoverpage: lib/isodoc/generic/html/html_generic_titlepage.html
htmlintropage: lib/isodoc/generic/html/html_generic_intro.html
htmlstylesheet: lib/isodoc/generic/html/htmlstyle.scss
scripts: lib/isodoc/generic/html/scripts.html
standardstylesheet: lib/isodoc/generic/html/generic.scss
header: lib/isodoc/generic/html/header.html
wordcoverpage: lib/isodoc/generic/html/word_generic_titlepage.html
wordintropage: lib/isodoc/generic/html/word_generic_intro.html
wordstylesheet: lib/isodoc/generic/html/wordstyle.scss
#i18nyaml: lib/isodoc/generic/html/i18n-en.yml
boilerplate: lib/asciidoctor/generic/boilerplate.xml
docid_template: "{{ organization_name_short }} {{ docnumeric }}"
published_stages: ["published", "withdrawn"]
committees: []
relations: []
stage_abbreviations:
  draft-proposal:
  draft-development:
  draft-testing:
  draft-implementation:
  in-force:
  retired:
metadata_extensions: []
doctypes: ['standard']
default_doctype: standard
termsdefs_titles:
  - Terms and definitions
  - Terms, definitions, symbols and abbreviated terms
  - Terms, definitions, symbols and abbreviations
  - Terms, definitions and symbls
  - Terms, definitions and abbreviations
  - Terms, definitions and abbreviated terms
symbols_titles:
  - Symbols and abbreviated terms
  - Symbols
  - Abbreviated terms
  - Abbreviations
normref_titles:
  - Normative references
bibliography_titles:
  - Bibliography

Then in my asciidoc file I just have :customize: metanorma.yml under the title.

The Simple Adoption approach with the yml file implies that's all I need- rather than attempting the ruby gem approach (this being my first attempt at metanorma and ruby)- unless I've missed something in the instructions?

CAMOBAP commented 2 years ago

@archaeogeek thanks for information

@ronaldtse to be honest I didn't try such scenario of using metanorma. Are it still valid one, if so it looks like documentation is outdated because we don't use scss in production as far as I remember, could you please confirm?

CAMOBAP commented 2 years ago

@ronaldtse I'm not sure how to update this documentation, we need to report task about it in https://github.com/metanorma/metanorma.com

@opoudjis maybe you can help and suggest some workaround here? maybe we can just drop htmlstylesheet, standardstylesheet, wordstylesheet?

opoudjis commented 2 years ago

Dear everybody,

The gem sassc is used to compile SCSS stylesheets into CSS stylesheets; SCSS allows us to inject variables (such as the names of fonts) into the stylesheet.

When we release standardised gems, part of our release involves compiling SCSS into CSS. Because we wanted to remove the dependency of Metanorma on sassc by default, when we compile the SCSS into CSS, we remove the need for sassc.

If you pull a pre-release Metanorma flavour gem from github, you find you have to include sassc into any Gemfile, because the CSS stylesheets are not checked in to Github: they are only compiled on release.

If you build a brand new flavour, as is the case here, you have two options:

  1. Use CSS stylesheets instead of SCSS; that way, there is nothing to compile (so all
  2. Add "require sassc" to your Gemfile.

The third option of dropping htmlstylesheet, standardstylesheet, wordstylesheet... I would not advise, because styling matters; without a stylesheet, output is going to look pretty bad.

I'll mention sassc now as a requirement in https://www.metanorma.org/builder/topics/simple-adoption/ ; the dropping of the sassc gem requirement is recent, and this documentation hasn't been updated to reflect that.

opoudjis commented 2 years ago

@CAMOBAP I can't really help with docker, because I don't know my way around it. @archaeogeek the simplest thing for you to do, rather than try and get sassc to work, is to rename the scss stylesheets as css stylesheets, and replace any instances of variables (e.g. font-family:$monospacefont;) with full values.

CAMOBAP commented 2 years ago

@opoudjis thanks a lot for workaround, this is more then enough

@archaeogeek could you please follow suggested workaround, or please let us know if this isn't option for you.

@ronaldtse I have two notes:

archaeogeek commented 2 years ago

@opoudjis / @CAMOBAP Thanks for pursuing this. I just wanted to clarify something- if I am using the metanorma.yml approach and docker, when you say "rename the scss stylesheets as css stylesheets", presumably you mean making the changes inside the docker container and not simply changing the references to them in my local yml file?

CAMOBAP commented 2 years ago

@archaeogeek as fat as I understood metanorma.yml comes from your host machine, so it should be not a problem to modify it.

opoudjis commented 2 years ago

@CAMOBAP I've already done the documentation update: https://github.com/metanorma/metanorma.org/commit/41e2cde0c7633940112f1b5e2be15cdcdd749ce6

CAMOBAP commented 2 years ago

@opoudjis thanks a lot, BTW how do you think should we include sass into our docker image?

archaeogeek commented 2 years ago

Hello, it has taken me a while to figure out how to do it, but I have succeeded in trying the work-around of renaming the *.scss files to *.css inside the docker container, and that does seem to fix my problem.

For future reference, I did the following (there might be a better approach but it worked):

  1. Get a running container that would stay persistent with my working directory mounted, and find it's name: docker run -it --entrypoint bash -v "$(pwd)":/metanorma/ metanorma/mn:latest
  2. Use docker exec to rename the files: docker exec -it {mycontainername} /bin/bash
  3. Inside the container rename (actually copy and rename) the scss files in the folder /usr/local/bundle/gems/metanorma-generic-1.11.1/lib/isodoc/generic/html
  4. Edit my local metanorma.yml to reference the css rather than the scss
  5. Change the compile command to use docker exec rather than docker run: docker exec -it -w /metanorma {mycontainername} metanorma compile -t generic -x html 1037-uk-gemini-standard-2-3-revised.adoc

Since that seems to work OK, I think I can build a docker-compose file that includes the css versions of the files to make this a bit easier to reproduce until there's a fix in the standard images.

Thanks for all your help!

ronaldtse commented 2 years ago

@archaeogeek thank you so much for the update and for charging through. We will update our documentation to reflect your experience and incorporate the necessary changes in our next release to make it easier. Thanks!

opoudjis commented 2 years ago

As for including sassc in docker... i think the answer is yes, but I don't remember why sassc was considered harmful to begin with. @ronaldtse ?

Because I don't understand docker, could I ask that @CAMOBAP do the documentation update? I'm assuming this needs to be a new page under https://www.metanorma.org/builder/ , rather than part of https://www.metanorma.org/install/docker/ , but I'm actually not sure at all.

ronaldtse commented 2 years ago

sassc was considered harmful

  1. sassc requires the user to install a package that is not commonly available on many platforms, and compilation often fails on those platforms.
  2. sassc is also not necessary to be run since the user is only including a static CSS file in all flavors.

@CAMOBAP can you please help update documentation? Thanks.

CAMOBAP commented 2 years ago

@ronaldtse https://github.com/metanorma/metanorma.org/pull/535 done