Test suite failing with many different errors

pabluk commented 8 years ago

After fix the dependency issue https://github.com/Antojitos/chalupas/issues/11 the test suite is running again but all tests have failed:

running build_ext
test_docx_to_html (tests.main.ChalupasTestCase)
DOCX to HTML ... ERROR
test_docx_to_md (tests.main.ChalupasTestCase)
DOCX to MD ... ERROR
test_docx_to_rst (tests.main.ChalupasTestCase)
DOCX to RST ... ERROR
test_html_to_docx (tests.main.ChalupasTestCase)
HTML to DOCX ... FAIL
test_html_to_md (tests.main.ChalupasTestCase)
HTML to MD ... FAIL
test_html_to_rst (tests.main.ChalupasTestCase)
HTML to RST ... FAIL
test_md_to_docx (tests.main.ChalupasTestCase)
HTML to DOCX ... FAIL
test_md_to_html (tests.main.ChalupasTestCase)
MD to HTML ... FAIL
test_md_to_rst (tests.main.ChalupasTestCase)
HTML to RST ... FAIL
test_rst_to_docx (tests.main.ChalupasTestCase)
RST to DOCX ... FAIL
test_rst_to_html (tests.main.ChalupasTestCase)
RST to HTML ... FAIL
test_rst_to_md (tests.main.ChalupasTestCase)
RST to MD ... FAIL

see https://travis-ci.org/Antojitos/chalupas/jobs/118807260 for details about the errors.

pabluk commented 8 years ago

It seems it's related to the pandoc version 1.12.2.1-1build2 installed on Ubuntu 14.04 Trusty (used by Travis), it doesn't include docx as input format:

$ pandoc --help
pandoc [OPTIONS] [FILES]
Input formats:  docbook, haddock, html, json, latex, markdown, markdown_github,
                markdown_mmd, markdown_phpextra, markdown_strict, mediawiki,
                native, opml, rst, textile
Output formats: asciidoc, beamer, context, docbook, docx, dzslides, epub, epub3,
                fb2, html, html5, json, latex, man, markdown, markdown_github,
                markdown_mmd, markdown_phpextra, markdown_strict, mediawiki,
                native, odt, opendocument, opml, org, pdf*, plain, revealjs,
                rst, rtf, s5, slideous, slidy, texinfo, textile
                [*for pdf output, use latex or beamer and -o FILENAME.pdf]

$ dpkg -l | grep pandoc
ii  pandoc                            1.12.2.1-1build2                    amd64        general markup converter
ii  pandoc-data                       1.12.2.1-1build2                    all          general markup converter - data files

The same command in Ubuntu 15.10 Wily shows docx as a valid input format:

$ pandoc -h                                                                                                                                                   
pandoc [OPTIONS] [FILES]
Input formats:  docbook, docx, epub, haddock, html, json, latex, markdown,
                markdown_github, markdown_mmd, markdown_phpextra,
                markdown_strict, mediawiki, native, opml, org, rst, t2t,
                textile, twiki
Output formats: asciidoc, beamer, context, docbook, docx, dokuwiki, dzslides,
                epub, epub3, fb2, haddock, html, html5, icml, json, latex, man,
                markdown, markdown_github, markdown_mmd, markdown_phpextra,
                markdown_strict, mediawiki, native, odt, opendocument, opml,
                org, pdf*, plain, revealjs, rst, rtf, s5, slideous, slidy,
                texinfo, textile
                [*for pdf output, use latex or beamer and -o FILENAME.pdf]

$ dpkg -l |grep pandoc
ii  pandoc                               1.13.2.1~dfsg-1build6                 armhf        general markup converter
ii  pandoc-data                          1.13.2.1~dfsg-1build6                 all          general markup converter - data files

pabluk commented 8 years ago

The support for docx as input format was added in pandoc 1.13 http://pandoc.org/releases.html#pandoc-1.13-15-august-2014

pabluk commented 8 years ago

It's worst, Travis use Ubuntu 12.04 Precise by default so it's installing pandoc 1.9.1.1-1, we need to figure out how to install a recent version of pandoc on Ubuntu 12.04 https://docs.travis-ci.com/user/installing-dependencies/

Luckily our production servers are using Ubuntu 15.10 :smile:

amessinger commented 8 years ago

How I forgot I install a .deb archive from pandoc website on my dev machine to solve this very issue :x !!!

amessinger commented 8 years ago

Maybe we should switch to a curl + dpkg instead of apt-get

pabluk commented 8 years ago

Great! I will try this option https://docs.travis-ci.com/user/installing-dependencies/#Installing-Packages-without-an-APT-Repository

amessinger commented 8 years ago

shouldn't it be done by ansible?

pabluk commented 8 years ago

shouldn't it be done by ansible?

We need to fix it for both of them, the ansible recipe and the travis environment.

The fix https://github.com/Antojitos/chalupas/commit/dfde9020fd7e6b9a4b99bec33170df6cdbb70888 works, the docx tests don't fail anymore https://travis-ci.org/Antojitos/chalupas/jobs/118822916 but there are some other errors.

pabluk commented 8 years ago

Using self.assertEqual https://github.com/Antojitos/chalupas/commit/2c93832acfcb2d0c8a72e16aa929d43552bef511 now we have more details about failing tests:

AssertionError: 'application/msword' != 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'

It seems that the magic module detects different mime types.

@amessinger which version of python-magic have you used for you tests ? The test suite uses python-magic==0.4.11 https://github.com/Antojitos/chalupas/blob/master/setup.py#L36

Antojitos / chalupas

Test suite failing with many different errors #12