search

strongdm / comply

Compliance automation framework, focused on SOC2
https://comply.strongdm.com
Apache License 2.0
1.32k stars 248 forks source link

Comply build keeps on running forever #100

Closed chahalamit009 closed 3 years ago

chahalamit009 commented 3 years ago

Comply build commands keeps on running forever in github actions using go setup all other commands run smoothly

ajsharp commented 3 years ago

Seeing this too on Mac, though able to successfully run via Docker.

cybertr0n-com commented 3 years ago

Same here. CentOS stream just hangs:

comply build templates/index.ace -> output/index.html

None of the PDF's are build from the mark up files.

It does work in Docker though.

cybertr0n-com commented 3 years ago

NVM - This was due to missing texlive-xetex and pandoc for me.

wilderuncat commented 3 years ago

Same issue, it works in docker but not in regular CLI. I tried installing pandoc and texlive-xetex but no luck. On MacOS Catalina

statik commented 3 years ago

I was able to get comply running locally on macOS and in github actions recently. I am interested in helping figure out the additional documentation or code changes needed to enable this to run more smoothly.

I think #98 is one of the PRs that needs to be merged. Additionally, I think comply needs to be able to find the pandoc and pdflatex commands. On macOS I was able to install these using brew install pandoc and brew install --cask basictex

gertverhoog commented 3 years ago

Hi all, I have a very similar issue, except comply build seems to hang somewhere after processing a number of files. In the hope that the following is helpful, this is what I've done.

OS: macOS big Sur 11.3.1 (build 20E241) Comply version 1.5.2 Shell: zsh

# First install the tools we need:
$ brew update
$ brew install pandoc librsvg
$ brew install homebrew/cask/basictex
# (I have also tried this with uninstalling basictex and then installing the full mactex instead; same result)

$ brew tap strongdm/comply
$ comply -v
comply version 1.5.2

# the following two commands ensure zsh has an up-to-date PATH
$ eval "$(/usr/libexec/path_helper)"
$ rehash

# confirm that we can find pandoc and tex:
$ which pandoc
/usr/local/bin/pandoc
$ which pdflatex
/Library/TeX/texbin/pdflatex

# Create a new set of templates
$ mkdir mysoc
$ cd mysoc
$ comply init
# (I edited comply.yml to add my gitlab details)
$ comply build

At this point, it starts producing outputs (I do get the index.html and a number of PDF files), although TeX gets confused at the start:

templates/index.ace -> output/index.html
Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 2\tabcolsep...
l.79 ...umnwidth - 2\tabcolsep) * \real{0.86}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 8\tabcolsep...
l.257 ...mnwidth - 8\tabcolsep) * \real{0.21}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 6\tabcolsep...
l.162 ...mnwidth - 6\tabcolsep) * \real{0.22}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 6\tabcolsep...
l.200 ...mnwidth - 6\tabcolsep) * \real{0.30}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 8\tabcolsep...
l.280 ...mnwidth - 8\tabcolsep) * \real{0.20}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 8\tabcolsep...
l.340 ...mnwidth - 8\tabcolsep) * \real{0.20}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 4\tabcolsep...
l.211 ...mnwidth - 4\tabcolsep) * \real{0.64}}@{}}

policies/change.md -> output/MYSOC-SCP.pdf
policies/log.md -> output/MYSOC-LMP.pdf
policies/conduct.md -> output/MYSOC-COCP.pdf
narratives/system.md -> output/MYSOC-SAN.pdf
policies/processing.md -> output/MYSOC-PIP.pdf
policies/vendor.md -> output/MYSOC-VMP.pdf
policies/datacenter.md -> output/MYSOC-DP.pdf
policies/policy.md -> output/MYSOC-PTP.pdf
policies/cyber.md -> output/MYSOC-CRP.pdf
policies/workstation.md -> output/MYSOC-WP.pdf
policies/incident.md -> output/MYSOC-SIRP.pdf
narratives/products.md -> output/MYSOC-PSN.pdf
policies/password.md -> output/MYSOC-PWP.pdf
policies/access.md -> output/MYSOC-AOTP.pdf
policies/office.md -> output/MYSOC-OSP.pdf
policies/continuity.md -> output/MYSOC-BCP.pdf
narratives/organizational.md -> output/MYSOC-ON.pdf
policies/remote.md -> output/MYSOC-REAP.pdf
policies/application.md -> output/MYSOC-ASP.pdf
policies/development.md -> output/MYSOC-SDLCP.pdf
policies/media.md -> output/MYSOC-MCP.pdf
narratives/control.md -> output/MYSOC-CEN.pdf
policies/confidentiality.md -> output/MYSOC-CP.pdf
policies/information.md -> output/MYSOC-ISP.pdf
narratives/security.md -> output/MYSOC-SEN.pdf

At this point, nothing happens. The process doesn't return. ps and top don't show any other relevant processes other than comply build, and it's been more than 20 minutes (which would be very long for a network timeout or something).

Is there anything I could do to see what comply is stuck on? Is there a way to get more verbose output?

nemozny commented 3 years ago

Same here. I've run comply like 5 times and it always gets stuck and never produces these:

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        5/20/2021   3:28 PM           7302 POC-AP.pdf.md
-a----        5/20/2021   3:28 PM          14670 POC-DCP.pdf.md
-a----        5/20/2021   3:28 PM          10602 POC-DRP.pdf.md
-a----        5/20/2021   3:28 PM           5667 POC-EP.pdf.md
-a----        5/20/2021   3:28 PM            554 POC-PMP.pdf.md
-a----        5/20/2021   3:28 PM          10775 POC-RIAP.pdf.md
-a----        5/20/2021   3:28 PM           7091 POC-RP.pdf.md

The rest of the files are normally converted to PDF, as expected. Edit: Windows & MiKTeX

AstroBoy269 commented 3 years ago

I'm having the same kind of issue and don't mean to pile on. I'm running

It appears to have issues with 7 files and outputs files like CHSH-AP.pdf.md orCHSH-DCP.pdf.md however the utility continues to run until it hangs after it creates the CHSH-SEN.pdf No more status messages are displayed no other files are created it just hangs. It seems like the files that get written with the .pdf.md extension all have fancy tables embedded at the end. To validate that I removed the fancy tables and was rewarded with a successfully created file.

It appears that when you start comply it copies all the markdown files over into the output directory and then with the extension *pdf.md. If it's able to successfully convert the file into a PDF then the .pdf.md file will be gone and will be replaced with the the *.pdf file. this has got to be what causes the "Error producing PDF" errors. And the files not converted definitely correspond with the number of "Error producing PDF" messages.

But I don't know if that's what causes comply to appear to hang. I do know the name of the last file displayed is successfully getting created.

I'm happy to help troubleshoot this issue but would need some advice/guidance.

Thanks

Hi all, I have a very similar issue, except comply build seems to hang somewhere after processing a number of files. In the hope that the following is helpful, this is what I've done.

OS: macOS big Sur 11.3.1 (build 20E241) Comply version 1.5.2 Shell: zsh

# First install the tools we need:
$ brew update
$ brew install pandoc librsvg
$ brew install homebrew/cask/basictex
# (I have also tried this with uninstalling basictex and then installing the full mactex instead; same result)

$ brew tap strongdm/comply
$ comply -v
comply version 1.5.2

# the following two commands ensure zsh has an up-to-date PATH
$ eval "$(/usr/libexec/path_helper)"
$ rehash

# confirm that we can find pandoc and tex:
$ which pandoc
/usr/local/bin/pandoc
$ which pdflatex
/Library/TeX/texbin/pdflatex

# Create a new set of templates
$ mkdir mysoc
$ cd mysoc
$ comply init
# (I edited comply.yml to add my gitlab details)
$ comply build

At this point, it starts producing outputs (I do get the index.html and a number of PDF files), although TeX gets confused at the start:

templates/index.ace -> output/index.html
Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 2\tabcolsep...
l.79 ...umnwidth - 2\tabcolsep) * \real{0.86}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 8\tabcolsep...
l.257 ...mnwidth - 8\tabcolsep) * \real{0.21}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 6\tabcolsep...
l.162 ...mnwidth - 6\tabcolsep) * \real{0.22}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 6\tabcolsep...
l.200 ...mnwidth - 6\tabcolsep) * \real{0.30}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 8\tabcolsep...
l.280 ...mnwidth - 8\tabcolsep) * \real{0.20}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 8\tabcolsep...
l.340 ...mnwidth - 8\tabcolsep) * \real{0.20}}@{}}

Error producing PDF.
! Undefined control sequence.
<argument> @{} >{\raggedright \arraybackslash 
                                              }p{(\columnwidth - 4\tabcolsep...
l.211 ...mnwidth - 4\tabcolsep) * \real{0.64}}@{}}

policies/change.md -> output/MYSOC-SCP.pdf
policies/log.md -> output/MYSOC-LMP.pdf
policies/conduct.md -> output/MYSOC-COCP.pdf
narratives/system.md -> output/MYSOC-SAN.pdf
policies/processing.md -> output/MYSOC-PIP.pdf
policies/vendor.md -> output/MYSOC-VMP.pdf
policies/datacenter.md -> output/MYSOC-DP.pdf
policies/policy.md -> output/MYSOC-PTP.pdf
policies/cyber.md -> output/MYSOC-CRP.pdf
policies/workstation.md -> output/MYSOC-WP.pdf
policies/incident.md -> output/MYSOC-SIRP.pdf
narratives/products.md -> output/MYSOC-PSN.pdf
policies/password.md -> output/MYSOC-PWP.pdf
policies/access.md -> output/MYSOC-AOTP.pdf
policies/office.md -> output/MYSOC-OSP.pdf
policies/continuity.md -> output/MYSOC-BCP.pdf
narratives/organizational.md -> output/MYSOC-ON.pdf
policies/remote.md -> output/MYSOC-REAP.pdf
policies/application.md -> output/MYSOC-ASP.pdf
policies/development.md -> output/MYSOC-SDLCP.pdf
policies/media.md -> output/MYSOC-MCP.pdf
narratives/control.md -> output/MYSOC-CEN.pdf
policies/confidentiality.md -> output/MYSOC-CP.pdf
policies/information.md -> output/MYSOC-ISP.pdf
narratives/security.md -> output/MYSOC-SEN.pdf

At this point, nothing happens. The process doesn't return. ps and top don't show any other relevant processes other than comply build, and it's been more than 20 minutes (which would be very long for a network timeout or something).

Is there anything I could do to see what comply is stuck on? Is there a way to get more verbose output?

statik commented 3 years ago

I'm working on tracking down the hang. If you send SIGQUIT to a go program, it will dump stack traces including for each of the outstanding goroutines. After I get comply to hang, here is what I am seeing from ctrl-\

^\SIGQUIT: quit
PC=0x7fff20585cde m=0 sigcode=0

goroutine 0 [idle]:
runtime.pthread_cond_wait(0x1b8b278, 0x1b8b238, 0x7fff00000000)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/sys_darwin.go:384 +0x39
runtime.semasleep(0xffffffffffffffff, 0xb2950144)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/os_darwin.go:63 +0x8d
runtime.notesleep(0x1b8b030)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/lock_sema.go:181 +0xdb
runtime.mPark()
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/proc.go:1340 +0x39
runtime.stopm()
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/proc.go:2257 +0x92
runtime.findrunnable(0xc000053800, 0x0)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/proc.go:2916 +0x72e
runtime.schedule()
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/proc.go:3125 +0x2d7
runtime.goexit0(0xc0004b9b00)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/proc.go:3434 +0x1de
runtime.mcall(0x106ba76)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/asm_amd64.s:327 +0x5b

goroutine 1 [select]:
github.com/strongdm/comply/internal/render.Build(0x175b1eb, 0x6, 0xc00035ed00, 0xc00023b8f0, 0x15afdc9)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/site.go:121 +0x299
github.com/strongdm/comply/internal/cli.buildAction(0xc0001426e0, 0x3, 0xc0002c9220)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/cli/build.go:18 +0x3f
github.com/urfave/cli.HandleAction(0x1630400, 0x1788528, 0xc0001426e0, 0xc0001426e0, 0x0)
    /Users/emurphy/go/pkg/mod/github.com/urfave/cli@v1.22.4/app.go:526 +0x105
github.com/urfave/cli.Command.Run(0x1759d12, 0x5, 0x1758a02, 0x1, 0x0, 0x0, 0x0, 0x177c09f, 0x3c, 0x0, ...)
    /Users/emurphy/go/pkg/mod/github.com/urfave/cli@v1.22.4/command.go:174 +0x579
github.com/urfave/cli.(*App).Run(0xc000338380, 0xc000020040, 0x2, 0x2, 0x0, 0x0)
    /Users/emurphy/go/pkg/mod/github.com/urfave/cli@v1.22.4/app.go:279 +0x808
github.com/strongdm/comply/internal/cli.Main()
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/cli/app.go:36 +0x4f
main.main()
    /Users/emurphy/go/src/github.com/strongdm/comply/comply.go:6 +0x25

goroutine 26 [select]:
net/http.(*persistConn).writeLoop(0xc00040a7e0)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:2382 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:1744 +0xc9c

goroutine 25 [IO wait]:
internal/poll.runtime_pollWait(0x3c55da0, 0x72, 0xffffffffffffffff)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc00041a318, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
    /usr/local/Cellar/go/1.16.3/libexec/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
    /usr/local/Cellar/go/1.16.3/libexec/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc00041a300, 0xc000468000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.16.3/libexec/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc00041a300, 0xc000468000, 0x1000, 0x1000, 0x103a2bc, 0xc000081c38, 0x1065c60)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc000406098, 0xc000468000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/net.go:183 +0x91
net/http.(*persistConn).Read(0xc00040a7e0, 0xc000468000, 0x1000, 0x1000, 0xc000424660, 0xc000081d40, 0x1006d15)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:1922 +0x77
bufio.(*Reader).fill(0xc00042aae0)
    /usr/local/Cellar/go/1.16.3/libexec/src/bufio/bufio.go:101 +0x108
bufio.(*Reader).Peek(0xc00042aae0, 0x1, 0x0, 0x1, 0x4, 0x1, 0x3)
    /usr/local/Cellar/go/1.16.3/libexec/src/bufio/bufio.go:139 +0x4f
net/http.(*persistConn).readLoop(0xc00040a7e0)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:2083 +0x1a8
created by net/http.(*Transport).dialConn
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:1743 +0xc77

goroutine 8 [IO wait]:
internal/poll.runtime_pollWait(0x3c55ae8, 0x72, 0xffffffffffffffff)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc00038a118, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
    /usr/local/Cellar/go/1.16.3/libexec/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
    /usr/local/Cellar/go/1.16.3/libexec/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc00038a100, 0xc000371000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.16.3/libexec/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc00038a100, 0xc000371000, 0x1000, 0x1000, 0x103a2bc, 0xc000485c38, 0x1065c60)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc000010420, 0xc000371000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/net.go:183 +0x91
net/http.(*persistConn).Read(0xc000122d80, 0xc000371000, 0x1000, 0x1000, 0xc000424360, 0xc000485d40, 0x1006d15)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:1922 +0x77
bufio.(*Reader).fill(0xc000273500)
    /usr/local/Cellar/go/1.16.3/libexec/src/bufio/bufio.go:101 +0x108
bufio.(*Reader).Peek(0xc000273500, 0x1, 0x0, 0x1, 0x4, 0x1, 0x3)
    /usr/local/Cellar/go/1.16.3/libexec/src/bufio/bufio.go:139 +0x4f
net/http.(*persistConn).readLoop(0xc000122d80)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:2083 +0x1a8
created by net/http.(*Transport).dialConn
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:1743 +0xc77

goroutine 9 [select]:
net/http.(*persistConn).writeLoop(0xc000122d80)
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:2382 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/Cellar/go/1.16.3/libexec/src/net/http/transport.go:1744 +0xc9c

goroutine 16 [semacquire]:
sync.runtime_Semacquire(0xc00046c008)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc00046c000)
    /usr/local/Cellar/go/1.16.3/libexec/src/sync/waitgroup.go:130 +0x65
github.com/strongdm/comply/internal/render.pdf(0x175b1eb, 0x6, 0x0, 0xc000424cc0, 0xc000498a80)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pdf.go:41 +0x165
created by github.com/strongdm/comply/internal/render.Build
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/site.go:107 +0x191

goroutine 67 [semacquire]:
sync.runtime_Semacquire(0xc000498a88)
    /usr/local/Cellar/go/1.16.3/libexec/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc000498a80)
    /usr/local/Cellar/go/1.16.3/libexec/src/sync/waitgroup.go:130 +0x65
github.com/strongdm/comply/internal/render.Build.func2(0xc000498a80, 0xc000424d20)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/site.go:116 +0x73
created by github.com/strongdm/comply/internal/render.Build
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/site.go:114 +0x226

goroutine 70 [chan send]:
github.com/strongdm/comply/internal/render.pandoc(0xc000193bc0, 0x10, 0xc0000a8060)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pandoc.go:23 +0xbb
github.com/strongdm/comply/internal/render.renderToFilesystem.func1(0xc00046c000, 0xc0006a80c0, 0xc0000a8060, 0xc00060e990)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:42 +0x267
created by github.com/strongdm/comply/internal/render.renderToFilesystem
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:30 +0xf3

goroutine 72 [chan send]:
github.com/strongdm/comply/internal/render.pandoc(0xc000241170, 0x11, 0xc0000a8060)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pandoc.go:23 +0xbb
github.com/strongdm/comply/internal/render.renderToFilesystem.func1(0xc00046c000, 0xc0006a80c0, 0xc0000a8060, 0xc00060eea0)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:42 +0x267
created by github.com/strongdm/comply/internal/render.renderToFilesystem
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:30 +0xf3

goroutine 79 [chan send]:
github.com/strongdm/comply/internal/render.pandoc(0xc000241590, 0x11, 0xc0000a8060)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pandoc.go:23 +0xbb
github.com/strongdm/comply/internal/render.renderToFilesystem.func1(0xc00046c000, 0xc0006a80c0, 0xc0000a8060, 0xc00060fc20)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:42 +0x267
created by github.com/strongdm/comply/internal/render.renderToFilesystem
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:30 +0xf3

goroutine 80 [chan send]:
github.com/strongdm/comply/internal/render.pandoc(0xc0004335b0, 0x10, 0xc0000a8060)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pandoc.go:23 +0xbb
github.com/strongdm/comply/internal/render.renderToFilesystem.func1(0xc00046c000, 0xc0006a80c0, 0xc0000a8060, 0xc00060fdd0)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:42 +0x267
created by github.com/strongdm/comply/internal/render.renderToFilesystem
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:30 +0xf3

goroutine 88 [chan send]:
github.com/strongdm/comply/internal/render.pandoc(0xc000241ba8, 0x11, 0xc0000a8060)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pandoc.go:23 +0xbb
github.com/strongdm/comply/internal/render.renderToFilesystem.func1(0xc00046c000, 0xc0006a80c0, 0xc0000a8060, 0xc000401830)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:42 +0x267
created by github.com/strongdm/comply/internal/render.renderToFilesystem
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:30 +0xf3

goroutine 91 [chan send]:
github.com/strongdm/comply/internal/render.pandoc(0xc00023cfe0, 0x10, 0xc0000a8060)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pandoc.go:23 +0xbb
github.com/strongdm/comply/internal/render.renderToFilesystem.func1(0xc00046c000, 0xc0006a80c0, 0xc0000a8060, 0xc000132090)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:42 +0x267
created by github.com/strongdm/comply/internal/render.renderToFilesystem
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:30 +0xf3

goroutine 92 [chan send]:
github.com/strongdm/comply/internal/render.pandoc(0xc000241db8, 0x12, 0xc0000a8060)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/pandoc.go:23 +0xbb
github.com/strongdm/comply/internal/render.renderToFilesystem.func1(0xc00046c000, 0xc0006a80c0, 0xc0000a8060, 0xc000132480)
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:42 +0x267
created by github.com/strongdm/comply/internal/render.renderToFilesystem
    /Users/emurphy/go/src/github.com/strongdm/comply/internal/render/document.go:30 +0xf3

rax    0x104
rbx    0x5378e00
rcx    0x7ffeefbfe068
rdx    0x2b00
rdi    0x1b8b278
rsi    0x440100004500
rbp    0x7ffeefbfe110
rsp    0x7ffeefbfe068
r8     0x0
r9     0xa0
r10    0x0
r11    0x246
r12    0x1b8b238
r13    0x16
r14    0x440100004500
r15    0x2b00
rip    0x7fff20585cde
rflags 0x247
cs     0x7
fs     0x0
gs     0x0
statik commented 3 years ago

I have a fix for the problem where comply hangs if there is any error code returned from the pandoc command, the fix is in #104. That will stop the hang, and then the user can debug the pandoc error without waiting a very long time for CI jobs to time out.

I believe the fix for the underlying pandoc errors that several folks in this thread have run into is in #98, and adding those two packages to the latex template fixes those particular pandoc errors. Of course there may be other pandoc errors that would also trigger the hang.

camposer commented 3 years ago

I agree that the error handling can be improved. In my case, the problem got fixed just pulling the strongdm/pandoc docker image:

$ docker pull strongdm/pandoc
$ cd my-company
$ go run ../comply.go build
templates/index.ace -> output/index.html
policies/password.md -> output/my-company-PWP.pdf
policies/continuity.md -> output/my-company-BCP.pdf
policies/workstation.md -> output/my-company-WP.pdf
policies/risk.md -> output/my-company-RIAP.pdf
policies/datacenter.md -> output/my-company-DP.pdf
policies/privacy.md -> output/my-company-PMP.pdf
policies/cyber.md -> output/my-company-CRP.pdf
policies/processing.md -> output/my-company-PIP.pdf
policies/office.md -> output/my-company-OSP.pdf
policies/availability.md -> output/my-company-AP.pdf
policies/access.md -> output/my-company-AOTP.pdf
narratives/system.md -> output/my-company-SAN.pdf
policies/confidentiality.md -> output/my-company-CP.pdf
policies/application.md -> output/my-company-ASP.pdf
policies/encryption.md -> output/my-company-EP.pdf
narratives/security.md -> output/my-company-SEN.pdf
policies/policy.md -> output/my-company-PTP.pdf
narratives/organizational.md -> output/my-company-ON.pdf
policies/media.md -> output/my-company-MCP.pdf
policies/change.md -> output/my-company-SCP.pdf
narratives/products.md -> output/my-company-PSN.pdf
policies/vendor.md -> output/my-company-VMP.pdf
policies/incident.md -> output/my-company-SIRP.pdf
policies/log.md -> output/my-company-LMP.pdf
policies/remote.md -> output/my-company-REAP.pdf
narratives/control.md -> output/my-company-CEN.pdf
policies/conduct.md -> output/my-company-COCP.pdf
policies/retention.md -> output/my-company-RP.pdf
policies/disaster.md -> output/my-company-DRP.pdf
policies/classification.md -> output/my-company-DCP.pdf
policies/information.md -> output/my-company-ISP.pdf
policies/development.md -> output/my-company-SDLCP.pdf