Builds using an old revision

[simonmar]

When I do a build, the result seems to correspond to an old version of the repo. Furthermore, the Stats tab tells me that the most recent commit is 2012-10-02, when I have pushed many changes since then (including today, 2012-11-06).

http://ano-atlas2.herokuapp.com/books/1230000000929

[runemadsen]

Thanks @simonmar. If I press the "changes" tab, I can see the changes you made today. The stats are only updated after builds. I'll look into this and keep you updated.

[runemadsen]

Hmm... Something weird is happening here. I'll keep you updated.

[simonmar]

@runemadsen great, thanks. In the meantime I can build locally, so it's not urgent.

[runemadsen]

@sandersk This is really weird. I've checked out the entire build process, and the updated files are on the worker server at the time the PDF build is triggered. It also uploads the new file to S3, but the content of the PDF is from an old commit. I suspect that it has something to do with the PDF generation, but I have no clue how it should reference old content.

This is the log output. Can you see anything weird?

I, [2012-11-07T19:57:55.133176 #26268]  INFO -- : ########## Starting PDF Job ##########
I, [2012-11-07T19:57:55.133675 #26268]  INFO -- : Parsing arguments: ["1230000000929", 13550, "263632c8128907f9c45f275934894da7f17c800b"]
I, [2012-11-07T19:57:55.133809 #26268]  INFO -- : Updating git repo
I, [2012-11-07T19:57:55.133894 #26268]  INFO -- : Running shell command: cd /usr/local/app/worker/tmp/pdf/1230000000929;git fetch origin --quiet
I, [2012-11-07T19:57:56.645383 #26268]  INFO -- : Running shell command: cd /usr/local/app/worker/tmp/pdf/1230000000929; git reset 263632c8128907f9c45f275934894da7f17c800b --hard --quiet
I, [2012-11-07T19:57:57.093386 #26268]  INFO -- : Building Docbook
I, [2012-11-07T19:57:57.094717 #26268]  INFO -- : Running shell command: xmllint --xinclude -o /usr/local/app/worker/tmp/pdf/1230000000929/book.xml /usr/local/app/worker/tmp/pdf/1230000000929/book.xml
I, [2012-11-07T19:57:57.499840 #26268]  INFO -- : Running shell command: /usr/local/app/worker/renderer/pdfrenderer/csspdf --invalid-ok /usr/local/app/worker/tmp/pdf/1230000000929/book.xml
W, [2012-11-07T19:59:04.865739 #26268]  WARN -- : $/usr/local/app/worker/renderer/pdfrenderer/csspdf --invalid-ok /usr/local/app/worker/tmp/pdf/1230000000929/book.xml[status: pid 26287 exit 0]
Building web PDF

===========Starting Syntax Highlighting==========
===========Completed Syntax Highlighting===========

Converting file /usr/local/app/worker/tmp/pdf/1230000000929/book.xml to HTML with stylesheet /usr/local/app/worker/renderer/pdfrenderer/xsl/series/animal.xsl...
Converting HTML to PDF with CSS stylesheet /usr/local/app/worker/renderer/pdfrenderer/css/series/animal/animal_web.css using AntennaHouse
/usr/AHFormatterV6_64/bin/AHFCmd -i /usr/local/app/worker/renderer/pdfrenderer/ah-settings.xml -css /usr/local/app/worker/renderer/pdfrenderer/css/series/animal/animal_web.css -x 4 -d /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.worker.86.html -o /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.worker.429.pdf
Finished ok
Moving temporary PDF /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.worker.429.pdf to /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.web.pdf

E, [2012-11-07T19:59:04.865923 #26268] ERROR -- : $/usr/local/app/worker/renderer/pdfrenderer/csspdf --invalid-ok /usr/local/app/worker/tmp/pdf/1230000000929/book.xml[status: pid 26287 exit 0]
2012-11-07 19:58:15,938 INFO Input file: /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.worker.852.xml
2012-11-07 19:58:15,938 INFO Output file: /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.worker.852.xml.highlighted.xml
2012-11-07 19:58:16,046 WARNING Unable to successfully validate XML content in /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.worker.852.xml. Attempting to parse anyway.
2012-11-07 19:58:16,059 INFO Pygmentizing 208 code listings with @language attributes, out of 262 total code listings
2012-11-07 19:58:16,579 INFO Outputting highlighted DocBook to /usr/local/app/worker/tmp/pdf/1230000000929/book.xml.worker.852.xml.highlighted.xml
AHFCmd : AH CSS Formatter V6.0 MR2 for Linux64 : 6.0.2.5372 (2012/05/16 18:26JST)
         Copyright (c) 1999-2012 Antenna House, Inc.

AHFCmd :Formatting finished normally :total 165 pages

I, [2012-11-07T19:59:05.876960 #26268]  INFO -- : Running shell command: pdftk /usr/local/app/worker/tmp/pdf/1230000000929/book.pdf dump_data output | grep -i NumberOfPages
W, [2012-11-07T19:59:07.035378 #26268]  WARN -- : $pdftk /usr/local/app/worker/tmp/pdf/1230000000929/book.pdf dump_data output | grep -i NumberOfPages[status: pid 27659 exit 0]
NumberOfPages: 165

I, [2012-11-07T19:59:07.210316 #26268]  INFO -- : Running shell command: cd /usr/local/app/worker/tmp/pdf/1230000000929; git clean -f -d --quiet

[adamwitwer]

Some data in case it's helpful: When I build locally, the newest content is included.

One possible problem (but this is kind of a guess): @simonmar is AsciiDoc savvy and has created an umbrella file (drafts.asciidoc) with includes to other parts of the book. Because Atlas automatically creates a book.asciidoc file with includes of its own, we've got a series of includes that may be tripping something up during the build process.

But I may be totally off base in this guess since I can build PDFs locally against the book.asciidoc file that Atlas creates (and which includes drafts.asciidoc), and the new content is included in the build. We'll keep looking.

[sandersk]

Hi @simonmar,

When you get a moment, could you take a look at the latest PDF available on the dashboard and confirm that it's now in sync with all your latest changes? I think the problem is fixed now, although I'm still a little puzzled at the cause.

@runemadsen and @adamwitwer, I set aside the existing pdf/ checkout of this book on the worker server so that Atlas would do a fresh checkout of the book with the next build, and that solved the problem. As far as I can tell, the issue with the old checkout (now named 1230000000929_temp if you want to take a look) was that the book.xml in the checkout wasn't getting updated in the ASC->DB step. I'm not sure why that was the case, though, as I'm not able to reproduce that particular problem by running the AsciiDoc conversion command directly on the command line on the worker server. Rune, ping me if you'd like more details; happy to troubleshoot with you further.

Thanks, Sanders

[sandersk]

Actually, another thing that's a little weird here is that I believe the book.xml files should be cleared from the checkout once the build is complete. I wonder why that's not happening in this case.

Sanders

[simonmar]

The latest build does seem to be using the up to date sources, thanks.

I think at one point I had a book.asciidoc in the repository, which seemed to confuse Atlas so I removed it. Maybe that triggered the problem?

Also I have book.xml in the .gitignore, I'm not sure whether that's a factor or not.

[runemadsen]

Thanks guys. I think that makes total sense. If the book.xml is in the .gitignore file, then it won't be removed when we reset the repo. @sandersk, could it be that the PDF generation won't override an already existing book.xml file? That would explain everything.

[sandersk]

@runemadsen Good thought, but I'm still a little puzzled because the shell script we use for PDF generation doesn't overwrite book.xml; it copies book.xml to a temp file and then does its work on the temp file. We do use an asciidoc command that converts ASC to DB and writes to book.xml, but I tested this command on the worker server this evening, and it overwrote the existing book.xml just fine.

The only thing I can think of is that when the same asciidoc command is run as part of the larger worker infrastructure, there's some sort of permissions issue that is indeed preventing book.xml from being overwritten during the ASC-to-DB conversion, but that seems unlikely to me.

[simonmar]

The issue is still happening - it's now stuck again at a different revision. Will removing book.xml from .gitignore help?

[runemadsen]

Thanks Simon. Please try to remove it and let me know if it works. Also, you should be able to use a book.asciidoc file. It should be read directly into the app in the book builder. What problems where you seeing?

On Thu, Nov 8, 2012 at 12:20 PM, Simon Marlow notifications@github.comwrote:

The issue is still happening - it's now stuck again at a different revision. Will removing book.xml from .gitignore help?

— Reply to this email directly or view it on GitHubhttps://github.com/MakerPress/atlas-public-feedback/issues/141#issuecomment-10196544.

[simonmar]

I removed book.xml from .gitignore, and it does now seem to be building the latest version.

When I had a book.asciidoc file, I don't recall the details, but something weird happened in the Build tab. I think it wouldn't let me select book.asciidoc as the file to build.

[runemadsen]

Ah, that makes sense. The system is made, so if there's a book.asciidoc file, it'll parse the file and load all the files into the index builder. So you don't need to manually edit the book.asciidoc file, you can just us select the files, and it'll save to book.asciidoc automatically. I'm working on some slight changes that may improve this, though.

[dabeaz]

I'm having a similar problem as this in http://ano-atlas2.herokuapp.com/books/1230000000393. Changes are accepted via git and shown in the "Changes" tab. However, they never appear in any of the PDF builds now.

[runemadsen]

Hi @dabeaz. Can you point me to a paragraph that is in a file, but now showing up in the PDF? Then I can debug it for you.

[dabeaz]

 Problems are focused in chapter_12_utility.asciidoc, specifically, the files 'UtilitySysadmin/simple-logging.asciidoc' and 'UtilitySysadmin/logging_library.asciidoc'.   I can't get the latter file to appear at all.   This renders around pg. 340 of the PDF. Thanks. -Dave

----- Original Message ----- From: MakerPress/atlas-public-feedback To:"MakerPress/atlas-public-feedback" Cc:"David Beazley" Sent:Tue, 20 Nov 2012 13:45:45 -0800 Subject:Re: [atlas-public-feedback] Builds using an old revision (#141)

Hi @dabeaz [1]. Can you point me to a paragraph that is in a file,

but now showing up in the PDF? Then I can debug it for you.

 —

Reply to this email directly or view it on GitHub [2].

Links:

[1] https://github.com/dabeaz [2] https://github.com/MakerPress/atlas-public-feedback/issues/141#issuecomment-10574590

[runemadsen]

Thanks. I think I fixed the problem. Can you confirm?

[dabeaz]

 I just ran another build and I don't see any difference than before.  It's still not using the most recent versions of previously mentioned files. Cheers,Dave

----- Original Message ----- From: MakerPress/atlas-public-feedback To:"MakerPress/atlas-public-feedback" Cc:"David Beazley" Sent:Tue, 20 Nov 2012 15:26:04 -0800 Subject:Re: [atlas-public-feedback] Builds using an old revision (#141)

Thanks. I think I fixed the problem. Can you confirm? 

 —

Reply to this email directly or view it on GitHub [1].

Links:

[1] https://github.com/MakerPress/atlas-public-feedback/issues/141#issuecomment-10578965

[sandersk]

Hi Dave,

Sorry we let this thread drop. Are you still experiencing problems with the builds not incorporating the latest revisions of your book files? If so, we'll take another look.

Thanks, Sanders

[meisenberg-oreilly]

I'm seeing a similar problem with my book: http://ano-atlas2.herokuapp.com/books/1234000000538. I'm able to build an ePub that shows the current version of the content, but the rebuilt PDF reflects an earlier version.

[sandersk]

Hi Michael,

Could you double-check that the old PDF isn't just cached in your Web browser? If you try hitting Refresh in your Web browser, does it give you the updated PDF? If not, we'll take a look and see if we can pinpoint what's going on.

Thanks, Sanders

[meisenberg-oreilly]

Hi Sanders, thanks--a refresh fixed the problem. Maybe worth an FAQ entry?