berteh / ScribusGenerator

Create beautiful documents with data. Open source pdf (and Scribus) template and mail-merge alternative.
http://berteh.github.io/ScribusGenerator/
MIT License
243 stars 37 forks source link

Last pages of pdf not rendered #181

Closed bigbore closed 2 years ago

bigbore commented 3 years ago

I currently use Scribus 1.5.5 and am testing scribus-generator (python 3) on Scribus 1.5.6.1 and 1.5.7. I have a 4 page mail-merge that comes out with the last 4 pages blank in the pdf. I have tried any number of page sets from 2 to 10 and the last 4 pages are always blank in the pdf. I have included the last log entries for 6 sets of 4 pages. It works the same in 1.5.6.1 and 1.5.7.

2021-04-27 12:00:27 - DEBUG - loadFromString:748 - loaded 11 user settings 2021-04-27 12:00:55 - INFO - init:73 - ScribusGenerator initialized 2021-04-27 12:00:55 - DEBUG - init:75 - OS: nt - Python: 3.7.9 - ScribusGenerator v2.9.2 python3 2021-04-27 12:00:55 - DEBUG - run:84 - active options: {"close": 1, "csvencoding": "utf-8", "csvfile": "C:/Users/Win7 Test VM/Documents/Masons/May 2021 address.csv", "from": "1", "keepsla": 0, "outdir": "C:/Users/Win7 Test VM/Documents/Masons", "outformat": "PDF", "outname": "test", "separator": ",", "single": 1, "to": "6"} 2021-04-27 12:00:55 - INFO - run:93 - parsing scribus SLA file C:/Users/Win7 Test VM/Documents/Masons/March Trestleboard.sla 2021-04-27 12:00:55 - DEBUG - run:102 - Scribus SLA template version is 1.5.6.1 2021-04-27 12:00:55 - DEBUG - run:108 - saving current Scribus Generator settings in your source file 2021-04-27 12:00:55 - DEBUG - run:124 - parsing data CSV file C:/Users/Win7 Test VM/Documents/Masons/May 2021 address.csv 2021-04-27 12:00:55 - DEBUG - run:159 - custom data range is: 1 - 6 2021-04-27 12:00:55 - INFO - run:174 - source document consumes 1 data record(s) from 5. 2021-04-27 12:00:55 - INFO - run:190 - variables from data files: ['FullName', 'Address1', 'City', 'State', 'Zip'] 2021-04-27 12:00:55 - DEBUG - run:199 - subsitute, with data entry index being 1 2021-04-27 12:00:56 - DEBUG - substituteData:501 - replacements is: {'%VAR_FullName%': xxxxxxx', '%VAR_Address1%': 'xxxxxx', '%VAR_City%': 'xxxxxx', '%VAR_State%': 'xx', '%VAR_Zip%': xxxxxxx'} 2021-04-27 12:00:56 - DEBUG - run:210 - generating reference content from dataBuffer at #1 2021-04-27 12:00:56 - DEBUG - run:219 - current template has #36 pageobjects 2021-04-27 12:00:56 - DEBUG - run:226 - merging content from dataBuffer up to entry index #1 2021-04-27 12:00:56 - DEBUG - shiftPagesAndObjects:386 - shifted page 0 element of 0.0 2021-04-27 12:00:56 - DEBUG - run:199 - subsitute, with data entry index being 2 2021-04-27 12:00:56 - DEBUG - substituteData:501 - replacements is: {'%VAR_FullName%': 'xxxxxx', '%VAR_Address1%': 'xxxxxxxx', '%VAR_City%': 'xxxx', '%VAR_State%': 'xx', '%VAR_Zip%': xxxxxx'} 2021-04-27 12:00:56 - DEBUG - run:226 - merging content from dataBuffer up to entry index #2 2021-04-27 12:00:56 - DEBUG - shiftPagesAndObjects:386 - shifted page 1 element of 832.0032 2021-04-27 12:00:56 - DEBUG - run:199 - subsitute, with data entry index being 3 2021-04-27 12:00:56 - DEBUG - substituteData:501 - replacements is: {'%VAR_FullName%': 'xxxxx ', '%VAR_Address1%': 'xxxxxx', '%VAR_City%': 'xxxxx', '%VAR_State%': 'xx', '%VAR_Zip%': 'xxxxx'} 2021-04-27 12:00:56 - DEBUG - run:226 - merging content from dataBuffer up to entry index #3 2021-04-27 12:00:56 - DEBUG - shiftPagesAndObjects:386 - shifted page 2 element of 1664.0064 2021-04-27 12:00:56 - DEBUG - run:199 - subsitute, with data entry index being 4 2021-04-27 12:00:56 - DEBUG - substituteData:501 - replacements is: {'%VAR_FullName%': xxxxxxx', '%VAR_Address1%': 'xxxxxx', '%VAR_City%': 'xxxx', '%VAR_State%': 'xx', '%VAR_Zip%': 'xxxxx'} 2021-04-27 12:00:56 - DEBUG - run:226 - merging content from dataBuffer up to entry index #4 2021-04-27 12:00:56 - DEBUG - shiftPagesAndObjects:386 - shifted page 3 element of 2496.0096 2021-04-27 12:00:56 - DEBUG - run:199 - subsitute, with data entry index being 5 2021-04-27 12:00:56 - DEBUG - substituteData:501 - replacements is: {'%VAR_FullName%': 'xxxxxxx', '%VAR_Address1%': 'xxxxxxx', '%VAR_City%': 'xxxxx', '%VAR_State%': 'xx', '%VAR_Zip%': 'xxxxx'} 2021-04-27 12:00:56 - DEBUG - run:226 - merging content from dataBuffer up to entry index #5 2021-04-27 12:00:56 - DEBUG - shiftPagesAndObjects:386 - shifted page 4 element of 3328.0128 2021-04-27 12:00:56 - INFO - removeEmptyTexts:418 - removed 0 empty texts items 2021-04-27 12:00:56 - INFO - writeSLA:303 - scribus file created: C:/Users/Win7 Test VM/Documents/Masons/test.sla 2021-04-27 12:01:13 - INFO - run:253 - pdf file created: C:/Users/Win7 Test VM/Documents/Masons/test.pdf

berteh commented 3 years ago

Hello. Sorry to read your run into some problems. From the logs it seems Scribus Generator is behaving as expected, shifting the generated objects down to the page where they should reside.

Could your attach your SLA template (and sample CSV) here for me to look ? just add ".txt" extension to enable the upload.

bigbore commented 3 years ago

Thanks for your response. I don't think the log reflects what is going on. I am traveling so I won't be able to get back to you till next weekend. 1.5.5 works properly. Bill

⁣Get BlueMail for Android ​

On May 1, 2021, 15:12, at 15:12, Berteh @.***> wrote:

Hello. Sorry to read your run into some problems. From the logs it seems Scribus Generator is behaving as expected, shifting the generated objects down to the page where they should reside.

Could your attach your SLA template (and sample CSV) here for me to look ? just add ".txt" extension to enable the upload.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/berteh/ScribusGenerator/issues/181#issuecomment-830701651

berteh commented 2 years ago

should be fixed in new python3 branch (for scribus 1.5.6+): https://github.com/berteh/ScribusGenerator/archive/python3.zip

could you please test and give me feedback ? thanks !

bigbore commented 2 years ago

I tested this fix in Scribus 1.5.6.1 and found the same problem, except that now it does not render the first address line instead of the last one. Bill Berteh wrote: should be fixed in new python3 branch (for scribus 1.5.6+): https://github.com/berteh/ScribusGenerator/archive/python3.zip https://github.com/berteh/ScribusGenerator/archive/python3.zip could you please test and give me feedback ? thanks ! — Reply to this email directly, view it on GitHub https://github.com/berteh/ScribusGenerator/issues/181#issuecomment-1010451716 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPCY4G6BN4ACSBMRMUTHT3UVS2YDANCNFSM43VZ7M7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread.Message ID: @.***>

berteh commented 2 years ago

Thanks for testing. Could you please attach your sla template to this issue thread? I cannot reproduce the problem so far.

bigbore commented 2 years ago

I tested the new script in 1.5.8 and it works. It still adds a blank set of pages to the end but it renders all the pages in the csv list. I can easily clip off the blank pages. I'm not highly skilled in Scribus so I'm not sure what a template is. I made up an empty test document to run the tests, then ran it on a full-up document and list. It worked fine in both cases except where it rendered a blank set at the end. I will be running more tests on it to make sure it is stable but it looks good. Thanks for looking at this. Bill Berteh wrote: Closed #181 https://github.com/berteh/ScribusGenerator/issues/181 . — Reply to this email directly, view it on GitHub https://github.com/berteh/ScribusGenerator/issues/181#event-5876775054 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPCY4GQIGOXWESJKZAGTU3UVS2YFANCNFSM43VZ7M7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread.Message ID: @.***>

UltraSalem commented 2 years ago

It's been a while since I've used this script, but I think to stop getting the blank pages, you want to take the %SG_NEXT-RECORD% off the final place on the last page of your document if it has one, as the script knows when it's hit the end of the document to start again? berteh can probably comfirm :) but something might be worth testing in the meantime.

berteh commented 2 years ago

Yes I confirm: ScribusGenerator does detect the end of your template document (the SLA with the variables in it) and start a new document (or page in merge mode) while loading the next data entry... so please never put a %SG_NEXT-RECORD% tag at the end of a document.

But I don't think that was the issue with the original template of @bigbore since he seems to not be using the next-record feature from the log.

Just kindly re-open this issue if you still experience this problem and attach your template so I can give it a closer look. thanks !