AO3 epub image issues, fictionalley linebreak problems

[GoogleCodeExporter]

Are you using the CLI, web service or calibre plugin?
CLI

What is the story URL?
http://www.fictionalley.org/authors/metisket/refraction.html
http://archiveofourown.org/works/915078

What is the output format you used?
epub, html

What steps will reproduce the problem?
The usual downloading process.

What is the expected output? What do you see instead?

1. The problem with fictionalley stories I'm encountering: For some reason, 
whether it be in html format or epub format, the downloaded story gets some 
extra linebreaks, where there are none in the actual weblink. Also, 
fictionalley downloads lack the complete/in-progress line. 

2. The above given AO3 link is an example of a story with images in the 
content. They pose no problem when the download format is html, but when I want 
to download as epub, the following error message appears: 

FFDL:INFO:adapter_archiveofourownorg.py(143):url: 
http://archiveofourown.org/works/915078/navigate?view_adult=true
FFDL:INFO:adapter_archiveofourownorg.py(144):metaurl: 
http://archiveofourown.org/works/915078?view_adult=true
FFDL:INFO:base_writer.py(208):Save directly to file: 
/home/archives/fanficdownloader-2.0.04/ff/Between the Mountain and the River 
(Completed).epub
FFDL:INFO:story.py(806):Failed to load or convert image, skipping:
http://fc06.deviantart.net/fs70/f/2013/218/0/b/between_the_mountain_and_the_rive
r___cover_by_bristrek-d6guwro.png
Exception: need more than 0 values to unpack
FFDL:INFO:story.py(806):Failed to load or convert image, skipping:
http://fc03.deviantart.net/fs70/f/2013/218/6/e/an_unexpected_memory_by_bristrek-
d6guy20.png
Exception: need more than 0 values to unpack
FFDL:INFO:story.py(806):Failed to load or convert image, skipping:
http://th06.deviantart.net/fs71/PRE/f/2013/218/d/7/before_the_council_by_bristre
k-d6guxbg.png
Exception: need more than 0 values to unpack
FFDL:INFO:story.py(806):Failed to load or convert image, skipping:
http://fc07.deviantart.net/fs71/f/2013/218/a/9/end_banner_for_between_the_mounta
in_and_the_river_by_bristrek-d6guya2.png
Exception: need more than 0 values to unpack

Also, if I may ask, where do I find out if a new CLI zip file has been 
released? Or is the calibre plugin the only way to find out a new release is 
out?

Thank you for your help. :)

Original issue reported on code.google.com by amelie...@web.de on 30 Jan 2015 at 2:54

[GoogleCodeExporter]

Re: fictionalley stories 

In a cursory look at the Refraction story I don't see any problems.  Please 
point out a few specific spots in the text where you're seeing extra line 
breaks.  

I may not be able to do anything about it--fictionalley tends to have really 
poor story HTML especially in older stories like that one.

fictionalley doesn't offer any flags for complete vs in-progress, so there's no 
way for FFDL to tell.

Re: AO3 images

FFDL uses PIL with the CLI to process images for EPUB.  It's working OK for me 
with Python 2.7.9 and PIL 1.1.7 on Windows 8.  It appears now that PIL is 
getting dated.  What image processing library do you have installed?

Or, you can skip all image processing and use the images as-is by putting 
no_image_processing: true in your personal.ini for [epub].

Re: CLI zip distro

Updates are announced in the FFDL user group:
https://groups.google.com/forum/#!forum/fanfic-downloader

The latest version is always available at:
http://fanficdownloader.googlecode.com/hg/fanficdownloader.zip

Original comment by retiefj...@gmail.com on 30 Jan 2015 at 6:08

[GoogleCodeExporter]

Well, about Refraction. There is no specific spot. It's everywhere. I've 
attached the file (and two other newer ones I chose randomly that came out the 
same) as it downloads for me to demonstrate better what I mean.

As for the images thing. I do have PIL installed (Python 2.7.3 and PIL 1.1.7 on 
ubuntu 12.04).

As I had previously never downloaded any story with image content, I have no 
way of knowing if this problem with the images has always existed or if it was 
a recent development. 

Your suggestion of skipping image processing has proven successful, though, so 
it doesn't matter any more. Thank you. :)

Also thank you for the links. I had not known about the ffdl user group.

However, if I may, I forgot to mention this as well before: Whenever AO3 has a 
news banner, such as the recent one ("Due to an influx of spam works, we are 
temporarily suspending invitations from our automated invite queue. Please read 
more in our AO3 News post, available also in [...]" etc.), the banner text 
replaces the summary of the story in question. Is there a way to avoid this? 

Thank you for your help. :)

Original comment by amelie...@web.de on 1 Feb 2015 at 9:22

Attachments:

• refraction.zip

[GoogleCodeExporter]

Re: Extra lines

I think you may have replace_br_with_p: true.  That's the only way I'm seeing 
extra newlines.  replace_br_with_p is a heuristic process, which means it isn't 
perfect and can sometimes make things worse.  Especially with poor HTML, which 
fictionalley specializes in.  

Re: AO3 banners

Are you certain you're using the latest FFDL?  

Because that sounds like a problem we had a long time ago and was long since 
fixed.  Plus I feel confident that other people would have also reported that 
issue if it were occurring for them.

AO3 is right now showing an orange banner reading: "Site Issues
Since our last code update,..."  It does not appear in the download ebooks for 
me.

Original comment by retiefj...@gmail.com on 2 Feb 2015 at 12:09

[GoogleCodeExporter]

About fictionalley: Thank you.  I've set replace_br_with_p: false for 
fictionalley, and it seems to work. 

The banner problem was with me using version 2.0.04 (which was what I was using 
up until that version 2.1.1 CLI). But would that one have been too old already? 
At least, I saw no mention of such a problem in the download page of the group 
forum link, and so assumed it was safe to ask. But true, other people would 
have reported it too. Guess I thought it could be set as on and off in the ini 
or something.

Anyway. I have now downloaded the latest version, 2.1.03, and there are indeed 
no problems. 

Thank you for your patience and efforts. :)

Original comment by amelie...@web.de on 2 Feb 2015 at 4:25

[GoogleCodeExporter]

Glad we could get your issues sorted out.

Original comment by retiefj...@gmail.com on 2 Feb 2015 at 4:53

• Changed state: Invalid