storiesonline.net download issue

[ch4os4ever]

Tried to download the following story (ulr=https://storiesonline.net/s/15981/wolves-and-dragons-of-the-blood-revelations) and got this error:

Status Title Author Comment URL Error Unknown Unknown 'NoneType' object is not subscriptable https://storiesonline.net/s/15981

[ch4os4ever]

tried again with those 5 stories to same issue:

Status Title Author Comment URL Error Unknown Unknown 'NoneType' object is not subscriptable https://storiesonline.net/s/74428 Error Unknown Unknown 'NoneType' object is not subscriptable https://storiesonline.net/s/76523 Error Unknown Unknown 'NoneType' object is not subscriptable https://storiesonline.net/s/76565 Error Unknown Unknown 'NoneType' object is not subscriptable https://storiesonline.net/s/15981 Error Unknown Unknown 'NoneType' object is not subscriptable https://storiesonline.net/s/11243

ulrs: https://storiesonline.net/s/74428/time-of-eden-and-elves https://storiesonline.net/s/76523/wolves-and-dragons-of-the-blood-book-2-a-spartans-war https://storiesonline.net/s/76565/wolves-and-dragons-of-the-blood-book-3-discovery https://storiesonline.net/s/15981/wolves-and-dragons-of-the-blood-revelations https://storiesonline.net/s/11243/wolves-and-dragons-of-the-blood-true-beginnings-fantasy-sex-story

[JimmXinu]

I'm not seeing that issue.

Are you using the current version (or latest test version)?

Have you changed your theme/skin? FFF only works with Classic; although issues with that usually express differently..

[ch4os4ever]

I am using latest versions both on calibre and on the plugin, by classic you mean the theme like this pic then yes, its in classic, just tried again with this link:https://storiesonline.net/s/74428

and got same error:

Status Title Author Comment URL Error Unknown Unknown 'NoneType' object is not subscriptable https://storiesonline.net/s/74428

Not sure what to do next, thanks in advance,

[ch4os4ever]

here versions of calibre and plugin

[kido5217]

I'm getting the same error.

Traceback (most recent call last):
  File "/usr/bin/fanficfare", line 33, in <module>
    sys.exit(load_entry_point('FanFicFare==4.4.5', 'console_scripts', 'fanficfare')())
  File "/usr/lib/python3.9/site-packages/fanficfare/cli.py", line 310, in main
    do_download(url,
  File "/usr/lib/python3.9/site-packages/fanficfare/cli.py", line 410, in do_download
    adapter.getStoryMetadataOnly()
  File "/usr/lib/python3.9/site-packages/fanficfare/adapters/base_adapter.py", line 308, in getStoryMetadataOnly
    self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
  File "/usr/lib/python3.9/site-packages/fanficfare/adapters/adapter_storiesonlinenet.py", line 194, in doExtractChapterUrlsAndMetadata
    alist = authfrom.findAll('a', {'rel' : 'author'})
AttributeError: 'NoneType' object has no attribute 'findAll'

[JimmXinu]

I'm not able to duplicate that problem with any of the story URLs given.

The storiesonline.net sites track how many stories you view (or download) each day by user. Are you maybe bumping into a view limit?

Also, FFF cannot login if you are configured to use two factor authentication.

[ch4os4ever]

I am not using the 2 factor authentication, and I created a brand new account just to see if it was a problem with the configuration of the old one, yet problem persists. Unsure what to try next. Sorry I could not be of more help into reproducing error, but with me happens every time.

[ch4os4ever]

Do browser caches affect the fff? Since it uses calibre I thought this would not be the case and only account configurations matter, I am using a brand new account still on classic mode, no preferences edited and 2 step verification not enabled.

[JimmXinu]

My understanding is that Classic is not the default mode for new accounts.

Attached is a version of the plugin that includes additional debug output showing exactly what the page served contains.

Please install it in your Calibre, restart Calibre in debug mode, try downloading again, and post the resulting debug log here.

FanFicFare.zip

[ch4os4ever]

will try, thanks

[ch4os4ever]

Well, here it is, same bug occurred tmpy1hakjnw.txt , so hope this log will help out. Thanks in advance.

[ch4os4ever]

Tried again, same issue, here is log, hope it helps. tmpqchmp1cm.txt

[ch4os4ever]

tmpo43fhdca.txt

[Edocsil]

@import url('/res/css/stories2.css');
@import url('/res/css/stories-auto-dark-mode.css');

My guess is that the website is broken and it's not saving the settings properly. It's happened before.

[ch4os4ever]

How do I fix that? I cleared my browser cache and open anonymous browsing, still I notice the dark mode sticks on the story viewing mode. How do I get the default/classic mode to stick? Tried changing back and saving but it persists. I created a brand new account and got same bug. Not sure what else I can do

[JimmXinu]

I have the same import lines, so it's not that. And browser cache is completely unrelated for this site.

storiesonline.net has indeed been buggy before in such a way that didn't allow you to change themes. However, it's working for me now. Keep in mind, you have to click the 'Save Listings Settings' button after changing the theme--it doesn't save automatically. Plus, when you have a different theme, you should get a different error from FFF that mentions setting Theme to Classic.

The logs attached are indicating that @ch4os4ever's error is happening during log in--a different error than @kido5217.

Attached is another test version with even more debug output, this time during login. Please try and post debug log again. FanFicFare.zip

[ch4os4ever]

tmpfwmwc1cu.txt

done, i see that dark mode remains, i tried changing and saving on the site but it always bounces back, even with the cache cleared.

[ch4os4ever]

Id like to say thanks in advance, fff is the only download tool I know of for stories-online and since i spend most of the weekdays without internet (Work middle of nowhere) I like to downloading full stories that interest me from many sites to read over the smartphone. Really unfortunate this problem has been occurring.

[ch4os4ever]

I was poking around the site bug report forum and found this, but I have no idea how to change the windows options they mention, anyone could post a tutorial or instructions on that, I am rather unskilled in programing/advanced computer stuff.

https://storiesonline.net/d/s6/t8704/dark-mode-will-not-reset

[ch4os4ever]

tmpoxt9seuo.txt

I have changed my windows color setting back to default in hopes this was what the forum bug thread was about. Here is the log after that (Same problem occurred, but maybe not this dark mode)

[ch4os4ever]

tmp3xw_9aji.txt

[JimmXinu]

There's something really weird going on here. The HTML element that FFF is failing to find when you run it is clearly there. I copied the HTML from your debug output directly into the code and it parsed fine for me.

That suggests there's something going wrong with your setup, rather than the site or FFF's code.

Differences I can see:

• You're using Calibre 32bit while I use 64bit. But I just tested it in 32bit (portable), worked fine, as it should.
• You have an outdated DeDRM plugin that's failing to load--you might try to remove it. (Failed to initialize plugin: 'C:\\Users\\edber\\AppData\\Roaming\\calibre\\plugins\\DeDRM.zip')
• You have Obok DeDRM plugin installed that I don't (I have several others you don't). I don't know if that would make a difference, but I'd try removing it.

Other than those, at this point I would also try rebooting the computer and re-installing Calibre, too.

[ch4os4ever]

tmpcibve3qo.txt

removed the drm plugin and problem persists, will try to reinstall calibre on 64 now

[ch4os4ever]

tmpxri0n_e2.txt

So uninstalled calibre, rebooted pc, installed 64v, rebooted again for good measure, put the latest fff you sent above, and tried again with same story (https://storiesonline.net/s/74428/time-of-eden-and-elves) and the issue still occurred. Included log file but I am losing hope that is something I can fix. Still thanks for looking into it and hope this time something comes up.

[JimmXinu]

It's failing to find an element that is clearly there. I can run the same code on the same HTML and it works.

I'll continue to think about it, but for now I'm stumped.

[ch4os4ever]

after some python studying and trial and error I installed all via cmd and tried once more, it seems same error ocurred for the result on the cmd console was this

Username: Traceback (most recent call last): File "C:\Users\edber\PycharmProjects\untitled\venv\Scripts\fanficfare-script.py", line 33, in sys.exit(load_entry_point('FanFicFare==4.4.1', 'console_scripts', 'fanficfare')()) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\cli.py", line 315, in main fail) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\cli.py", line 410, in do_download adapter.getStoryMetadataOnly() File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\adapters\base_adapter.py", line 308, in getStoryMetadataOnly self.doExtractChapterUrlsAndMetadata(get_cover=get_cover) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\adapters\adapter_storiesonlinenet.py", line 171, in doExtractChapterUrlsAndMetadata self.performLogin(url) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\adapters\adapter_storiesonlinenet.py", line 141, in performLogin params['v']=soup.find('input', {'name':'v'})['value'] TypeError: 'NoneType' object is not subscriptable C:\Users\edber\FanficFareFreeport>

not sure this will be of help but posting anyway

[JimmXinu]

If it weren't for the fact that the debug logs earlier showed the HTML and showed that <input name='v'...> was there (and can be parsed by bs4/html5lib when I try it), I would say it was something wrong with your storiesonline account or maybe a man-in-the-middle firewall breaking things.

If you want to try it, I've attached a adapter_storiesonlinenet.py file with even more debug output.
adapter_storiesonlinenet.py.txt

Put it in place of c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\adapters\adapter_storiesonlinenet.py (after renaming to remove the .txt--github wouldn't accept a .py file.) At this point I have no expectation it will do anything but further confirm that it doesn't work on your computer.

In fact, if you have another Windows computer available, even temporarily, I would suggest trying that--you can install Calibre Portable on a USB drive and install plugins in it.

[ch4os4ever]

tried and got this message C:\WINDOWS\system32>fanficfare --download-list=https://storiesonline.net/s/74428/time-of-eden-and-elves DEPRECATION: The OpenSSL being used by this python install (OpenSSL 1.0.2k 26 Jan 2017) does not meet the minimum supported version (>= OpenSSL 1.1.1) in order to support TLS 1.3 required by Cloudflare, You may encounter an unexpected Captcha or cloudflare 1020 blocks. Login Failed, Need Username/Password. edwie13 Password: Username: Traceback (most recent call last): File "C:\Users\edber\PycharmProjects\untitled\venv\Scripts\fanficfare-script.py", line 33, in sys.exit(load_entry_point('FanFicFare==4.4.1', 'console_scripts', 'fanficfare')()) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\cli.py", line 315, in main fail) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\cli.py", line 410, in do_download adapter.getStoryMetadataOnly() File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\adapters\base_adapter.py", line 308, in getStoryMetadataOnly self.doExtractChapterUrlsAndMetadata(get_cover=get_cover) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\adapters\adapter_storiesonlinenet.py", line 177, in doExtractChapterUrlsAndMetadata self.performLogin(url) File "c:\users\edber\pycharmprojects\untitled\venv\lib\site-packages\fanficfare\adapters\adapter_storiesonlinenet.py", line 147, in performLogin params['v']=soup.find('input', {'name':'v'})['value'] TypeError: 'NoneType' object is not subscriptable C:\WINDOWS\system32>

[JimmXinu]

Use -d option for the debug output. Don't use --download-list option--that's for downloading from a page containing a list of story URLs. You can pipe all the output to a file like so:

fanficfare -d https://storiesonline.net/s/74428/time-of-eden-and-elves > output.txt 2>&1

[ch4os4ever]

Im not really a high level programmer (Work on automation and embedded electronics) so python and windows shell are out of my comfort zone, anyway here is the result output.txt

[JimmXinu]

These 'wlpc' sites use a weird login system that requires a GET and then 2 consecutive POSTs, each with a hidden form variable from the previous req.

In your case, the first POST is returning an empty string instead of the expected page with form. (Previously, we were looking at output from the GET only). That's the point of failure.

If you want, I could give you a version to test that just goes on after the POST and assumes the login succeeded, but I don't expect it would work.

If it weren't for the fact that you saw the same error in Calibre, I would suggest updating your Python version--yours is 4 years old.

The very last grasping-at-straws idea that comes to mind is to set your user/pass in personal.ini instead of entering it manually. I don't really expect that to help either--manual entry works for me.

At this point (assuming my previous suggestion of trying a different computer isn't feasible) I would suggest you try a different download tool. The WebToEpub browser plugin for Chrome or Firefox appears to support storiesonline.net and just downloaded that story for me. If runs in browser, so it doesn't deal with login.

[ch4os4ever]

I was not aware of WebToEpub existence but tried it it and it worked. Seems this will be the solution for now on storiesonline, still your fff is my goto downloader on spacebattles and sufficientvelocity as well as a few other sites and worked with no flaws until I tried storiesonline. If I get the chance to use another PC I will try again and report to you the result. Thanks for the help. Your call to close this or leave open until the actual problem is found. Best regards.