Drag/Drop into "download from URLs" is not working properly in Firefox 97

[LoisGNS]

Because logs of stuff wasn't working well in my older version of Firefox, I gave in and updated to version 97 (64 bit, on Windows 7). Now I can no longer highlight a page with URLs and drag/drop it into the "download from URLs" dialog. What does happen is that if I open a page in Fanfiction.net, then open the Download from URLs in FFF, nothing appears to happen when I highlight & drag/drop into the dialog.

I have tried both with FanFiction.Net and AO3. The Chrome browser still works as expected for both sites, so there's something about how pages on the new Firefox version are being read by FFF. Can be fixed, either in FFF or by finding and adjusting some obscure (or not-so-obscure) setting in FireFox? I rarely use Chrome, so it's an added step (with Fanfiction.net's cloudflare and capchas there are already more steps than I'd like).

Thanks.

[JimmXinu]

By chance, I also use Firefox and you caught me before I updated it to v97, so I could try both v96 and v97. Mine is 64bit on Win10 with Calibre v5.36 and FFF v4.10.1.

Both Firefox v96 and v97 drag and drop into FFF work the same for me. High light text (ctrl-a), drag onto FFF button on main toolbar or into edit box when 'Download from URLs' dialog is open, list of URLs appears in FFF. Tested both ffnet and AO3 on v97.

So I can't duplicate the issue.

[LoisGNS]

Only differences are I am on Windows 7 and my FFF is 4.10.0, which seems to be the most recent one that Calibre knows about). Oh, I'm using the Portable version of Calibre if that makes any difference. When I drag & try to drop either on the button or on the text box, I get the "no" symbol. I ran with debug and it didn't show anything that seemed very helpful (that I could tell, anyway!). If something is supposed to display to indicate it even saw my drag/drop attempt, it's not displaying.

EpubMerge: DEBUG: 2022-02-15 21:30:01,951: calibre_plugins.epubmerge.epubmerge_plugin(155): macmenuhack file_path:C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\fanficfare_macmenuhack.txt FFF: DEBUG: 2022-02-15 21:30:01,963: calibre_plugins.fanficfare_plugin.fff_plugin(218): Plugin FanFicFare macmenuhack file_path:C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\fanficfare_macmenuhack.txt calibre Debug log calibre 5.7.2 Portable embedded-python: True is64bit: False Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE') 32bit process running on 64bit windows ('Windows', '7', '6.1.7601') Python 3.8.5 Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: DeDRM (7, 0, 0) && Count Pages (1, 11, 2) && EpubMerge (2, 12, 0) && FanFicFare (4, 10, 0) && Find Duplicates (1, 8, 3) && Standard Ebooks (1, 0, 0) calibre 5.7.2 Portable embedded-python: True is64bit: False Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE') 32bit process running on 64bit windows ('Windows', '7', '6.1.7601') Python 3.8.5 Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: DeDRM (7, 0, 0) && Count Pages (1, 11, 2) && EpubMerge (2, 12, 0) && FanFicFare (4, 10, 0) && Find Duplicates (1, 8, 3) && Standard Ebooks (1, 0, 0) Turning on automatic hidpi scaling devicePixelRatio: 1.0 logicalDpi: 96.0 x 96.0 physicalDpi: 72.0354505169867 x 71.99999999999999 Using calibre Qt style: True [0.00] Starting up... [0.02] Initializing db... [0.53] db initialized [0.53] Constructing main UI... Loaded cached store plugin for: Библио.бг at version: 2 Loaded cached store plugin for: Project Gutenberg at version: 8 Loaded cached store plugin for: Kobo at version: 8 Loaded cached store plugin for: Nexto at version: 6 [2.98] main UI initialized... [2.98] Started up in 2.98 seconds with 8815 books Starting QuickView

When I do a similar operation in Chrome, I get the following additional lines (I didn't bother using FFD to load up the cache, so the actual download-failures were expected):

FF: DEBUG: 2022-02-15 21:39:31,002: calibre_plugins.fanficfare_plugin.dialogs(485): self.extraoptions['anthology_url']:NOT FOUND FFF: DEBUG: 2022-02-15 21:39:31,006: calibre_plugins.fanficfare_plugin.fff_plugin(1113): FanFicFare v4.10.0 FFF: DEBUG: 2022-02-15 21:39:31,032: configurable.py(1077): 1configuration.get_browser_cache:None FFF: DEBUG: 2022-02-15 21:39:31,033: configurable.py(1005): use_flaresolverr_proxy:true FFF: WARNING: 2022-02-15 21:39:31,033: configurable.py(1008): FlareSolverr v2+ doesn't work with images: include_images automatically set false FFF: WARNING: 2022-02-15 21:39:31,033: configurable.py(1009): Set use_flaresolverr_proxy:withimages if your are using FlareSolver v1 and want images FFF: DEBUG: 2022-02-15 21:39:31,033: flaresolverr_proxy.py(39): using FlareSolverr_ProxyFetcher FFF: DEBUG: 2022-02-15 21:39:31,033: configurable.py(1033): use_browser_cache:true FFF: DEBUG: 2022-02-15 21:39:31,034: configurable.py(1035): browser_cache_path:C:\Users\Lois\AppData\Local\fanficdownloader\cache\QtWebEngine\Default\Cache FFF: DEBUG: 2022-02-15 21:39:31,164: blockfilecache.py(40): Using BlockfileCache FFF: DEBUG: 2022-02-15 21:39:31,164: configurable.py(1047): use_basic_cache:true FFF: DEBUG: 2022-02-15 21:39:31,165: configurable.py(1081): 2configuration.get_browser_cache:<fanficfare.browsercache.BrowserCache object at 0x16AD4610> FFF: DEBUG: 2022-02-15 21:39:31,165: adapter_fanfictionnet.py(112): URL: https://www.fanfiction.net/s/14012905/1/Gallant-Mr-Darcy FFF: DEBUG: 2022-02-15 21:39:31,166: fetcher.py(234): ========== MISS (GET) BasicCache https://www.fanfiction.net/s/14012905/1/Gallant-Mr-Darcy FFF: DEBUG: 2022-02-15 21:39:31,166: basebrowsercache.py(119): do_map_cache_keys() FFF: DEBUG: 2022-02-15 21:39:33,460: basebrowsercache.py(122): Cached 0 entries FFF: DEBUG: 2022-02-15 21:39:33,460: basebrowsercache.py(55): do_cprofile time:2.293131113052368 FFF: DEBUG: 2022-02-15 21:39:33,460: fetcher.py(273): ========== MISS (GET) BrowserCache https://www.fanfiction.net/s/14012905/1/Gallant-Mr-Darcy FFF: DEBUG: 2022-02-15 21:39:33,461: flaresolverr_proxy.py(97): ---------- REQ (GET) FlareSolverr_ProxyFetcher https://www.fanfiction.net/s/14012905/1/Gallant-Mr-Darcy FFF: DEBUG: 2022-02-15 21:39:33,461: fetcher.py(462): ---------- REQ (POST) RequestsFetcher http://localhost:8191/v1 FFF: ERROR: 2022-02-15 21:39:35,504: calibre_plugins.fanficfare_plugin.dialogs(702): Exception: {'title': 'Unknown', 'author_sort': ['Unknown'], 'author': ['Unknown'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': None, 'begin': None, 'end': None, 'comment': 'Connection to flaresolverr proxy server failed. Is flaresolverr started?', 'url': 'https://www.fanfiction.net/s/14012905/1/Gallant-Mr-Darcy', 'site': 'www.fanfiction.net', 'series': '', 'added': False, 'pubdate': None, 'publisher': None, 'uniqueurl': 'https://www.fanfiction.net/s/14012905/1/Gallant-Mr-Darcy[None-None]', 'listorder': 0, 'collision': 'Update EPUB if New Chapters'}:Connection to flaresolverr proxy server failed. Is flaresolverr started? Traceback (most recent call last): File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\connection.py", line 169, in _new_conn File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\util\connection.py", line 96, in create_connection File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\util\connection.py", line 86, in create_connection ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 699, in urlopen File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 394, in _make_request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\connection.py", line 234, in request File "http\client.py", line 1255, in request File "http\client.py", line 1301, in _send_request File "http\client.py", line 1250, in endheaders File "http\client.py", line 1010, in _send_output File "http\client.py", line 950, in send File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\connection.py", line 200, in connect File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\connection.py", line 181, in _new_conn urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x19948178>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\requests\adapters.py", line 439, in send File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 755, in urlopen File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\urllib3\util\retry.py", line 574, in increment urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8191): Max retries exceeded with url: /v1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x19948178>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\flaresolverr_proxy.py", line 102, in request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\flaresolverr_proxy.py", line 84, in do_fs_request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\fetcher.py", line 464, in request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\requests\sessions.py", line 542, in request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\requests\sessions.py", line 655, in send File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\requests\adapters.py", line 516, in send requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8191): Max retries exceeded with url: /v1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x19948178>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "calibre_plugins.fanficfare_plugin.dialogs", line 689, in do_loop File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1327, in prep_download_loop File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1199, in get_story_metadata_only File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 308, in getStoryMetadataOnly File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\adapters\adapter_fanfictionnet.py", line 114, in doExtractChapterUrlsAndMetadata File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\requestable.py", line 114, in get_request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\requestable.py", line 106, in get_request_redirected File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\fetcher.py", line 391, in get_request_redirected File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\fetcher.py", line 240, in fetcher_do_request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\fetcher.py", line 286, in fetcher_do_request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\fetcher.py", line 136, in fetcher_do_request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\fetcher.py", line 364, in do_request File "C:\Calibre 5\Calibre Portable\Calibre Settings\plugins\FanFicFare.zip\fanficfare\flaresolverr_proxy.py", line 104, in request fanficfare.exceptions.FailedToDownload: Connection to flaresolverr proxy server failed. Is flaresolverr started? get_library_config - start get_library_config - start QueueProgressDialog - self.input_order= ['epub', 'azw3', 'mobi', 'docx', 'txt', 'text', 'lit', 'prc', 'fb2', 'html', 'htm', 'xhtm', 'shtml', 'xhtml', 'zip', 'odt', 'rtf', 'pdf', 'cbc', 'md', 'recipe', 'opf', 'cbr', 'rb', 'pml', 'rar', 'fbz', 'chm', 'docm', 'shtm', 'textile', 'updb', 'azw', 'downloaded_recipe', 'djv', 'pobi', 'pdb', 'htmlz', 'txtz', 'pmlz', 'markdown', 'snb', 'azw4', 'djvu', 'cbz', 'lrf', 'tcr'] For book 'MSNBC [Tue, 15 Feb 2022]', using format epub Worker Launch took: 0.05 seconds Job: 0 Count Page/Word Statistics finished Count Page/Word Statistics

do_count_statistics - book_path=C:\Users\Lois\AppData\Local\Temp\calibre_k5mp_hxn\a696dzrf_count_pages\8973.epub, pages_algorithm=1, page_count_mode=Estimate, statistics_to_run=['PageCount', 'WordCount'], custom_chars_per_page=1500, icu_wordcount=True

do_count_statistics - job started for file book_path=C:\Users\Lois\AppData\Local\Temp\calibre_k5mp_hxn\a696dzrf_count_pages\8973.epub

-------------------------------

Logfile for book ID 8973 (MSNBC [Tue, 15 Feb 2022])

    Method of counting _page_count_mode=Estimate _download_sources=[]

    results= {'PageCount': 400, 'WordCount': 56457}

    Found 400 pages

    Found 56457 words

8973

do_statistics_for_book:  C:\Users\Lois\AppData\Local\Temp\calibre_k5mp_hxn\a696dzrf_count_pages\8973.epub 1 Estimate [] ['PageCount', 'WordCount'] 1500 True

    Page count: 400

    Word count using icu_wordcount - trying to count_words

    Word count - used count_words: 56457

    Word count: 56457

About to refresh GUI - book_ids_to_update= [8973, 8973]

[LoisGNS]

Update: The problem is definitely on the Firefox end: I can't drag/drop to anything. I found some info online, and the only suggestion that finally worked was to run Calibre ( and anything else into which I might want to drag/drop) as administrator. It's apparently some security "fix" for which none of the other suggested workarounds was useful. Among the fixes suggested was to use copy/paste instead, but that doesn't work with fff - have to drag & drop for it to parse correctly. Copy/paste did work, in that the text went into the box, but the URLs weren't parsed out.

So I guess you can close the issue (but consider a suggestion to allow using copy/paste in addition to drag/drop in the download from urls box)?

[JimmXinu]

The problem with changing paste behavior is that, in past, it has had unintended consequences when trying to do normal editing.

For example, FFF used to have some custom code for appending pasted text that made it very difficult to cut/copy/paste parts of URLs because it always added newlines to anything pasted in. Removing that made it behave more normally, but now users have to hit enter between individually pasted story URLs.

The code used with drag/drop discards anything that isn't a recognizable story URL. If applied to pastes, it might prevent a user from being able to construct a story URL by pasting parts at a time. IE, site URL, then story ID pasted separately.

I will think about it some more, but no promises anything will come of it.

[JimmXinu]

It is possible to put an insertFromMimeData() function into the DroppableQTextEdit class in calibre-plugin/dialogs.py and parse story URLs from pasted HTML like how drag & drop does.

However, I still fear there would be unintended side effects. For example, copying a piece of what appears to be plain text from the story URLs edit box is pasted back into the same box as a text/html mime "rich text" fragment with HTML and CSS around it. Calling .setAcceptRichText(False) on the DroppableQTextEdit, surprisingly, doesn't appear to change that. Plus, my experience is that the three different OS flavors often have differences in GUI behavior as well.

However, I am willing to put it into the test version for at least a bit since I'll be doing some more experimental stuff anyway. There should be an automatic notice here when I push that commit.

[JimmXinu]

Merged in main.