Help - shit - I've moved html files

[talestomaz]

Hi,

When I used ZotFile, I had a procedure that, from time to time, I would select my whole Zotero library and ask to rename/move it. This would make eventual single PDF files stored in the Zotero storage to be moved to my custom folder. I would do this because ZotFile was not always reliable in doing that automatically when I saved a file via the browser addon (Firefox). In this procedure, ZotFile wouldn't touch the HTML files (webpages) though, only the PDFs.

It turns out that I've done this more or less automatically yesterday with ZotMoov / Zotero 7, and the HTML files were also moved to my custom folder. If at least they were linked - as it happens with the PDF ones -, I could have even enjoyed this procedure. But the problem is that they are not. Zotero still wants to find the HTMLs in the original folder. If I manually move every single HTML back to their original folder, it works just fine. But we are talking about 659 files... Is there any way to reverse the move?

Thanks!

[wileyyugioh]

Can you provide more information on how you moved the files automatically through ZotMoov? If you did it by selecting all and using the menu item it should have also moved the html files. Can you provide some screenshots?

[talestomaz]

Sure. I selected all items in my main panel, right-clicked and chose "ZotMoov: Move Selected to Director". Screenshot:

What happened: HTML were moved to my PDF folder, but no proper linking on Zotero. Screenshot of the folder:

Without proper linking I'm dead. Either the HTML files should not be moved at all (as I was used to with ZotFile) or they should be moved and be transformed into linked file (I don't know if this is even possible with HTML). I could manually drag back all HTML files to their original storage folder, this works, I have already tested with a few, but these are almost 700 files...

[TW96]

Shouldn't Zotmoov just honour its list of Allowed File Extensions for this operation ? If HTML is not in that list, it should not be moved. But if HTML is in the list, Zotmoov should move it and update Zotero's data for the HTML file's (linked) location.

If I recall correctly, I think Zotfile just ignored HTML files, even if they were in its list of allowed files. I suspect that was because it was written before Zotero changed to storing webpage snapshots in a single HTML file (they used to be a folder full of the many files that make up a web page, which made linked snapshots impractical).

It is definitely useful to have the ability to move HTML files as linked files (Zotero definitely allows HTML files to be linked files) - for example, if you want literally every content file type that Zotero would normally store locally to instead be a linked file in your cloud-mirrored folder. That would be useful if for example you want to leave Zotero File syncing turned on, for example for a small number of locally stored PDFs that you want to access via the mobile app. If you have a lot of (non-linked) HTML webpage snapshots, they can take up a lot of space under Zotero\storage - and if they exceed your online file storage quota then you can't use file syncing for those few PDFs.

[talestomaz]

I'd expect it to respect that list too, but there I have only PDF, EPUB and DOC. Even so HTML was moved...

Correct, Zotfile ignored HTML files.

It is definitely useful to have the ability to move HTML files as linked files (Zotero definitely allows HTML files to be linked files) - for example, if you want literally every content file type that Zotero would normally store locally to instead be a linked file in your cloud-mirrored folder.

This would indeed be perfect for my own setting. I never counted on this because I didn't think it's possible. Now it seems. But first I need to recover my HTML files which were moved and NOT linked...

[wileyyugioh]

The Allowed File Extensions was only for the automatically moved files, not the right-click menu. I'm gonna update the addon in the future so it will also apply to the right-click menu.

Make a backup of your Zotero directory and open up the developer console. Select all of the files and run

as = Zotero.ZotMoov._getSelectedItems()
as = Array.from(as)
as = as.filter((a) => {
    return a.getFilename().split('.').pop() == 'html'
})
as = as.map((a) => {
    return [a.getFilePath(), Zotero.ZotMoov._getCopyPath(a, Zotero.Prefs.get('extensions.zotmoov.dst_dir', true), Zotero.Prefs.get('extensions.zotmoov.enable_subdir_move', true), Zotero.Prefs.get('extensions.zotmoov.subdirectory_string', true))]
})

Open up the array and make sure that each pair of paths look right. Then run

for (let a of as)
{
    IOUtils.copy(a[1], a[0], { noOverwrite: true })
}

This will hopefully copy the files from the ZotMoov directory back into Zotero storage. After you verify this fixes the problem, you can delete them.

[talestomaz]

Sorry for my ignorance, but how do I open this developer console? I've done a quick search, but didn't find a conclusive answer...

[wileyyugioh]

Tools > Developer > Error Console

[TW96]

Or do you mean Tools\Developer\Run Javascript ?

[talestomaz]

Tools > Developer > Error Console

Wow, it has worked like a charm, thanks!

Since through this painful episode I have also learned that I could actually change my procedure to also have my HTMLs in the same folder, do you have any recommendation on how to proceed from now on? I'd need all snapshots to be renamed in the standard of the PDFs (in my case, Author - Year - Title) and moved to the custom storage folder.

I'd need this both to newly added files and already existing ones.

For moving the existing ones, apparently the Zotmoov move command could be enough. I've tested now in smaller batches and it's working (keeping the links). Should there be any problem if I try again with the whole HTML library? Then final question is what to do with the renaming. So far I had to select one by the snapshots (attachments), right-click and ask to rename. If I select the parent items (necessary for a bulk renaming), I don't have to option to rename. Any suggestion here?

[wileyyugioh]

For renaming the webpages, you can do advanced search attachment file type > is > webpage and save the search. Then when you do ctrl+a it should select all of the webpages and you can just click the rename from metadata to rename all of the websites.

Then you can move your items as usual. To be honest, I'm not sure why it failed the last time and only moved the pdfs, so maybe make a Zotero backup before you move all of your items. I added more safeguards to prevent the existing Zotero files from breaking if something goes wrong since the beginning of this thread, but better to take the extra five minutes to be safe.

[TW96]

If you have a lot of HTML web snapshots, they can take up a lot of space. So make sure you have room for them in your linked attachments folder. The only other issue with moving snapshots would be if you have many from before Zotero changed to saving them to a single HTML file in 2020 (v 5.0.93). Prior to that, they were instead saved to a folder with all the many small associated files for a web page. I am not sure what the effect of trying to move those would be (there may have been one HTML file stored which then accessed the stored folder ?). When Zotero made that change, I didn't have that many snapshots. So I was able to go back to the original web pages and save them again, under the new single-HTML file method (after deleting the old snapshot). https://www.zotero.org/support/5.0_changelog#changes_in_5093_november_4_2020