One-click migration to V11 Core Compendium Folders

[anthonyronda]

From Foundry's V11p2 update notes

Get Organized with Compendium Folders Compendium Packs can now contain Folders! Often requested by our users and publishing partners, this long-awaited feature makes it possible to import and export Folders between your World and a Compendium while keeping your Documents organized. You can search, collapse, and expand Compendium Folders just like in the sidebar, and interact with the Documents contained within them in the same way. As an added bonus, we've also increased the maximum number of folders you can have nested in the sidebar from 3 to 4.

https://foundryvtt.com/releases/11.293

Is it in scope for the project to provide a macro for converting Compendium Folders module folders to V11+ Core compendium folders?

[earlSt1]

Hi, yep i've got a migration script working for v11 folders. I'll just need to figure out a way to let the user run it (since compendiums can be locked) and have some backup in place just in case. It will probably be located in the module settings somewhere. When I ran the script on some old compendiums of mine it worked great though :smile:

[anthonyronda]

I'm thrilled :)

In the module settings sounds fine, really anywhere is good 👍

[earlSt1]

I'm thrilled :)

In the module settings sounds fine, really anywhere is good 👍

I've been working on the v11 branch and have the migration script ready to use as an option in the module settings. If you'd like to use it to get ready for v11 you can use this manifest: https://raw.githubusercontent.com/earlSt1/vtt-compendium-folders/11x-update/test/module.json

You can also try out the sidebar migration, which should happen when you first launch the world and once run will convert the sidebar folders to core folders

[anthonyronda]

Thanks Craig! I'm catching up on the stream from Friday and I thought I'd drop this here since it's directly relating to this issue

https://www.twitch.tv/videos/1793705154?t=0h19m25s

They unveiled further work on the compendium data model with V11dev2

https://foundryvtt.com/releases/11.295

I'll let you know how this fares in dev1 and dev 2

[earlSt1]

Haha dont worry, I have some insider knowledge so was expecting this update, which is why i was waiting 😉 I've been in touch with Atro about the migration and fortunately I did some work about a year ago which has made the migration path a lot simpler than it would have been. The above manifest is compatible with the latest dev release (v11 dev2)

[anthonyronda]

Heads up, getting errors on half of my compendiums when running the first migration step on them

TypeError: can't access property "contents", cfFolders is null
    migrateToTarget http://localhost:30000/modules/compendium-folders/migrateFIC.js:10
    migrate http://localhost:30000/modules/compendium-folders/migrateFIC.js:5
    migrateCompendium http://localhost:30000/modules/compendium-folders/fic-folders.js:287
    _updateObject http://localhost:30000/modules/compendium-folders/compendium-folders.js:461
    _onSubmit http://localhost:30000/scripts/foundry.js:6507
    _activateCoreListeners http://localhost:30000/scripts/foundry.js:6441
    _render http://localhost:30000/scripts/foundry.js:5708
    _render http://localhost:30000/scripts/foundry.js:6413
    render http://localhost:30000/scripts/foundry.js:5637
    _onClickSubmenu http://localhost:30000/scripts/foundry.js:82689
    jQuery 2
[foundry.js:6510:15](http://localhost:30000/scripts/foundry.js)
    _onSubmit http://localhost:30000/scripts/foundry.js:6510
    _activateCoreListeners http://localhost:30000/scripts/foundry.js:6441
    _render http://localhost:30000/scripts/foundry.js:5708
    _render http://localhost:30000/scripts/foundry.js:6413
    render http://localhost:30000/scripts/foundry.js:5637
    _onClickSubmenu http://localhost:30000/scripts/foundry.js:82689
    jQuery 2

The migration never actually starts.

But then I tried stepping through the code at migrateToTarget to diagnose, and it suddenly worked for those same compendiums it had failed on.

Something may need to be awaited that currently isn't?

[anthonyronda]

Additional issue: When trying to delete the backup folder, for all compendiums of all document types, I get a different error

Uncaught (in promise) TypeError: db._handleDeleteDocuments is not a function
[No packages detected]
    _onDelete http://localhost:30000/scripts/foundry.js:21418
    callback http://localhost:30000/scripts/foundry.js:13629
    documents http://localhost:30000/scripts/foundry.js:13605
    #handleDeleteDocuments http://localhost:30000/scripts/foundry.js:13605

This what I get when trying to delete an individual #[CF_tempEntity]

Uncaught (in promise) Error: Cannot read properties of undefined (reading 'invalid')
[No packages detected]

For this error, I can offer that the system is using Foundry's V10+ system document validation, and I've had trouble with updating/deleting invalid documents before. But this is still happening in Table compendiums and Journal Compendiums

[earlSt1]

Thanks for the reports! So those last 2 issues seem to be a v11p2 problem. I'm getting the same thing with CF disabled, but if you refresh the page they register as being deleted. The first one should work though. I've just scanned through theh code that I have and cant find any missing awaits. The only situation i can see that error happening is if the compendium doesnt have folders in it. Does the compendium have a large amount of folders in it?

[anthonyronda]

It was somewhere between 100-200 folders I believe for each of those. I did get the sense that only the large ones had issues

[anthonyronda]

You're correct, they're indeed gone on refresh. Thanks :)

[earlSt1]

It was somewhere between 100-200 folders I believe for each of those. I did get the sense that only the large ones had issues

Would you be able to send the compendiums to me so I can have a look? Discord recently increased the filesize of attachments if that makes it easier (my discord is Erceron#0370)

[anthonyronda]

I should be able to get that to you soon.

Journals were one of the compendium types I had to do this procedure "Debugger step-through" procedure, and I just noticed that all of them were blanked (there are backups! 😄). The JEs were not touched much in V10, so they each had 1 page. The CF Migration re-creates them without any pages.

[anthonyronda]

I'm happy to close this. The request has been fulfilled and all issues discussed above are resolved by running migrations in Chrome. Thanks again