search

moo-man / FVTT-DD-Import

Allows Importing Dungeondraft map files into FoundryVTT
MIT License
59 stars 27 forks source link

Fails to import without an error #119

Closed tLJack closed 1 year ago

tLJack commented 2 years ago

On Foundry 9 build 269 when importing a file from Dungeondraft the importer fails.

Console: No error

Settings: Settings

arakan94 commented 2 years ago

Happens to me too :(

tLJack commented 2 years ago

Looks like the load event listener in image2Canvas never gets called.

arakan94 commented 2 years ago

Interestingly enough, it only happens on large files.. Later, I imported many small files (around 5-10 MB) without any issue.

tLJack commented 2 years ago

Thanks! Worked for me as well when reducing the file size.

bfabe8 commented 2 years ago

Looking into this a little myself, it seems like the loading of the image errors out - if you attach an error event listener to the image, it's called. I'm unsure why, but my guess is that the base64 image string is too long for the browser or something similar.

This may be a browser specific issue - .dd2vtt files that do not work in Firefox work in Chrome.

A fix seems to be to convert the b64 image data to a Blob and load it into the image that way - I've gotten it to work in my testing through that method

hjk321 commented 1 year ago

Nearly two months in and no response from dev? Module's dead folks. Anyway, confirmed on Foundry v10 as well, a 30MB dd2vtt fails to upload but a 10MB works ok

arakan94 commented 1 year ago

Nearly two months in and no response from dev? Module's dead folks.

Author's probably just busy - there was update for V10 two weeks ago.. And if all fails, it's public repository - anyone can just pick it up and continue..

arakan94 commented 1 year ago

A fix seems to be to convert the b64 image data to a Blob and load it into the image that way - I've gotten it to work in my testing through that method

Hmm, sounds good - can you upload it as merge-request?

bfabe8 commented 1 year ago

A fix seems to be to convert the b64 image data to a Blob and load it into the image that way - I've gotten it to work in my testing through that method

Hmm, sounds good - can you upload it as merge-request?

I'll look into it when I have time.

moo-man commented 1 year ago

Author's probably just busy

This would be correct as I have many, many other systems and modules to maintain. Issues with this importer are hard to address as they are difficult to reproduce with the incredibly wide range of different server configurations and internet situations users have.

Regardless, sorry for the lack of communication, @bfabe8 I would greatly appreciate a PR to fix this issue, thanks!

amalgamemnon commented 1 year ago

Since this issue has been open since late July and it's now early November, are we going to be seeing a fix on this anytime soon? Having an arbitrary ~10MB limit on the universal files seems pretty silly, no?

edit: I'm noticing issues being called in my console when installing the module, as well. Not sure if this could be part of/causing the inability to bring in files of 10MB or larger?

image

moo-man commented 1 year ago

Those warnings are just manifest compatibility deprecations, nothing to do with your issue.

Yes it would be silly if there was such a limit, but there isn't anything that the importer is doing to impose one as far as I'm aware. I can use ~150 MB .dd2vtt files on a remote server with success. I simply don't know the root cause and suspect, as I mentioned above, that it is due to some server or network configuration.

amalgamemnon commented 1 year ago

many people are reporting that it's some issue with v10, as this issue wasn't present prior to that... I'll try and open an issue with the FoundryVTT devs and see if they're aware of anything that could cause this.

moo-man commented 1 year ago

The OP of the issue mentions V9.

FoundryVTT itself doesn't handle uploading or put a cap on data. What is your hosting situation?

bfabe8 commented 1 year ago

I'll see about getting my fix up for a pull request today, sorry about the delay - I honestly forgot about this. I believe this is a browser specific issue for Firefox where Firefox caps the amount of data when importing an image as a base64 string, so the fix was to convert the image to a blob first and then import that.

moo-man commented 1 year ago

Okay so I can reproduce this issue on Firefox as well, I will await your PR or try it myself in the coming days if I can get around to it.

In the meantime, use Chrome or similar browser to upload your uvtt files

amalgamemnon commented 1 year ago

yep, I'm using firefox. I'm assuming since the electron app is a chromium-browser and most folks are still using chrome, that's why this issue was difficult to reproduce/narrow

is there any chance this can be fixed by changing a setting in firefox?

in the meantime, i'm just using Edge to do the import stuff and then switching back to firefox for everything else.

emmoth commented 1 year ago

This also happens to me in v10 on Firefox trying to import a 87MB large city (only 70ppi). I really hoped to use 150ppi but the file ended up over 300MB and that didn't work.

I was able to import the 87MB file using Chrome instead, but the 300MB file didn't work at all, even tho my server is on the same LAN as I am.

m42e commented 1 year ago

Hi, the reason for that is, that there is no callback from the browser is the conversion is finished. It is not a real conversion at all, more like a hack. I am sorry but at least I do not know a better way of doing it.

Have you tried disabling the webp conversion?

emmoth commented 1 year ago

Hi, the reason for that is, that there is no callback from the browser is the conversion is finished. It is not a real conversion at all, more like a hack. I am sorry but at least I do not know a better way of doing it.

Have you tried disabling the webp conversion?

Hey, no I haven't tried that yet, will try it during the holidays. Thanks for the suggestion.

4cer commented 1 year ago

Have you tried disabling the webp conversion?

Does not help. One way to go about it is to use the dd2vtt only to get the walls/lights/etc into FVTT, then import the background image separately. I have outlined 2 workarounds in another issue (tackling the same problem): https://github.com/moo-man/FVTT-DD-Import/issues/123#issuecomment-1382590214

zenlor commented 1 year ago

I have a patch for working uploads, I did test it with pretty big maps (40Mb+ files) here:

https://github.com/litteram/FVTT-DD-Import/tree/big-files

I should prepare a PR. The branch is very messy right now, I did refactor the code to understand where it was failing. I'll make a PR when ready

edit: @moo-man if you are interested, looking at the changes I made months ago looks like the atob call fails, so I implemented a base64 to typed array function (taken from mozilla's mdn, these should be under CC-0, public domain licensing)

ChrisG0x20 commented 1 year ago

This fails in Firefox 109 using dd-import 2.4.0. Worked in Edge.

BoardGames482 commented 1 year ago

I was having this issue with Foundry itself, but after updating to v2.5.1 it seems to have resolved the large file issue and can convert to WeBP; wonderful! 😄

Edit It's slightly more complicated, but works in the end.

If the image is larger than 16383 it can't be converted to WebP

It seems it did make a new image file with a .webp extension, but it is actually a PNG file?!

I had four .dd2vtt files exported (from DungeonDraft) with a Grid PPI of 256 (each image was 8192x8192 pixels) and then attempted to lay them out in a grid. The resulting image was 16384x16384, which is apparently too large to fit into a WebP file (Based on this resource which says the maximum dimensions are 16383x16383) ?

When opening the .webp file in IrfanView it claims

the file is a PNG file with incorrect extension

Created image is a PNG with a .webp extension

When using .dd2vtt files exported with a Grid PPI of 128 I again observed the same behaviour of the final image being a PNG with a .webp extension. However, this time the image is easily converted to an actual WebP file using IrfanView/GIMP or other image editing programs, since the image's dimensions are below the 16383 maximum.

Two things that could be improved:

  1. A quick check to see if the resulting file will be too large to convert to WebP (simply measure the chosen files with their respective layout to check the dimensions are below 16383). Then present the user with a dialog box stating that conversion to WebP is not possible, and two buttons:
    • proceed without WebP conversion
    • cancel import
  2. If the image dimensions are okay but the program fails to create a WebP file then present a dialog boxing informing the user that it couldn't complete the WebP conversion and the resulting image should just have a .png (or w.e. is appropriate) file extension (instead of improperly labeling it!)
moo-man commented 1 year ago

2.5.1 provided a simple and hopefully temporary workaround. Basically if you're inputting multiple files, nothing has changed and Firefox still won't work. If only a single file, it should work (excluding things like impractical file sizes or terrible upload speeds).

moo-man commented 1 year ago

Regarding the image size limit for webp, that is also the limit for Foundry map sizes in the first place, or rather, it's the max texture size for GPUs.

moo-man commented 1 year ago

I've removed the workaround and finally found a fix for Firefox. Note that it takes longer (and the resulting Webp conversion seems to have a larger file size than Chrome's) but from my testing, it works!