search

decaporg / decap-cms

A Git-based CMS for Static Site Generators
https://decapcms.org
MIT License
17.66k stars 3.03k forks source link

netlify-cms-proxy-server throws errors when collective filesize of medialibrary exceeds some limit #6731

Open airtonix opened 1 year ago

airtonix commented 1 year ago

regarding: https://github.com/decaporg/decap-cms/issues/5472#issuecomment-1468781556

Disabling publish_mode as in https://github.com/Knogobert/ntn-boilerplate/issues/2#issuecomment-785077531 resolved the issue for me.

This didn't work, I did some git bisect'ish work and discovered that my guy had been uploading 30mb images (lol).

Removing a number of these suddenly made the error go away.

interesting to note: The error only occured when trying to view the CmsCollectionFile types of collections.

Describe the bug

have lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and of images that are 30mb and over in size for the media library to load.

To Reproduce

create a massive amount of images....

Expected behavior

no error

Screenshots

see linked issue thread: https://github.com/decaporg/decap-cms/issues/5472#issuecomment-1468781556

Applicable Versions:

CMS configuration

Additional context

martinjagodic commented 1 year ago

Hi @airtonix, please edit the issue so that it follows the template. A link to a comment on another issue is not enough. If it's a separate concern please describe it here in detail and if it's not, this should be discussed on the main issue thread.

airtonix commented 1 year ago

@martinjagodic I'd have thought that :

a) the power of the internet is based on "links to things" was a strong and positively good habit b) that the title is explanitory enough.

martinjagodic commented 1 year ago

Thanks for updating the issue @airtonix. The thing with links is that they expire and it's harder to understand the issue if no context is provided. Although the title may seem obvious to the poster, it helps others to have more info, even if it's trivial.

luanraithz commented 4 months ago

I'm facing the same problem, I have about 1.2GB of images inside the media folder, the code seams to read every image into memory.

https://github.com/decaporg/decap-cms/blob/bc59b51a30c6e6953343acc0413de288889a4384/packages/netlify-cms-proxy-server/src/middlewares/localFs/index.ts#L95

The API timeouts at 60 seconds in my case. After that the UI works just fine.

I'm wrong? Is there any workaround?

luanraithz commented 3 months ago

Is there a real need to load the whole content into memory? The path or URL should do the job, right?

sznowicki commented 1 month ago

I'd like to bump this issue as it currently hits us as well.