Current upload size limit of 100MB: document or refactor?

[ikreymer]

The current API we're using restricts each WACZ file to 100MB. We would need to use different API to be able to upload files >100MB. We should either: document this limit and perhaps add a check in the UI to reject files >100MB OR refactor to remove this limit.

However, since this essentially allows unlimited uploads on a shared key, perhaps we should keep the limit for now, until we integrate support for allowing users to enter their own key / register with web3.storage / etc...

@SuaYoo @Shrinks99 @RangerMauve what are your thoughts on this?

[Shrinks99]

I'm okay with not providing people with unlimited storage for free? 😅

There should definitely be a note about this in the current UI. If the plan is to provide people with some amount of storage going forward (perhaps even temporary) I like the option of displaying a warning and prompting people to change their back-end if they are trying to upload files that are >100MB.

EDIT: Added note in https://github.com/webrecorder/wacz-uploader/commit/65cff9eb3b7b25f23682415c1ef6be0cb8f26b2b

[SuaYoo]

+1 limit and document upload size limit for default storage token. The UI for backend settings is almost there, just hidden, created https://github.com/webrecorder/wacz-uploader/issues/12

[edsu]

Should it be required for users to enter their own key?

[RangerMauve]

Following up on this, talked to Ilya about switching the uploader to use CAR files instead of raw files which might get rid of the restriction.

However, it might be good to restrict stuff anyway if we're using the default token, as well as have a disclaimer about it not being "permenant" or "private" storage and that they should bring their own token if they want it to persist indefinitely.

IMO having a hardcoded key for folks to just try out is useful for quick demos and getting folks to try it out before going through the hassle of setting up an estuary or web3.storage account.