search

IQSS / dataverse-frontend

An upcoming and modernized UI for Dataverse
Apache License 2.0
19 stars 16 forks source link

Analysis and design of file uploading in the SPA #384

Closed GPortas closed 4 months ago

GPortas commented 4 months ago

Overview of the Feature Request

Analyze and design how to implement file uploads in the SPA. Study the existing mechanisms in the API and the options that React offers us to do it in the optimal way.

Some ideas shared in Slack:

From the client side, when uploading a file, we can create an objectUrl to preview the image of the uploaded file from a local blob ( if its an image) else show some icon according to the file extension.

For actually sending those files to the api we will need to create a formData and append every file to it

I’m still not sure how to implement file upload from the client side and if a use case is necessary, or direct API call (or both)

What kind of user is the feature intended for? Developer

What inspired the request? Frontend team discussion

What existing behavior do you want changed? None

Any brand-new behavior do you want to add to Dataverse? None

Any open or closed issues related to this feature request? None

ekraffmiller commented 4 months ago

In the weekly frontend meeting, the feature of direct upload to s3 was discussed as an important feature to implement. There is a javascript component that is used in the current JSF for this. We will need to learn more to evaluate how to add this to the SPA.

qqmyers commented 4 months ago

FWIW: The DVWebloader, which uses ~ the same Javascript and only supports direct upload, might be worth a look too - in that it handles uploading of a whole folder structure and isn't so tied to how PrimeFaces works.

ekraffmiller commented 4 months ago

Another aspect of uploading to review is handling zip files

GPortas commented 4 months ago

After discussion with the core team and the frontend team, we will use the DataFile Direct Upload/Replacement API as the first mechanism supported by the SPA Beta environment for file uploads.

We have decided to start with this mechanism against file system-based storage, as we anticipate that the latter will require evolution to improve its performance. For example, there is currently no API support for uploading multiple files to the file system-based storage, and the single upload option introduces some drawbacks to application performance (e.g., multiple temporary directories saved for a single file, updating db + solr for each file upload... ). In addition, currently the beta environment already supports direct upload.

In the future, we will need to expand support for other mechanisms, including uploading files to file system-based storage. but for now, since the SPA is deployed in beta and not in other installations (which do not necessarily have to use direct upload), this mechanism meets our current goal for Q2, which is to achieve basic file uploads for the SPA.

I have created an issue https://github.com/IQSS/dataverse-client-javascript/issues/152 to work on a use case that encapsulates the logic with all steps required to upload a file using direct upload. We may find potential API extensions when working on this issue, but for now, we'll stick to documentation and examples like dvwebloader to implement the use case.