Bulk import enhancements

RaythaHQ / raytha

Raytha is a powerful CMS with an easy-to-use interface and fast performance. It offers custom content types, a template engine, and various access controls. It supports multiple storage providers and an automatically generated REST API. Upgrade your development workflow with Raytha.

MIT License

154 stars 30 forks source link

Bulk import enhancements #144

Closed apexdodge closed 11 months ago

apexdodge commented 1 year ago

We want to build the following enhancements to our Import from CSV feature.

Validate each content item before inserting. If it fails validation, add it to a list of errors. This list then is turned into a CSV file that is downloaded at the end of the bulk import process so the user knows which records failed to import successfully.
Swap out the CSV reader with one that is MIT licensed instead of Apache 2.0.
Support for attachment field type.

smd-11 commented 1 year ago

@apexdodge ,would like to work on this.

Is it fine if this https://github.com/drcolombo/CsvReader (MIT License) is used.
Need your inputs on how support for attachment field type can be implemented(for Image we can have base64 string of that image in csv but for pdf's and other file formats not sure)

apexdodge commented 1 year ago

@smd-11, yes, this is a good task, thank you.

Yes, that library looks good. Go for it.
I'm thinking the user must provide a URL to a file. If the URL is valid, download the file into memory, upload using IFileStorageProvider, then save a MediaItem to the db table, and finally the Guid from the MediaItem that was just saved can be used as the value to save to the attachment field type.

Helpful example of file storage provider: https://github.com/RaythaHQ/raytha/blob/7a12f5685a66c2c75bdca8cc07edfdf45734535a/src/Raytha.Web/Areas/Admin/Controllers/MediaItemsController.cs#L77

Helpful example of validation of fields: https://github.com/RaythaHQ/raytha/blob/7a12f5685a66c2c75bdca8cc07edfdf45734535a/src/Raytha.Application/ContentItems/Commands/EditContentItem.cs#L42

You may want to rebase your fork from dev.

Let me know if you have any questions from that.

Thanks!

smd-11 commented 1 year ago

@apexdodge created PR for merging.

apexdodge commented 1 year ago

PR merged, moving to testing. Big thanks @smd-11

apexdodge commented 11 months ago

Fixed a blocker that was discovered during testing.

When choosing to use local storage instead of cloud storage, a crash occurs due to missing httpcontext in background task. This is now resolved and testing can be resumed.

apexdodge commented 11 months ago

Released with v1.1.3