Export attachments alongside CSV file

zodac commented 1 year ago

What is the problem you are trying to solve with this feature?

As of v0.9.0, we can export the homebox inventory as a CSV file, but it will not include references to any attachements (photos, PDFs, etc.). This means when restoring from a backup a user would need to manually add all attachments back to these items.

What is the solution you are proposing?

As an initial update, it would be great to be able to export the filePath/name/type of any attachements in the CSV, so they do not need to be manually assigned to each item. This would allow people migrating to at least backup their /data directory and use the CSV.

Ideally, it would be great to export both the CSV and the attachments and compress into a ZIP file as the export archive.

What alternatives have you considered?

Manually backing up the /data directory and re-assigning any attachments to the imported items.

Additional context

No response

Contributions

[X] I have searched through existing issues and feature requests to see if my idea has already been proposed.
[ ] If this feature is accepted, I would be willing to help implement and maintain this feature.
[ ] If this feature is accepted, I'm willing to sponsor the development of this feature.

hay-kot commented 1 year ago

This means when restoring from a backup a user would need to manually add all attachments back to these items.

The CSV import/export is not meant to be a backup. It's meant to be used for bulk updates and creation.

If you want to backup your data you should backup the data directory.

As an initial update, it would be great to be able to export the filePath/name/type of any attachments in the CSV, so they do not need to be manually assigned to each item. This would allow people migrating to at least backup their /data directory and use the CSV.

I don't think there's a nice way to represent this in the CSV file. What sort of API did you have in mind? Can you give a specific example of what the CSV file might look like and the zip file with associated files?

I don't see a way to make the CSV easier to use than manually adding attachments.

The only other approach I had considered was having a CLI that can work directly with the API to do this kind of thing

--

I do think it would be worth it to implement an export feature that let's you get all your data out of HomeBox in an application agnostic way, but my plan there was to just have another button to export all attachments into a zip file.

zodac commented 1 year ago

So, I definitely appreciate it's not going to be a quick update or done anytime soon. But I was thinking something like this:

Currently, an item has a set of attachments, and each attachment has some data (name, filepath, type). So the DB model for an item would have that saved somewhere, but it's not in the CSV output.
The CSV export could be updated to include these attachments (probably some format like name:path:type;name:path:type), and the bulk import could also be updated to populate these fields when importing the CSV file.
At the same time, when an export is performed, the contents of the /data directory could be compressed and provided alongside the CSV file (to avoid the need to manually back up the directory).

I think exporting all attachments would save most users from needing to pull data out of a container or worry about directories, but I think the harder job is putting those attachments back where they came from. :)

It looks like all attachments are stored in a single directory - perhaps if you go down the 'Download all attachments' option instead, the attachments could be stored in a directory corresponding to the asset ID, or something similar? Something to help give an idea which attachment goes where.

Cheers for the quick answer @hay-kot !

verybadsoldier commented 1 year ago

How about including all attachments as base64 strings in the csv? All data could be exported then in the file. The downside would obviously be that the file might become quite large.

hay-kot commented 1 year ago

Been giving this some more thought, here's what I'm thinking.

Have a new button to "Export Attachments" which bundles all the attachments into zip file and let's the user download it.

Structure would look something like this

zip.zip
  manifest.json
  {item_id}
    attachment_1.pdf
    attachment_2.jpg
  {item_id}
    attachment_1.pdf
    attachment_2.jpg

The manifest.json would be a list of all exported items, their locations in the zip file, and summary details of the item, like name and description

zodac commented 1 year ago

That sounds awesome, would make recovery/migration so much simpler. :)

79vette commented 1 year ago

Been giving this some more thought, here's what I'm thinking.

Have a new button to "Export Attachments" which bundles all the attachments into zip file and let's the user download it.

Structure would look something like this
zip.zip
  manifest.json
  {item_id}
    attachment_1.pdf
    attachment_2.jpg
  {item_id}
    attachment_1.pdf
    attachment_2.jpg
The manifest.json would be a list of all exported items, their locations in the zip file, and summary details of the item, like name and description

This would be an excellent feature. Being able to export all data, including attachments, would be really powerful and would make Homebox so much more useful!

crispybegs commented 1 year ago

+1 for this feature

KonstantinShemyak commented 6 months ago

+1 for the feature! Use case: I need my Homebox to be available for family members, so it must be deployed in a place accessible from the Internet. But I don't want anyone to have access to my account at the cloud provider. So, they cannot backup /data directory.

(Besides that, the other users are not that techy; they can click a button to download a backup, and a button to restore from the backup - but not "backup a directory at somewhere".)

hay-kot / homebox