Archiving design files

[xaur]

Following the discussion in @linnutee's proposal I think we can use an issue to collect ideas about more robust archiving of design files.

The ways to publish design outputs that I see now are uploading them as attachments to GitHub issues and releases. Other ways are to publish them on YouTube, Behance, Notion, Adope, LottieFiles, Sketch, etc. All these are proprietary platforms that host our files and offer no way to get them all at once and update incrementally. This is very different from source code produced by the development domain, where I can periodically update a dozen of Git repositories and get a full copy of everything. Even if we face any issues with GitHub, the Git repos could be re-hosted elsewhere. And the fact that full repositories are stored and synced by dozens of people makes this data very resilient. In case of design, all that is currently stored in Git is the README file.

My concern is that should anything happen to GitHub or the other platforms, community's access to design files will be interrupted.

To be fair, Git is not the best tool to store large binary files but I think it could handle repos below ~1 GB and work as an intermediate solution. For storing more data we can explore other tools, but even the good old file server with rsync would be more resilient than GitHub attachments.

I think we can try collecting all "release"-grade design files in a proof-of-concept Git repo and see how big it gets, how painful the usage will be, how to better organize it, etc. I can help with some Git tentacles.

Two questions I came up with when thinking about the structure are:

• do you have any text documents that are worth saving in Git?
• do you need easy and quick access to past versions of design files, or are they rarely/never accessed?

[kyleFirethought]

@xaur

Re: Animation Library.

So it should be super easy to upload an archive of the Lottie animations here as they are small .JSON files. This would eliminate the need to upload larger rasterized images or even original project files and these individual .JSON files can be used to re-create any given animated composition in Adobe After Effects.

The above falls into the category:

do you have any text documents that are worth saving in Git?

Much of the larger pieces I currently host on my company GDrive. These are the more problematic larger files where we have traditionally struggled to source a non-proprietary location.

Thanks for raising this issue. This is very much a conversation that needs to be had.

[xaur]

@kyleFirethought thanks. Using text-based source files is always preferred since they are small, Git is great at versioning them, GitHub provides great collaboration tools for them, and they act as the original source that everything else can be generated from.

The more we can migrate to open and text-based source formats, the better. For example, I would love to see HTML+CSS-based presentation slides over the binary pptx and pdf files. I do realize it's not always possible but imo we should do our best, esp. for new work.

Lottie JSON are great for Git but they are not those "text documents" I had in mind. I mean English texts describing design decisions and such, ones that linnutee referred to as "commentary and additional context".

These are the more problematic larger files where we have traditionally struggled to source a non-proprietary location.

Do you mean the main problem is size and not some copyright restrictions? How big are the files you think should be distributed among other design outputs?

[kyleFirethought]

Thought I would revisit this subject as there have been some movements:

I have a local branch with Git Large File System installed which should allow us to upload slightly larger design files to GitHub.

Video is out of the question at present. I think I used all of December's bandwidth allocation for the Decred Repo trying to upload those DID videos.

Most of the current outputs work file and this is all being organised in GH rather seamlessly. Would be great to see the icon set and other items pushed but I'm sure it's only time. @xaur

If you have any further suggestions It would be great to hear them.

[xaur]

Cool, I never used LFS and it's good to have it explored by someone. LFS may not be the best solution we settle on for archiving, but it's better than having files scattered across centralized proprietary "a few wrong words and you're out" platforms, email or chat attachments. So it is already an improvement.

Ideally we would patch our processes too to have design artifacts reliably preserved for the community as part of the production pipeline (so that there is no hunting afterwards and nothing is ever lost).

In the big picture it's still that missing file archive that could benefit so many use cases. There are many ways to implement it, most importantly it must be easy to sync by archivists and not too hard to upload to (with authentication ofc). Integrity protections and audit log would be great, too. I can volunteer to keep it well organized but I can't run the servers or research/setup the tech (not enough time currently).

Last time I checked jz had some interesting prototypes to explore.

[xaur]

Relevant for this issue, I am archiving design outputs made for the DJ proposal in a specialized repo: https://github.com/xaur/decred-journal-files

Git is not ideal for this, but in our case the load seems to be manageable. The repo grows at around 5-10 MiB/month and is currently currently 62 MiB.

[ta-lind]

Will refer in the dcrdesign release with the next update eom.