search

asterics / AsTeRICS-Grid-Boards

Holds default configurations used by AsTeRICS Grid
GNU Affero General Public License v3.0
0 stars 1 forks source link

Keep json formatted #1

Open mtrakal opened 1 month ago

mtrakal commented 1 month ago

Hello, I wanted to translate grids/communicators but files are single line, so it will not be possible validate changes... Is issue for you, keep files formatted?

I use VS code to format code, is that OK for you? I'll prepare PR...

How I prepare files, are steps correct?

klues commented 1 month ago

Wow, didn't expect any issues or pull requests on that repo that fast. It's work in progress - currently the files of default grid configurations are stored in the main AsTeRICS Grid repository, but this has some disadvantages. In the future the data will be organized within this repository (but it's not yet used).

I don't think I want to keep the configuration files as formatted json, since it's the data that is direclty downloaded and imported in AsTeRICS Grid in production and therefore it should be as small as possible.

So for translating anything I would recommend one of these:

  1. Option 1:
  2. Option 2:
    • you could format the json, add the translations directly within the json, minify the json and then do the PR with the changed file (however I imagine it's not very comfortable to edit a file with 50 000 lines)

However, I also like the idea of being able to do translations with a good diff view here on Github. So maybe we can find a way of keeping both formatted (for editing) and single-line json (for downloading). But I'm currently not sure what's the best way to achieve it. The ultimate goal would be a webpage, where it's possible to manage translations and automatically create pull requests here. Maybe we could combine it with crowdin somehow:

I think this would be a good idea, but I still have to think about the details. For now please proceede like written above.

mtrakal commented 1 month ago

Hello, it's just a good time anomaly :D. I started translating one grid a few months ago, but never finished it (no time for it)... and now I started again... But I want to know how the grid sources are taken because my last try to translate was long time ago and grid source could be changed in mean time. So I think about "how to synchronize" existing old / not finished/ translation with latest grid and found this new repo :).

For the future, it would be nice, if grids itself are without translations (maybe keys only or one source language?) and translations were taken in build time from some translation file/crowdin etc.. It would improve translating speed and how easily translations can be added.

I think, that you should keep json in formatted form here and minify them in compile time. What about gzip grd/json files on webserver / when using HTTP protocol? It will reduce much more, than just minify json (don't know if it's used already)

I can minify it by myself, but doing diff is harder in that case

mtrakal commented 1 month ago

since it's the data that is direclty downloaded and imported in AsTeRICS Grid in production and therefore it should be as small as possible

What about using custom GitHub actions (you could "minify" json in step (3) before upload static pages). In case, that you plan to download jsons from GitHub pages and not using git submodule or just fetch this repo before build.

klues commented 1 month ago

For the future, it would be nice, if grids itself are without translations (maybe keys only or one source language?) and translations were taken in build time from some translation file/crowdin etc.. It would improve translating speed and how easily translations can be added.

In fact in a very early version, I had grid files and translations separated. However I've changed it, because it's much more comfortable in many other aspects, e.g. at translating things within the app or at downloading single grids. However, as said, maybe we can find a way of extracting separate translations files out of the standard configs here in this repository. I think the difficult part is how to make it possible to still be able to adapt the contents of one of the grids using the app.

What about gzip grd/json files on webserver / when using HTTP protocol? It will reduce much more, than just minify json (don't know if it's used already)

Github pages (the way how the files are delivered) is automatically using gzip for .json files. That's also the reason I've named the files .grd.json here.

What about using custom GitHub actions

I've never used it, but yes, maybe it can help somehow.

mtrakal commented 1 month ago

Check this repo: https://github.com/mtrakal/github-pages-minify-json and output: https://mtrakal.github.io/github-pages-minify-json/

What does the magic: https://github.com/mtrakal/github-pages-minify-json/commit/d132d5b36859f02e713404c2f61bcb20eecc3728#diff-36a7fe6e9af1fc515ab07c70bee1dd529453c776db13abf7f5c6b2b3b34c4be3

&& truncate -s -1 "{}" is not necessary, but without it, it will make changes to current files. So with call truncate, all json files content is same as now. But UTF8 with BOM is now without BOM - if that should be issue for something, it could be encoded with BOM using someething like: https://stackoverflow.com/a/3293858/4024146 But it looks, that JSON should be without BOM: https://stackoverflow.com/questions/4990095/json-specification-and-usage-of-bom-charset-encoding

Whole workflow file: https://github.com/mtrakal/github-pages-minify-json/blob/main/.github/workflows/jekyll-gh-pages.yml

It's just copied from "sample" to build Jekyll (what doed Github Pages pipeline).

klues commented 1 month ago

Thanks for the examples, I'll have look at the Github workflows in detail after I've finished some other things regarding the grid sharing feature I'm currently working on.

In the end I think it should be possible that someone creates a PR on this repository with a new .grd backup file, an info.json with metadata about it and maybe some screenshots (so like the contents of e.g. this folder) and then some workflows / scripts are doing this: