feat(site-builder): treat unknown file extensions defaulting to arbitrary binary data

[Tzal3x]

Summary

Treat files without or with unknown extension as binary files.

Whenever the dir that is to be published contains a file with no extension (e.g. file) or an unknown extension (e.g. file.<unknown>), a binary content type will be assigned to it, and a warning will be shown.

The octet-stream content type indicates a response body with arbitrary binary data as defined in RFC 2046.

https://datatracker.ietf.org/doc/html/rfc2046#section-4.5.1

Example output

Given the following tree structure:

snake
    ├── Oi-Regular.ttf
    ├── file.UNKNOWN_EXTENSION  <----- [!]
    ├── index.html
    └── walrus.svg

And running ./target/release/site-builder --config site-builder/assets/builder-example.yaml publish examples/snake/

Returns:

[...]
Parsing the directory examples/snake/ and locally computing blob IDs ... 2024-07-18T11:59:04.889753Z  
WARN site_builder::site::resource: The extension UNKNOWN_EXTENSION string for file examples/snake/file.UNKNOWN_EXTENSION could not be decoded.
                        Defaulting to arbitraty binary content type: octet-stream.
                                                                         [Ok]
Storing resource on Walrus: /Oi-Regular.ttf ... [Ok]
Storing resource on Walrus: /file.UNKNOWN_EXTENSION ... [Ok]
Storing resource on Walrus: /index.html ... [Ok]
Storing resource on Walrus: /walrus.svg ... [Ok]
Updating the Walrus Site object on Sui ... 2024-07-18T11:59:11.853632Z  WARN sui_sdk::wallet_context: Client/Server api version mismatch, client api version : 1.27.2, server api version : 1.29.2
[warn] Client/Server api version mismatch, client api version : 1.27.2, server api version : 1.29.2

Execution completed
Operations performed:
  - created resource /Oi-Regular.ttf with blob ID 2YLU3Usb-WoJAgoNSZUNAFnmyo8cfV8hJYt2YdHL2Hs
  - created resource /file.UNKNOWN_EXTENSION with blob ID dxrDLHUYJTgMnctlYPvnWrS-RhlAUr5H3X3oruzxPcM
  - created resource /index.html with blob ID pZTI8Wk_CRgsYLrsgo2TUDEoR1_OiuFpjwUdZn6W7fU
  - created resource /walrus.svg with blob ID cfbbG-JRn8NHIWVJtNmo7-hEC02NpM5Rpmc8-AF8n1g

Created new site: test site
New site object ID: 0x0160cac917f5bcf0bde281b2331cc44fe1efd857ef00232bb41bb77671f07c71
Browse the resulting site at: https://18iimu4lot8c4jrkomn96zdzo748bv86a5o1exp7gyvx4wuwh.walrus.site

Here is the site Resource object of the file with unknown extension.

Closes #110

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
walrus-site	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 19, 2024 9:29am

[Tzal3x]

Thanks @Tzal3x , this is great. Just one quick question: this should work even if the file has no filename, but only the extension (e.g., .gitignore), correct?

Correct! 👍 Here is an site Resource example of a .svg file: https://suiscan.xyz/testnet/object/0xca8703f61474f846c354583c178a9e584b6a4b07b9c81a9e1c769485147eaf28