Read string from file at compile time

[lpil]

Syntax idea:

const sql: String = derive ReadFile("./query.sql")

This would (effectively) generate this code:

const sql: String = "the contents of the file go here"

We would need to assert that the file is valid unicode.

[thehabbos007]

Would we allow this places other than module constants? I might look into this, but no need to assign me :)

[lpil]

I think constants would be a good place to start. We can always extend this feature later.

[tomwhatmore]

This sounds good to me, module constants seems like the best place to start for sure. I assume this can also be the first steps towards derive Template? I like the proposed derive syntax, but I'm not completely in love with ReadFile. I also don't have any amazing alternative suggestions off the top of my head but I'll have a think.

[lpil]

I was thinking the compiler could have a set of derivable things (values?) such as an inlined file, templates, etc. Seems fairly understandable and extendable (though not extendable by users).

Definitely open to ideas for names, syntax, etc :)

[tomwhatmore]

The other current derive proposals seems to be derive Atom("ok") and derive Template(from: "path/to/file"), both of which I like. The name is a noun and the thing you are creating. ReadFile is more about the source than the result and also sounds more like an instruction than a noun. I think them being consistently things would be ideal.

[lpil]

That's a great point! A noun would fit better.

FileContents ?

[oneness]

This is really nice feature. How about using comptime whatever_fn_with_args? Then, we open it up for users to put whatever type of compilation time resolution they decide to do. The above example will be just one use case of this feature without us deciding the how, which should be left to the user. I could be totally missing something here.

[lpil]

Could you give some examples? I don't understand the suggestion

[itsgreggreg]

Some thoughts I had on generalizing this feature. We'll use the keyword derive for now because I don't have a better alternative.

Some things I might want to derive:

• A file to String
• An api interface based off some spec
• A sql interface based off some spec
• what else?

What gleam might do, and again just thoughts written down here, is use what is passed to derive as a call spec for an external binary/script.

Something like const sql: String = derive read_file("./query.sql") would result in the compiler running an executable called read_file (in a specific directory, not system wide) with the arguments ./query.sql and perhaps String. read_file would be responsible for returning valid gleam source that compiled to the correct type. In the case of read_file "./query.sql" "String" that would be a gleam String aka "the contents of the file" but it could be any Gleam source that's valid in the context of a const. Gleam would then have to parse that source and inject it into the AST. Normal parse and type errors would apply.

Something like const sql: Sql = derive sql_for("./sql_spec.json") would result in some binary/script called sql_for getting passed the path and type and might result in the Gleam source for a Sql structure.

Perhaps read_file could be a built-in so the file didn't have to get read twice.

I have no idea how/if this would work on windows.

[lpil]

I see we have been thinking similar things! I've been interested in compile time code generation from external resources for some time.

Some things I might want to derive:

I'm also interested in deriving database query functions from a database schema and SQL query string or file. This could be fully type checked.

Something like const sql: Sql = derive sql_for("./sql_spec.json") would result in some binary/script called sql_for getting passed the path and type and might result in the Gleam source for a Sql structure.

That's an interesting idea. I've been thinking of this as a two stage process, with a short term first stage system that allows us to do things now, and a later longer term one.

The first I was thinking that we write little programs such as this sql_for that generate Gleam code, and invoke them before the build process. This is easy to do today, doesn't require language integration, and leaves things like cache invalidation (i.e. when do we rebuild the generated code?) up to the user. I don't want to call this codegen on every compilation as it will be slow, especially if it involves booting an escript to run Gleam code.

Longer term I was interested in having the Gleam compiler be able to run Gleam code as part of the build process, possibly via wasm. This would be fast and we could at this point have stable APIs for Gleam code generation as well as APIs for cache invalidation. From a language point of view this could be seen as implementing new derivation mechanisms perhaps. Or maybe macro attributes/decorators would be more fitting, like in Rust or Python.

You're suggesting something in the middle. It's nice because it can be used to create members of a module rather than creating a new module, but it does require more langauge integration, and I think it will be difficult to remove APIs like these.

[CrowdHailer]

This looks like a version of a type provider . forexample a constant from a file could be read as follows.

import gleam/raw<"filename">.{value as some_constant}