Using With Rust Application?

[1oglop1]

Hi, I spent the afternoon trying to build this and I only managed to build this using cargo-lambda - because it is using zig build https://www.cargo-lambda.info/commands/build.html#compiler-backends

In Codespace and after modification of .devcontainer I've got an error + warnings in the newer rust

warning: flag `-Z sparse-registry` has been stabilized in the 1.68 release, and is no longer necessary
  The sparse protocol is now the default for crates.io

   Compiling autocfg v1.1.0
   Compiling proc-macro2 v1.0.50
   Compiling quote v1.0.23
   Compiling unicode-ident v1.0.6
   Compiling syn v1.0.107
error[E0635]: unknown feature `proc_macro_span_shrink`
  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.50/src/lib.rs:92:30
   |
92 |     feature(proc_macro_span, proc_macro_span_shrink)
   |                              ^^^^^^^^^^^^^^^^^^^^^^

   Compiling libc v0.2.139
For more information about this error, try `rustc --explain E0635`.
error: could not compile `proc-macro2` (lib) due to previous error
warning: build failed, waiting for other jobs to finish...

[1oglop1]

PS. I managed to get .devcontainer running on Mac M1. By setting the platform in and

Dockerfile

FROM --platform=linux/amd64 mcr.microsoft.com/vscode/devcontainers/rust:0.202.9-bullseye

devcontainer.json

{
"runArgs": ["--platform=linux/amd64" ],
}

[metaskills]

Thanks for looking into the project. Can you share what you're trying to do? Update Rust? Something else?

It has been a while since I spun this up in a devcontainer. Back in the day I had to lock this down.

• https://github.com/rails-lambda/crypteia/blob/main/.devcontainer/Dockerfile#L1

I noticed some tests are failing but not looked into it. Would gladly take any help here.

[1oglop1]

@metaskills I was just trying to compile the extension on Mac M1 because the readme is misleading https://github.com/rails-lambda/crypteia/blob/1c4dc296305a3fe8090d9b947de0e813bcc39da3/README.md?plain=1#L93 and releases do not contain binaries in the assets. Only 1.0.0 has assets. It is most likely caused by CI error https://github.com/rails-lambda/crypteia/actions/runs/5038969425/job/13642324589#step:3:10

I remember getting the same error as the one in CI/CD https://github.com/rails-lambda/crypteia/actions/runs/5858183970/job/15881726916#step:3:561

Error: Dev container up failed: Command failed: /bin/sh -c ./.devcontainer/postCreate (exit code: undefined)
  The postCreateCommand in the devcontainer.json failed.
  Error: Command failed: /bin/sh -c ./.devcontainer/postCreate

Or something else as well. I assumed it has something to do with the arm64 architecture I simply fixed it by forcing x86_64 container.

The benefit of compiling with cargo-lambda it has built-in support for x86_64 / arm compilation and therefore no need for specific containers/systems.

I also discovered that project cannot build with the nightly rust 1.73 perhaps 1.72 so the file rust-toolchain.toml

[toolchain]
channel = "1.71.0"

should fix the build errors.

At the moment I do not have the capacity to look into CI/CD of this project but I think it may become a part of our infrastructure so I could spend some more time contributing back.

A little bit OT: ~Have I misunderstood the project and it is not usable with AWS managed runtime?~ ~I was hoping to use it for rust lambdas and managed runtime has faster startup than the container.~ It works with nodeJS, now I am trying to make it work with rust binary I wonder if I am missing anything.

[metaskills]

So, WRT to the x86... this project is not setup to work on arm64... at least from a dev environment. GitHub Actions is x86 (for now) and the whole project is setup to work from that perspective and build arm64 using QEMU emulation.

When I would work with this project in Codespaces, since I too have an M2 Mac, my Codespace would be x86. Ideally the Devcontainer folder would work for local VS Code development, but that would require changing a bunch of code and hence break the CI/CD & Package pipelines because they use the Devcontainer. Does all that make sense?

One way around this locally would be to install Colima which forces all Docker contexts to be x86 emulated on the Mac. But I should not ask folks to do that since running Colima default context is just going to make your Docker experience slow, but if you knew how to dynamic switch contexts... I do, pretty easy... then that could be a solution. Ideally, if GitHub would come out with arm64 runners for Actions... maybe the work to make it compatible with x86 and arm64 from a Dev, CI/CD, and Package would be worth the work.

A little bit OT:

Yup, should work with all managed runtimes on Amazon Linux 2. Last I checked. Simple Docker install should be fine.

[1oglop1]

I see, all that makes sense. Then IMO using flag --platform should not pose any difficulties and just recommend Mac development in codespaces. It is quite painful to work with docker on arm Mac :/ so I was happy that cargo lambda could simply build the binary for both arm or x86.

But I have no clue who LD_PRELOAD not working with it. I suspect it has to do something with linking.

I asked about this issue in cargo lambda, so let's see if something comes up. https://github.com/cargo-lambda/cargo-lambda/discussions/504

I am trying to compile my project also in docker and see if that makes any difference with LD_PRELOAD

UPDATE: even after all shenanigans dockers and OS's, LD_PRELOAD isn't working so I am quite confident with cargo lambda producing good binaries which may be worth using for CI/CD.

[metaskills]

But I have no clue who LD_PRELOAD not working with it.

So are you saying you are using a certain version, like this and it is not working? If so I'd like to hear more about that.

https://github.com/rails-lambda/crypteia#lambda-containers

[metaskills]

For example, would you try ghcr.io/rails-lambda/crypteia-extension-amzn:1.1.2 just in case? Have you enabled the debug ENV var to see what is happening on init?

[1oglop1]

For example, would you try ghcr.io/rails-lambda/crypteia-extension-amzn:1.1.2 just in case? Have you enabled the debug ENV var to see what is happening on init?

Yes, I did enable the debug Env var, it all works as expected.

I will try the image in a week (going off).

When navigating around the code I noticed the hack for Python and the mention of LibC vs Glib. Here I absolutely do not know what am I talking about because it's the first time I am trying to read anything about it and it's not easy to find digestible docs.

But it makes me wonder if it has to do anything with rust as well

but here are their rust versions https://docs.rs/libc/latest/libc/index.html https://docs.rs/glib/latest/glib/index.html

[1oglop1]

Here is the log from the extension, I suppose it works, also the output of ldd bootstrap confirms that library is loaded but it has no effect.

[metaskills]

trying to read anything about it and it's not easy to find digestible docs.

Ouch. Heard. Docs are hard :(

But it makes me wonder if it has to do anything with rust as well

Nah... PHP should be the only outlier. For some reason the team that wrote PHP did not use libc's getenv.

Quick questions... are you trying to use this extension with a Rust runtime?

[1oglop1]

Yes I wanted to use it with Rust runtime as well as other languages.

[1oglop1]

@metaskills do you have any idea why it does not work with the rust runtime?

[metaskills]

No idea. Googling suggests that there is nothing about a Rust app that would not be able to leverage LD_PRELOAD. I guess that could mean there are some ways you could writ the Rust app to not work with libc's getenv, such as Python did.

[1oglop1]

I will need to make a small poc and override different functions. There are other ways than redhook so may need to try that. Unfortunately I do not have time for this atm. so I will just read the envs from ssm in runtime. Maybe some weekend learning will allow me to build a poc. Can you point me to some docs around this?

Or even concepts because I know nothing about C Api etc...

On Fri, Aug 25, 2023, 13:57 Ken Collins @.***> wrote:

No idea. Googling suggests that there is nothing about a Rust app that would not be able to leverage LD_PRELOAD. I guess that could mean there are some ways you could writ the Rust app to not work with libc's getenv, such as Python did.

— Reply to this email directly, view it on GitHub https://github.com/rails-lambda/crypteia/issues/40#issuecomment-1693242592, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZCK5RF7SYQRENWTGUS6HLXXCHJXANCNFSM6AAAAAA3PJKJ7I . You are receiving this because you authored the thread.Message ID: @.***>

[metaskills]

Can you point me to some docs around this?

I'd have to Google around myself and would look to you for help with that. One thing I would do is write a very very very small Rust program to test with and verify if/why at the smallest level this is not (or is) working. If it is, I can add a test case here like the other languages. If not, then we can look into other solutions. Like, could this repo be a cargo package that you can build in the behavior right into your app?

[metaskills]

Gonna track in #30 unless there is a specific callout here such as #30 does.