search

elichai / syscalls-rs

Apache License 2.0
23 stars 4 forks source link

Insights from a different approach #10

Open darleybarreto opened 3 years ago

darleybarreto commented 3 years ago

Hi, this is a feature I look forward to help somehow, so I've been looking in the ecosystem to alternatives, particularly this repo itself and this PR. It would help to fast prototype a pure Rust approach with inline asm to when it hits stable (actually convert from llvm_asm), and a stable one using a C compiler, a bit similar to what you already do here. Moreover they have more targets and syscalls implemented, which would definitely of help. So what do you think?

Regards.

elichai commented 3 years ago

Hi :) I'd love to see a PR converting the current inline asm to the new asm format, and also support for more architectures.

about the other approach, I know @japaric steed approach (https://github.com/japaric/steed) but my approach isn't to implement the same unsafe glibc interface but in Rust, but instead to make a safe, sane and usable replacement to glibc in rust

elichai commented 3 years ago

I really do want to continue this project when I have more free time, although pthreads is going to be a huge pain to implement/replace

darleybarreto commented 3 years ago

Hi :) I'd love to see a PR converting the current inline asm to the new asm format, and also support for more architectures.

about the other approach, I know @japaric steed approach (https://github.com/japaric/steed) but my approach isn't to implement the same unsafe glibc interface but in Rust, but instead to make a safe, sane and usable replacement to glibc in rust

Oh, so the idea is to FFI directly to the underlying OS API? Thus one FFI for each target: Linux, *BSDs, Mac, Windows, Redox etc?

I really do want to continue this project when I have more free time, although pthreads is going to be a huge pain to implement/replace

While back I was thinking about this, what you think about using c2rust to translate a pthread (newlibc, ulibc, dietlibc, musl etc) and work from there to rustfy as much as possible?

elichai commented 3 years ago

Oh, so the idea is to FFI directly to the underlying OS API? Thus one FFI for each target: Linux, *BSDs, Mac, Windows, Redox etc?

Yes. I mostly(selfishly) care about linux and planned to do that first. also, AFAIU, both Windows and OSX don't have a stable Operating System calls (syscalls), their only stable API is WinAPI / OSX libc, nothing on the OS level.

While back I was thinking about this, what you think about using c2rust to translate a pthread (newlibc, ulibc, dietlibc, musl etc) and work from there to rustfy as much as possible?

  1. c2rust currently produces a bunch of UB, and 100% unsafe code.
  2. glibc and friends rely on UB even in C, it's too early to know what can/should we do in those instances (There's some talk to allow some illegal reads via volatile, otherwise we might need to use inline asm)
  3. it won't give me a Rusty nice API.

Having said all that, it might be a good starting point to bootstrap this project faster, and later on we'll refactor things into something nicer/safer.

darleybarreto commented 3 years ago

Do you have any idea how Go deals with all this? Do they use raw syscalls? Implementing pthreads from the ground up is definitely a huge amount work :/

EDIT: they do raw syscalls :/

elichai commented 3 years ago

Yeah they do raw syscalls, and they use their own custom "threads" that don't have a lot to do with pthreads. btw, this already bit them in the ass a few times ago (bad tzdata resolving, and dns resolving) and I think these days they do use glibc in networking code unless you pass -tags netgo,osusergo to the compiler

darleybarreto commented 3 years ago

I see, what you think should be the next steps (wrt APIs)? For pthreads I suppose would be browsing the docs and implemeting them, right?

elichai commented 3 years ago

yeah, but I would prefer to push pthreads until the end if it is possible, I'd start with:

  1. Update the ASM macros to the new syntax
  2. Add more ASM for other backends (e.g. ARM linux).
  3. Continue implementing needed syscalls https://github.com/elichai/syscalls-rs/issues/1

I believe that if we do everything except pthreads then it's enough to ask for a new tier 3 experimental target that will run using this library (and will help us see how much work is needed/missing to integrate this into libstd)

darleybarreto commented 3 years ago

Oh, I've missed those syscalls there. So we sure need to do direct syscalls at the end of the day, I thoght the Linux C API would replace their need. What you are trying to avoid is having Rust binaries linked with C ones, right? So could we simply translate syscall.rs from llvm_asm! to asm!?

elichai commented 3 years ago

You need to split between:

  1. glibc API.
  2. Linux user API.

glibc is the huge mess I'm trying to replace here And the linux user API gives you all the needed structures and definitions to call syscalls directly.

The syscalls that need converting are here: https://github.com/elichai/syscalls-rs/tree/master/src/arch

darleybarreto commented 3 years ago

I see, I can't get involved right now, but as soon as I can I'll ping you. Should I close this issue?

elichai commented 3 years ago

Sure, Thanks! always good to hear insights/thoughts from others :)

darleybarreto commented 3 years ago

Hi @elichai,

Recently I've seen rsix, which has two backends: linux_raw (raw syscalls for Linux) and libc for other OSes. It might be interesting to join efforts, but I don't know if their approach to syscalls would be of interest to you.