metalbear-co / mirrord

Connect your local process and your cloud environment, and run local code in cloud conditions.

https://mirrord.dev

MIT License

3.65k stars 100 forks source link

Support ls #120

Open eyalb181 opened 2 years ago

eyalb181 commented 2 years ago

Our files feature currently support most of the relevant syscalls, but we still need to add support for ls.

After #904, this is what is left to be done (prioritized)

[x] Go Linux x86-64 (same flow/integration test should work~)
[x] Python list directory
[x] Node list directory
[ ] ls binary - macOS
[ ] ls binary - Linux
[ ] All of the libc functions:
- [X] opendir
- [X] fdopendir
- [X] readdir
- [X] readdir_r
- [ ] telldir
- [ ] seekdir
- [ ] rewinddir
- [X] closedir
- [x] dirfd

I think the best way would be to drop each as separate PRs, first few tasks will resolve most of the libc functions :)

infiniteregrets commented 2 years ago

openat was fixed in #151, detours for ls still pending but low priority

t4lz commented 9 months ago

Maybe we should recheck some of the use-cases after #2003. Maybe more stuff works now?

Razz4780 commented 9 months ago

Ah yes, I checked ls on linux after #2003 It worked :)

MartinEmrich commented 9 months ago

Should this also include newfstatat()? I run a java application via mirrord, and it cannot find a file that's present remotely (via a configmap mount).

Running within strace, I see this attempt to access that path:

[pid 18182] newfstatat(AT_FDCWD, "/etc/xxx/yyy", 0x7f985ce7fd20, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)

In Java, I use java.nio.file.Files walk() method at that point.

2001 also mentions that Java class, but I have no segfault, and run 3.72.0 which is said to containt that fix.

t4lz commented 9 months ago

@MartinEmrich files under /etc/ are read locally by default. In order to make mirrord read that file remotely even though it is in /etc/, you can add the file's path to the feature.fs.read_only configuration value, or to feature.fs.read_write, if you also want to write to that file. Feel free to come to our discord for live support.

MartinEmrich commented 9 months ago

@t4lz Thanks, that did the trick, it works!

eyalb181 commented 7 months ago

@infiniteregrets Could you please retest the first four use cases (Python list - ls on Linux)?

infiniteregrets commented 7 months ago

macos: does not work on

python: i used os.walkdir and scandir
node: fs.readdir
same for ls, however I am assuming to make ls work we'd have to implement the fts API: https://opensource.apple.com/source/Libc/Libc-825.40.1/gen/fts.c.auto.html

infiniteregrets commented 7 months ago

support for ls in python and node was added by https://github.com/metalbear-co/mirrord/pull/2145