search

bloomberg / memray

Memray is a memory profiler for Python
https://bloomberg.github.io/memray/
Apache License 2.0
13.17k stars 392 forks source link

Add support for FreeBSD #593

Closed mohd-akram closed 4 months ago

mohd-akram commented 4 months ago

Is there an existing proposal for this?

Is your feature request related to a problem?

Currently, trying to install this on FreeBSD 14 yields RuntimeError: memray does not support this platform (freebsd14). I'm not sure if it requires anything specific from FreeBSD, but BSDs are similar to macOS.

Describe the solution you'd like

Support for FreeBSD

Alternatives you considered

No response

godlygeek commented 4 months ago

Well, PyPI doesn't allow pre-built wheels for BSDs, so there's no way we could possibly give first class support for them. If you'd be interested in porting Memray to FreeBSD, we'd be willing to accept patches for that, if they were small enough and unobtrusive enough. You could start by cloning the repo, deleting the line giving that error from the setup.py, and seeing what fails next. Our GitHub Actions workflows and cibuildwheel configuration should give a decent overview of what our native build dependencies are on each Linux distro we build on, and on macOS.

The hardest parts to port by far will be support for memray run --native, and support for intercepting malloc calls.

  1. You'll need a way to iterate over all of the shared libraries that have been loaded in the process's address space. Memray uses dl_iterate_phdr for that on Linux, and uses _dyld_get_image_header for it on macOS. FreeBSD would need some API that can be used to do the equivalent.
  2. You need a way to intercept calls to malloc in each of those shared libraries. Conceptually this is "just" patching the Procedure Linkage Table, but the way to do that is platform dependent. The files linked above are Memray's implementation of this for Linux (ELF) and macOS (machO). FreeBSD seems to use ELF, so the Linux implementation may be close to right for FreeBSD.
  3. You'll need an unwinder to unwind native stacks (we use libbacktrace on Linux, I'm not sure if it supports FreeBSD or not).
  4. You'll need a way to symbolify native symbols (we use libdw and libelf from elfutils to do this on Linux).

At a glance, it seems like dl_iterate_phdr might exist on FreeBSD, so if elfutils and libbacktrace support FreeBSD, this might "just" require an implementation of the PLT patching that works for FreeBSD, and otherwise treating FreeBSD like Linux everywhere that we've got platform-specific conditionals.

Since none of the maintainers of this project use FreeBSD, we're highly unlikely to ever do this work ourselves. If you're interested in doing the porting, we can answer questions to help guide you along, and unstick you if you get stuck. If you're not interested in being the one to do that porting, we'll just close this issue. This isn't something we're ever likely to do ourselves.

godlygeek commented 4 months ago

I'm going to close this for now, since I haven't heard anyone volunteer to do this work yet.

If anyone wants to try to take it on, feel free to reach out for help or advice.