ehrenb / machina

A scalable and recursive binary analysis pipeline
https://machina.behren.me
2 stars 0 forks source link

auto-render schemas as HTML in docs #22

Open ehrenb opened 1 year ago

ehrenb commented 1 year ago

Can't do multi-stage builds in the docs Dockerfile, because there is a circular dependency problem.

...
# multi-stage build to copy in worker source modules
# for autodoc'ing their source and schemas
# TODO: resolve how to mock imports for each, as 
# we dont want to have to install all 3rd party deps
# for all workers. see https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_mock_imports 

# FROM behren/machina-androdguard:latest as androguard
# FROM behren/machina-binwalk:latest as binwalk_img
# RUN mkdir /machina/binwalk && touch /machina/binwalk/__init__.py
# COPY --from=binwalk_img /machina/src /machina/binwalk

# FROM behren/machina-bz2:latest as bz2
# FROM behren/machina-exif:latest as exif
# FROM behren/machina-findurls:latest as findurls
# FROM behren/machina-gzip:latest as gzip

# FROM behren/machina-identifier:latest as identifier
# RUN mkdir /machina/identifier && touch /machina/identifier/__init__.py
# COPY --from=identifier /machina/src /machina/identifier

# FROM behren/machina-jar:latest as jar
# FROM behren/machina-similarity:latest as similarity
# FROM behren/machina-ssdeep:latest as ssdeep
# FROM behren/machina-tar:latest as tar
# FROM behren/machina-zip:latest as zip
# FROM behren/machina-ghidra-project-creator:latest as ghidra-project-creator

...

Also, to import these for autodc, we can use mock-import to suppress the import warnings instead of bloating the image with all dependencies:

conf.py


autodoc_mock_imports = [
    'python-magic'
]

For now, just keeping referential documentation in workers.csv in the docs repo.