panda-re / panda

Platform for Architecture-Neutral Dynamic Analysis
https://panda.re
Other
2.45k stars 475 forks source link

Slight tweaks and attempt 2 on centralizing Python Packages #1498

Closed AndrewQuijano closed 2 months ago

AndrewQuijano commented 2 months ago

Few things:

1- Most important, I updated the @ version for GitHub action workflows, it seems some are going to EOL soon, so to avoid anything breaking, might as well fix it for you now.

2- Slight tweaks to documentation, as now there are consistent updates to pip for PyPanda and the Debian Packages.

3- Install_Ubuntu.sh now grabs the Debian Package like from Docker Build

4- I noticed it seems something breaks if python3- stuff was removed from the builds, so I am going to try again to put everything in the requirements.txt and install them pretty early, so hopefully there are no breaks. I just figure it is good practice to try to have one source of truth when installing everything.

I should note that setup.sh from Debian Package worked on my WSL, Luke just make sure this hopefully correctly centralized the Python requirements for Panda and PyPanda

AndrewQuijano commented 2 months ago

I was a bit surprised to notice sphinx being installed in Ubuntu 20 and 22, but not 18. After a bit of poking around, seems like Sphinx is for document formatting.

Wouldn't the better place to install sphinx be in the documentation workflow? https://pypi.org/project/Sphinx/#description

Based on the Workflow file, this seems to confirm my hunch.

AndrewQuijano commented 2 months ago

Wait is pycparser and capstone python libraries needed? I only saw it used in the Docker, but I don't think you get those with install_ubuntu.sh.

And in my original attempt to refactor, I removed pycparser, colorama and protobuf, but it seemed colorama and protobuf had to restored.

AndrewQuijano commented 2 months ago

Oops, realized I accidentally broke the install_ubuntu.sh, it should work now with the whole install from Debian package now. I'm wondering if anything can be done about the warnings? Should we remove the $SUDO from the pip stuff?

image

AndrewQuijano commented 2 months ago

Up to you if we should also try to fix the publish_deb.yaml workflow in this PR, seems like it is tripping up on release name. Any tips on what to try to tweak on tag_name argument?

I would suspect it would be similar to the work done by you and Luke earlier once pypanda was now automatically deployed to Pip. But I'm not sure what versioning rules you might have for publish_deb.yaml

https://github.com/panda-re/panda/actions/runs/9185148186 https://github.com/panda-re/panda/actions/runs/9185148186/workflow

AndrewQuijano commented 2 months ago

Also, wanted to ask if you'd be OK to test the organization secret instead for pushing to DockerHub? I figure we should know if it works or not. The hope is, when LAVA is revived fully, we can use this to also push a LAVA docker image to panda-re dockerhub too.

AndrewQuijano commented 2 months ago

Also, any tips how to upgrade this to point to Ubuntu 18 64-bits? Think this minor fix is probably overdue as well.

https://github.com/panda-re/panda/blob/dev/.github/workflows/parallel_tests.yml#L117-L119

AndrewQuijano commented 2 months ago

I will do two PRs, one for updating Actions versions, and I want to preliminary test uploading Docker Images. In another PR, ill take on the install script cleanups