Convert usages of Pipenv to PDM

dhruvkb commented 6 months ago

Description

We have opted to use PDM as our package manager for the API and various other Python libraries in the monorepo (with the API migration happening in #4107). We still using Pipenv in many other places and in the interest of consistency between our toolchains, we would like to migrate all these projects over to PDM as well (potentially also reorganise them into the new Python packages directory).

[x] automations/python/ - #4625
[x] documentation/ - @dhruvkb #4266
[-] ingestion_server/ - moved to #4658
[x] utilities/provider_tallies/ - #4624
[x] utilities/dead_links/ - #4623
[x] utilities/project_planning/- #4622
[x] utilities/generate_test_locales/ - #4626

More

We also have many references to Pipenv, specifically Pipfile and Pipfile.lock scattered across the codebase. Once these projects are no longer using Pipenv, these references will also need to be updated.

[x] documentation/general/general_setup.md
[x] documentation/automations/guides/quickstart.md - #4625
[-] documentation/meta/maintenance/elasticsearch_cluster.md
[-] ingestion_server/Dockerfile - moved to #4658
[-] ingestion_server/.dockerignore - moved to #4658
[-] ingestion_server/justfile - moved to #4658
[-] .pre-commit-config.yaml - moved to #4658
[-] .github/actions/setup-env/action.yml - moved to #4658

Additional context

The goal is to make a number of small atomic PRs, that migrate only one project and its references per PR, so that they can be quickly verified to work and merged. Please comment below which tasks from above you will be working before starting work on them.

malik-aditya-r commented 6 months ago

Hey @WordPress/openverse-maintainers, If this issue is still available I'd like to work on it.

dhruvkb commented 6 months ago

Yes @malik-aditya-r, it is still available. Please mention which of the above you will be migrating and make small PRs for each target. I'll assign this to you.

malik-aditya-r commented 6 months ago

Hey @dhruvkb Thanks for assigning me the task. I'll like to start with automations/python and once that is successfully done move on to utilities/* and others. I'll start by going through the PR #4107 And will reach out if any questions arise.

obulat commented 3 months ago

@dhruvkb, it seems that there are only 2 non-ingestion-server items that need to be fixed here.

Would it be okay to extract the ingestion-server-related changes to a separate issue and move it to the "Ingestion server removal project"? .pre-commit-config.yaml and .github/actions/setup-env/action.yml also cannot be updated until the ingestion server is removed.

We could then close this one after the 2 remaining documentation issues are fixed here.

dhruvkb commented 3 months ago

That makes sense, I don't think migrating the ingestion server to PDM is worth the effort because it is on the way out. We can move the ingestion server (and the others blocked on it) to a new one, and close this issue.

obulat commented 3 months ago

general setup.md did not have any mentions of Pipenv elasticsearch.md refers to the ingestion server, so I'm moving it to #4658.

All changes here are done 🎉

WordPress / openverse