DerwenAI / kglab

Graph Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, NetworkX, RAPIDS, RDFlib, pySHACL, PyVis, morph-kgc, pslpython, pyarrow, etc.
https://derwen.ai/docs/kgl/
MIT License
574 stars 65 forks source link

Poetry build #300

Open jake-aft opened 1 year ago

jake-aft commented 1 year ago

Current behaviour

pyproject.toml uses flint to build

New expected behaviour

Issue: Move setup.py functionality to poetry for pre-check and package build and release - https://github.com/DerwenAI/kglab/issues/275

Build and publish the package with poetry in the pyproject.toml

I'd like to recommend that we put this on a branch to further test and validate it is doing everything we want/need it to do

Unsure of the Makefile that builds the docs and how it might be integrated into pyproject.toml with poetry Need to review some of the pre-commit-config options

I am working on poetry build / publish instructions these will need to be integrated with the docs build instructions for whomever will be building the package not sure if this is something we need in the repo?

The pyproject.toml needs to have the version changed to the version that will be correct at the time of merge

Change logs

added exclude to pyproject.toml of the tests/rdf_tests to not be included in the package

Updated pre-commit-config to check the poetry additions Updated pyproject.toml to use poetry build and dependency layouts

Size of the package is now ~50 K for both whl and zip

Mec-iS commented 1 year ago

Is it possible to have merged all the three PRs about installing/building in this branch and close the other ones? Once they are in the same branch will be easier to review.

jake-aft commented 1 year ago

@Mec-iS i think it is better to have the two separate.

The other pr #299 is really quite minor and mostly updated install instructions, so there is limited risk or impact. Adding Poetry and Pipenv to the install instructions.

This one is a significant change that uses poetry to build the kglab package for distribution. I do think this needs to be explored in some detail to make sure that we do want to move to a poetry build/dist model and the implications of moving to that from the existing build/dist tools.

As an example, I do not understand completely how the current documentation is generated, and how that may need to change or not if we move to poetry This type of thing is something that should be thought about by the team and others that have a more complete picture of the way the build is working today and be able to understand what things may be impacted by the changes.

jake-aft commented 1 year ago

@ceteri and @Mec-iS is the poetry build/publish still something you want to have fopr the product? I am going to have some time to revisit this if it is still wanted?

ceteri commented 1 year ago

Thank you kindly @jake-aft ! I've got time cleared out, and I'll work through the build. It's a little tricky given how the notebooks interact with generated docs, so I'll need to guide it a bit through this (most excellent) transition :)

jake-aft commented 1 year ago

@ceteri let me know if you want me to merge the two as @Mec-iS suggested. Also happy to help out on the docs if I would be of use?

ceteri commented 1 year ago

It's probably better to start from a branch based on the most recent main here, then back in the changes needed for a poetry build?

Otherwise there may not be a good way around the merge conflicts.

jake-aft commented 1 year ago

@ceteri I'll get the latest main, and resubmit