[Feature Request]: integrate `setuptools_scm` for better versioning from Git

[jamesbraza]

Feature Description

Currently, there's two versioning systems:

1. Git repo tags (lagging right now at v0.8.42)
2. Manual versioning by editing pyproject.toml

Lets sync these two by using setuptools_scm :rocket

Reason

I like to use Git tags in my links, to ensure they always work (whereas main branch-based links may not always work).

It would be nice to keep these in sync

Value of Feature

You use GitHub releases to publish new versions, instead of manually changing the version in pyproject.toml. It's more automated and seamless. Also, no need to manually create CHANGELOG, it's just done with GitHub releases.

[logan-markewich]

@jamesbraza A key point of the current changelog -- it's written by humans, for humans. I would rather not automate this step :) https://keepachangelog.com/en/1.1.0/

Also yea, the GitHub tags are lagging, need to figure that out lol

But the current tag system works well enough internally

[dosubot[bot]]

🤖

Thank you for your feature request and for providing a detailed explanation of the benefits of integrating setuptools_scm for versioning. I agree that it would be beneficial to have a more automated and seamless process for versioning and releasing new versions of LlamaIndex.

Currently, as you've pointed out, the versioning is done manually by editing the version field in the pyproject.toml file. This process can be prone to errors and inconsistencies, especially when it comes to syncing with Git repo tags.

Integrating setuptools_scm would indeed help to automate this process and ensure that the versioning is consistent across the Git repo tags and the pyproject.toml file. It would also automate the creation of the CHANGELOG, which is currently done manually.

I will start working on integrating setuptools_scm into the LlamaIndex codebase. I will create a new branch for this feature and once it's ready, I will submit a pull request for review. Once the pull request is approved and merged, the new versioning system will be in place.

I will keep you updated on the progress of this feature request. Thank you again for your valuable input. Your feedback helps us to improve LlamaIndex and make it more user-friendly and efficient.

Sources

• llama_index/VERSION
• pyproject.toml

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jamesbraza]

I think a CHANGELOG in the form of GitHub releases is still human readable. Why do you think something like https://github.com/astral-sh/ruff/releases isn't human readable? Imo that's more human friendly than the current CHANGELOG.md, because:

• It autolinks GitHub PRs
• It's standard across all GitHub

I think it'd would be excellent to remove the per-PR manual CHANGELOG creation step, in favor of well-worded PR descriptions. The biggest area of rework when contributing to LlamaIndex is having to deal with CHANGELOG conflicts each rebase.

Here's a nice workflow:

1. GitHub release: hit "generate release notes" button
   • This removes manual CHANGELOG per-PR step
   • A human can adjust as needed then
2. Release triggers a GitHub action that copies the release notes to CHANGELOG.md

[jamesbraza]

And most importantly you can do emoji reactions to the releases. Check how many reactions Hugo gets each release: https://github.com/gohugoio/hugo/releases

[david20571015]

Maybe poetry-dynamic-versioning is a good choice since we are using poetry?

[jamesbraza]

Looks like https://github.com/mtkennerly/poetry-dynamic-versioning is somewhat equivalent to setuptools_scm, and is actively maintained, so it's also a viable option! 👍

[dosubot[bot]]

Hi, @jamesbraza,

I'm helping the LlamaIndex team manage their backlog and am marking this issue as stale. The issue you opened requests integrating setuptools_scm to automate versioning and streamline the release process in the LlamaIndex repository. There's been a discussion about the benefits of automating versioning, potential alternatives like poetry-dynamic-versioning, and the impact on the contribution workflow. The maintainers have expressed interest in exploring the options and improving the versioning process.

Could you please confirm if this issue is still relevant to the latest version of the LlamaIndex repository? If it is, please let the LlamaIndex team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[jamesbraza]

Personally, I think it would be nice to not have to manually change pyproject.toml every release, so @dosubot let's keep this open. Up to the LlamaIndex team though

[logan-markewich]

version and it's associated file has been deleted, so no issue syncing these files

Mayne more annoying though now is trying to manage releases for 400+ packages... working on a github action to help manage this soon