Use uv in CI - Githubissues

datonic / datadex

📦 Serverless and local-first Open Data Platform

http://datadex.datonic.io

MIT License

240 stars 15 forks source link

Use uv in CI #67

Closed davidgasquez closed 1 month ago

thewtex commented 1 month ago

Worth consideration: pixi = cross-platform, easy-to-write Makefile + uv

davidgasquez commented 1 month ago

Thanks for the rec @thewtex! Not very familiar with pixi although I've heard good things. What are the main differences with uv + Makefile?

thewtex commented 1 month ago

Yes, a few examples of the niceties:

PyPI support but also conda-forge -> better scientific support, more reproducibility, more language support
The tasks (equivalent to Makefile targets) are specified in a simple standard, unixy format, and they work cross-platform (including Windows)
caching support for tasks
oh yeah, and emoji's :sake: :space_invader: :sparkling_heart:

CI in action!

Quick datadex start!

davidgasquez commented 1 month ago

Woah! Thanks for the details.

Caching and supporting Conda Forge sounds interesting. I think pixy does even use uv as their pipy dependency manager?

The main reason I'm hessitating is that Conda is not as popular outside the "science" ecosystems and uv seems enough (and growing in features) for most of the usecases.

By the way, lung-sarg looks awesome! :star_struck: I see you're already even using sensors and other advanced Dagster stuff. Is it ok if I add it under the list of projects using Datadex in the README?

davidgasquez commented 1 month ago

Closing this as I think I'm good with the CI using pip only and keeping things simple. :sweat_smile:

thewtex commented 1 month ago

Caching and supporting Conda Forge sounds interesting. I think pixy does even use uv as their pipy dependency manager?

Yes, they are working on it. They plans to still keep it a single, standalone rust binary.

The main reason I'm hessitating is that Conda is not as popular outside the "science" ecosystems and uv seems enough (and growing in features) for most of the usecases.

Ok. I get the PyPI-only and Python package-only approach. Science and reproducibility is my jam, and I have found that pixi makes sense for that use case.

By the way, lung-sarg looks awesome! 🤩 I see you're already even using sensors and other advanced Dagster stuff. Is it ok if I add it under the list of projects using Datadex in the README?

Yes, please do!

Thank you for your work on Datadex!

davidgasquez commented 1 month ago

Science and reproducibility is my jam, and I have found that pixi makes sense for that use case.

And should be everyone's jam too! :sweat_smile: I already had some issues in the past due to missmatching versions...

Seems uv recently added uv.lock files among other things. These should be similar to pixi.lock and should cover the reproducibility side.

Need to dig into that a bit more!