search

visual-layer / fastdup

fastdup is a powerful, free tool designed to rapidly generate valuable insights from image and video datasets. It helps enhance the quality of both images and labels, while significantly reducing data operation costs, all with unmatched scalability.
Other
1.6k stars 77 forks source link

Pinned versions of fastdup become unavailable #62

Closed jhendriksen-jh closed 1 year ago

jhendriksen-jh commented 1 year ago

Hello,

My team uses fastdup in one of our pipelines and we typically try to pin packages to avoid surprise updates when docker images are being built. We've been doing this for fastdup but have noticed that fairly frequently we'll encounter an error the pinned version can no longer be found and we have to pin to a newer version.

Is there a compelling reason for this? I can understand always wanting the "best" and newest version, but there's also been times when updating to the newest version has broken things - like v0.163 not being compatible with Ubuntu18.04. It would be easier to understand if the changes between these different version were documented in your PRs or releases.

Thanks for any explanation you can provide!

dbickson commented 1 year ago

Hi @jhendriksen-jh thanks for reaching out! The explanation is pretty mundane. As we have an underlying c++ engine, the pip size may reach 100 MB and we need around 10 version for the different OS and Python which takes around 1GB in total for a single release. Pypi's project quota at is only 10GB so we often get out of quota and need to delete older versions. We have a very fast development cycle so we may release a few versions in a week as we are adding a lot of functionality and improve performance. We are planning to release version 1 in a couple of months that will be significantly improved, and we are thinking about relaxing the license to a more common and familiar license. We will think how to pin a stable version, but we do encourage you to upgrade often, for versions below 1 the updates are usually backward compatible. In version 1 we may have some simplified interfaces.