Versioning

[ismael-mendoza]

Hi all, I mentioned this to @herjy earlier in a BlendingToolKit meeting. It would be really useful for us to have the latest version of scarlet tagged and pip installable in order to integrate scarlet more easily into BTK. Is this doable or is there a reason you are waiting on releasing a new version? Thanks in advance!

[aboucaud]

+1 for an updated tagged version of scarlet

This should make the CI more stable indeed and avoid issues like what happened with #265.

[pmelchior]

Adding a tag isn't a problem, ensuring that we set and use them consistently everywhere is. We need to overhaul our build system anyway by moving it entirely to github. Adding tags is tied up with that.

[ismael-mendoza]

Thanks @pmelchior - could you elaborate a bit more on the exact issue and is there any way we can help? Is it related to the fact that Scarlet is now part of the DM Pipeline (which has something more like weekly updates)?

A related question, is Scarlet Lite less likely to get updated frequently? or are both Scarlet and Lite updated on a regular basis and in development?

[pmelchior]

One issue is keeping lite stable for production while scarlet main is being actively developed (and moved to jax). That could be done in a number of ways. One would be to fork the code base. @fred3m should chime in.

The other issue (and the one I was thinking about) is that we're building tests and docs with travis ci, which keeps track of branches and such. We need to move this to github actions, which will break some aspects, such as how we store test rests and keep track of tags or branches.

[fred3m]

At this point scarlet lite is under a bit of flux as well. The science pipelines now have a fork of scarlet, so currently the lite module in scarlet is only being updated at https://github.com/lsst/scarlet, with changes no longer pushed to this repo. The longer term plan is to create a new scarlet-lite package that only contains the relevant modules from scarlet main. Since that is part of the science pipelines, scarlet lite is tagged weekly, along with the rest of the DM stack, and will have a new version once a year along with the rest of the science pipelines (however we do not expect it to depend on the rest of the DM stack, so it will still be a stand alone package).

As far as development, I am still actively developing it and frequently making improvements to help improve initialization and improve deblending in edge cases. So you can expect it to continue to be modified for at least the next few years.

[aboucaud]

Thank you @pmelchior and @fred3m for the update.

In our case, we are using scarlet as a dependency in BlendingToolKit. Would you recommend that we start switching to lsst/scarlet while waiting for the scarlet-lite repo ?

[pmelchior]

@aboucaud this would make sense from my perspective. Over the next month I anticipate changes here that will not be backwards compatible.

[thuiop]

We discussed this during our last meeting on Friday ; something that could be helpful would be tagging a commit for which the BTK version works, so that we can easily direct the users to a specific version without having to keep up SCARLET's development cycle. To that end, would it be possible to tag the last commit from the master bank (7eb3fe0) with something like "btkv1" ?

[aboucaud]

To add a precision on top of Thomas' comment, we had two things in mind at the BTK discussion

1. as this thread indicates, we would like to make sure that the CI passes no matter the upcoming developments of BTK dependancies. Currently we cannot pin a recent version for scarlet. This is why we thought of something lighter than versioning that would allow us to point to a particular version of scarlet
   => create a git tag on a commit that works with BTK v1.0.0
2. we are working on the BTK software paper and to address the question of reproducibility, we would like to ensure that the code examples shown in the paper will work in the future.
   => a tag name such as btk-paper would be quite explicit

The advantage of tags (unless you have specific hooks in your Travis CI) is that they are invisible and inoffensive, but do the job that we hopped for.

Please let us know what you think @pmelchior.

[pmelchior]

I can certainly tag a commit. Let me know which one you want.

[ismael-mendoza]

Thanks @pmelchior , it would be 7eb3fe0 the one @thuiop referenced above

[pmelchior]

done: https://github.com/pmelchior/scarlet/releases/tag/btk-v1