Reproducibility difficulty

[bpiegu]

Hello,

We were unable to run an analysis with the new version of the tool while it worked with a previous version. There are two ways to solve this problem: fix the code in a new version (which was done in our case) or fix the versions used. cf https://github.com/BiaPyX/BiaPy-GUI/issues/7

In fact, it is possible to fix a version of the biapy GUI, but it is not possible to do it for the biapy container. Indeed, you have chosen to always use the latest version of the biapy container.

The ideal would be to allow the user to set the version of the biapy container that the GUI will use. This would ensure the reproducibility of the results. Moreover, when publishing in peer-reviewed journals, it is requested to provide the version of the software used in order to be able to redo the analysis.

Best regards and thanks for your great tool!

Benoît

[danifranco]

Hi,

We are unable to implement an option to use a different container than the one the GUI expects due to internal checks that depend on BiaPy's core code. That being said, the reproducibility issues you mentioned are something we are keen to resolve.

So far, we've been making updates to have a stable version of BiaPy and its GUI. Is true that these updates have introduced several changes that impact backward compatibility. However, this is something we've been aware of, and we’ve already addressed it in BiaPy’s core code (we’ll be releasing an updated version in the next few days). While the GUI is a bit more complex, we’re working on it as well. Our goal is to implement an automatic translation of older configuration files (.yaml) to newer versions to ensure reproducibility. Currently, we’re considering configuration files from up to a year ago (January 2024). It would be helpful if you could share your configuration file, so we can make sure the changes we’re implementing also take your case into account, especially if it's older than January 2024.

Additionally, now that we’ve reached a stable version, we’ll create a version history of the GUI alongside its specific container. This list will be linked in the repository and on our website, ensuring 100% reproducibility going forward.

[danifranco]

We have release a new version of the GUI, v.1.1.3, with the backward compatibility mentioned and also an option to choose the containers that GUI version supports among the ones listed in https://biapyx.github.io/GUI_versions/. You will find a dropdown in the last window, e.g., "Run workflow".

Currently, this last version can only use the last BiaPy core version, v3.5.5, but in it may support future releases, depeding on how the core code changes. This new GUI is prepared to automatically recognize these new containers that can be used and will extend the container list in the mentioned dropdown. Hope this brings a bit more of flexibility and the option to use always the same container version ensuring reproducibility. We decided to implement this feature like that to ensure also all containers listed there are 100% compatible with the GUI version the end user is running.

[bpiegu]

Hi,

Thank you for your reply and for your responsiveness. I am very aware that biapy is a highly evolving code (which is really good). So, it is great news to learn that we will now be able to have version control. I think this will be useful for users who are concerned about being able to reproduce their analyses.

Benoît