Gittea

[romulasry]

For example: https://codeberg.org/freedroid/freedroid-src.git

See: https://release-monitoring.org/project/12106/

[erolkskn]

I'm working on it

[lmilbaum]

JFYI, Gitea is already supported by Renovate

[Zlopez]

It would be much easier to integrate renovate if we could just use API and ask for the latest version of the package in specific backend. In current case it's easier to write new backend that can work with Gitea.

[lmilbaum]

It would be much easier to integrate renovate if we could just use API and ask for the latest version of the package in specific backend. In current case it's easier to write new backend that can work with Gitea.

Would it be much easier to integrate renovate if we could just use API vs reading a file tracked in anitya's git repo?

[Zlopez]

Just now I'm not sure if it will be more work to integrate renovate in comparison to benefits we gain. Currently Anitya is reaching directly to where the project is hosted (either by API of the service or by parsing the HTML) with renovate we have another point of failure that could break Anitya.

Let's say that we will leverage renovate for some backends, right now if one service stops working the corresponding backend will stop working. If the renovate stops working all the backends using it stop working. It will introduce single point of failure and another layer of complexity to how Anitya works.

The only benefits I see now are that everything will be in one place and less room for user error setting the project up in release-monitoring incorrectly. As we already proved there isn't much added value in using renovate in case of version retrieval. It doesn't find versions that the Anitya doesn't knew about.

And about the API, just calling the renovate API from Anitya code will be much easier. For example if I could just do renovate.get_version(name_of_project) and get the latest version of the project (batch call would be nice as well) it would be much easier than downloading file from repo (you also need to push to repo as the new projects will be created in Anitya) and parsing it.

[lmilbaum]

Thank you for the active discussion. It is helpful for understanding your needs better.

Multiple arguments were mentioned. I'll try to provide my feedback to all of them one by one:

1. "Just now I'm not sure if it will be more work to integrate renovate in comparison to benefits we gain." Answer: A list of benefits I've quickly compiled:
   • Less lines of code to maintain => less bugs, less testing, reduced project complexity, maintenance time, etc..
   • Less infra resources/cycles to maintain
   • Fail & Recovery (reiterate) provided by Renovate
   • Logging provided by Renovate
   • "Everything will be in one place and less room for user error setting"
   • Anitya's dependencies are maintained with the same tool
   • Renovate has an active community which extends it in a fast pace. It probably supports data sources which are not yet supported by Anitya
2. "Currently Anitya is reaching directly to where the project is hosted (either by API of the service or by parsing the HTML) with renovate we have another point of failure that could break Anitya." Answer: Probably True
3. "Let's say that we will leverage renovate for some backends, right now if one service stops working the corresponding backend will stop working. If the renovate stops working all the backends using it stop working. It will introduce single point of failure and another layer of complexity to how Anitya works." Answer: Isn't Anitya a single point of failure for the projects which uses it? Renovate (high) availability is a must have requirement. Regarding Complexity, the answer could be yes and no. I don't think there is one clear answer to this one.
4. "The only benefits I see now are that everything will be in one place and less room for user error setting the project up in release-monitoring incorrectly. As we already proved there isn't much added value in using renovate in case of version retrieval. It doesn't find versions that the Anitya doesn't knew about." Answer: True. The PoC on 100 python packages, Renovate retrieved identical versions to the ones Anitya retrieved. But, this issue is an example of versions which are not yet supported by Anitya yet but are already supported by Renovate.
5. "And about the API, just calling the renovate API from Anitya code will be much easier. For example if I could just do renovate.get_version(name_of_project) and get the latest version of the project (batch call would be nice as well) it would be much easier than downloading file from repo (you also need to push to repo as the new projects will be created in Anitya) and parsing it." Answer: True. A file in Anitya's repo should be maintained if list of deps is updated. But, in my point of view it is not that complex to parse the file right after the Renovate PR is merged.

[Zlopez]

Thank you for the active discussion. It is helpful for understanding your needs better.

Multiple arguments were mentioned. I'll try to provide my feedback to all of them one by one:

1. "Just now I'm not sure if it will be more work to integrate renovate in comparison to benefits we gain." Answer: A list of benefits I've quickly compiled:

• Less lines of code to maintain => less bugs, less testing, reduced project complexity, maintenance time, etc..

I'm not sure if this is true, because we need another code for git interaction to be able to update, retrieve the files that renovate will update.

• Less infra resources/cycles to maintain
• Fail & Recovery (reiterate) provided by Renovate
• Logging provided by Renovate

Not sure if this wouldn't be more complex, it's usually easier if the logs are only in one place.

• "Everything will be in one place and less room for user error setting"
• Anitya's dependencies are maintained with the same tool
• Renovate has an active community which extends it in a fast pace. It probably supports data sources which are not yet supported by Anitya

This would be nice if we can just use it as another backend for Anitya

2. "Currently Anitya is reaching directly to where the project is hosted (either by API of the service or by parsing the HTML) with renovate we have another point of failure that could break Anitya." Answer: Probably True
3. "Let's say that we will leverage renovate for some backends, right now if one service stops working the corresponding backend will stop working. If the renovate stops working all the backends using it stop working. It will introduce single point of failure and another layer of complexity to how Anitya works." Answer: Isn't Anitya a single point of failure for the projects which uses it? Renovate (high) availability is a must have requirement. Regarding Complexity, the answer could be yes and no. I don't think there is one clear answer to this one.

It is, if the Anitya doesn't work the updates are not filled in bugzilla, but it wouldn't change in case of renovate. Without Anitya there will be no bug filled even if the renovate still work. But if Anitya still works and renovate isn't no update will be filled as well. Right now Anitya isn't depending on any service that much it couldn't work without it.

4. "The only benefits I see now are that everything will be in one place and less room for user error setting the project up in release-monitoring incorrectly. As we already proved there isn't much added value in using renovate in case of version retrieval. It doesn't find versions that the Anitya doesn't knew about." Answer: True. The PoC on 100 python packages, Renovate retrieved identical versions to the ones Anitya retrieved. But, this issue is an example of versions which are not yet supported by Anitya yet but are already supported by Renovate.

I'm just not sure if it's benefical to have code interacting with git repo just for one backend if we can leverage what we already have in place.

5. "And about the API, just calling the renovate API from Anitya code will be much easier. For example if I could just do renovate.get_version(name_of_project) and get the latest version of the project (batch call would be nice as well) it would be much easier than downloading file from repo (you also need to push to repo as the new projects will be created in Anitya) and parsing it." Answer: True. A file in Anitya's repo should be maintained if list of deps is updated. But, in my point of view it is not that complex to parse the file right after the Renovate PR is merged.

I would be rather if the file(s) will be in another repo to not mess with Anitya commit history. Also we should take in consideration that the renovate files needs to have specific versions of dependencies to check for newer ones and the question is how we can get this initial version. Should Anitya obtain it? Can renovate do it when the project is newly added? How does renovate actually recognizes the version is newer? We struggle with this in Anitya in some projects that are not using standard versions schemes.

As I see it we would need to do big changes to integrate renovate to Anitya (introduction of new code to interact with git repo containing renovate files, rewrite existing backends to use that new code, rewrite code for checking for new versions to leverage renovate, creating new projects should also now update the renovate files etc.) and we still can't use it for every backend currently supported by Anitya, so it will be just more code added specifically for renovate. Could you check which backends could be actually replaced by renovate in Anitya? If it would be a larger number of backends than it would make sense to use it. And is there any backend that the renovate supports and Anitya doesn't? Could you point me to renovate docs regarding Gittea?

[lmilbaum]

Thank you for the active discussion. It is helpful for understanding your needs better. Multiple arguments were mentioned. I'll try to provide my feedback to all of them one by one:

1. "Just now I'm not sure if it will be more work to integrate renovate in comparison to benefits we gain." Answer: A list of benefits I've quickly compiled:

• Less lines of code to maintain => less bugs, less testing, reduced project complexity, maintenance time, etc..

I'm not sure if this is true, because we need another code for git interaction to be able to update, retrieve the files that renovate will update.

True :-)

• Less infra resources/cycles to maintain
• Fail & Recovery (reiterate) provided by Renovate
• Logging provided by Renovate

Not sure if this wouldn't be more complex, it's usually easier if the logs are only in one place. Personally, I find multiple small logs easier for troubleshooting then one monolithic log. That is a personal preference. What I've tried to state is that Renovate is taking care of the workload it executes.

• "Everything will be in one place and less room for user error setting"
• Anitya's dependencies are maintained with the same tool
• Renovate has an active community which extends it in a fast pace. It probably supports data sources which are not yet supported by Anitya

This would be nice if we can just use it as another backend for Anitya

IMHO, it can

2. "Currently Anitya is reaching directly to where the project is hosted (either by API of the service or by parsing the HTML) with renovate we have another point of failure that could break Anitya." Answer: Probably True
3. "Let's say that we will leverage renovate for some backends, right now if one service stops working the corresponding backend will stop working. If the renovate stops working all the backends using it stop working. It will introduce single point of failure and another layer of complexity to how Anitya works." Answer: Isn't Anitya a single point of failure for the projects which uses it? Renovate (high) availability is a must have requirement. Regarding Complexity, the answer could be yes and no. I don't think there is one clear answer to this one.

It is, if the Anitya doesn't work the updates are not filled in bugzilla, but it wouldn't change in case of renovate. Without Anitya there will be no bug filled even if the renovate still work. But if Anitya still works and renovate isn't no update will be filled as well. Right now Anitya isn't depending on any service that much it couldn't work without it.

Valid concern. The beauty with Renovate is that even if there was a downtime, it catches up as soon as it comes up again.

4. "The only benefits I see now are that everything will be in one place and less room for user error setting the project up in release-monitoring incorrectly. As we already proved there isn't much added value in using renovate in case of version retrieval. It doesn't find versions that the Anitya doesn't knew about." Answer: True. The PoC on 100 python packages, Renovate retrieved identical versions to the ones Anitya retrieved. But, this issue is an example of versions which are not yet supported by Anitya yet but are already supported by Renovate.

I'm just not sure if it's benefical to have code interacting with git repo just for one backend if we can leverage what we already have in place.

Agreed. It can be used as a backend for all deps which are already supported.

5. "And about the API, just calling the renovate API from Anitya code will be much easier. For example if I could just do renovate.get_version(name_of_project) and get the latest version of the project (batch call would be nice as well) it would be much easier than downloading file from repo (you also need to push to repo as the new projects will be created in Anitya) and parsing it." Answer: True. A file in Anitya's repo should be maintained if list of deps is updated. But, in my point of view it is not that complex to parse the file right after the Renovate PR is merged.

I would be rather if the file(s) will be in another repo to not mess with Anitya commit history. Also we should take in consideration that the renovate files needs to have specific versions of dependencies to check for newer ones and the question is how we can get this initial version. Should Anitya obtain it? Can renovate do it when the project is newly added? How does renovate actually recognizes the version is newer? We struggle with this in Anitya in some projects that are not using standard versions schemes.

I think Renovate knows how to populate the current versions. Renovate uses multiple strategies to identify newer versions. One of them is regex.

As I see it we would need to do big changes to integrate renovate to Anitya (introduction of new code to interact with git repo containing renovate files, rewrite existing backends to use that new code, rewrite code for checking for new versions to leverage renovate, creating new projects should also now update the renovate files etc.) and we still can't use it for every backend currently supported by Anitya, so it will be just more code added specifically for renovate. Could you check which backends could be actually replaced by renovate in Anitya? If it would be a larger number of backends than it would make sense to use it. And is there any backend that the renovate supports and Anitya doesn't? Could you point me to renovate docs regarding Gittea?

Supported backends list (managers in renovate terminology) - https://docs.renovatebot.com/modules/manager/ Gitea short README - https://github.com/renovatebot/renovate/blob/3f1d967b9926870d819f287e637d92ba97646de2/lib/modules/datasource/gitea-releases/readme.md

[Zlopez]

Supported backends list (managers in renovate terminology) - https://docs.renovatebot.com/modules/manager/ Gitea short README - https://github.com/renovatebot/renovate/blob/3f1d967b9926870d819f287e637d92ba97646de2/lib/modules/datasource/gitea-releases/readme.md

Looking at the manager list and the issue with that is that they are using them based on the method for dependency management, so it's hard to match them with Anitya backends. Are we sure if we have project with PyPi backend in release-monitoring.org it's checking PyPi and not some other source? For example with gittea it doesn't seem to be manager just source to check the project in, but which manager we should use in this case?

The Anitya has backends based on the place where the project is hosted. It looks like to really use renovate we would need to change how the Anitya even works with projects and how it differentiate between them.

I need to think about that. Maybe using it as a backend on it's own will work. It would be nice if renovate could solve the rate-limiting issue we have on GitHub backend for example, but it's hard to map GitHub repositories to something the renovate actually understands as dependency to check.

[lmilbaum]

The place where the project is hosted is defined platform in Renovate's terminology. https://docs.renovatebot.com/modules/platform/

[lmilbaum]

I've created a small PoC to show how this issue is "solved" by Renovate. For the sake of the PoC, tags were used as the versioning method because there are multiple tags in the repo. Releases can also be used.

https://github.com/platform-engineering-org/renvoate-gitea-test

Dependency Dashboard PR

[Zlopez]

So in this case it seems that the configuration of renovate could be really complicated. As I can see it you need to define registryUrlTemplate for each new instance of gitea being added in Anitya. So this needs another code in Anitya for updating the renovate config each time this happens or some way to notify admin that there is a new project on gitea instance we don't know yet (and the admin needs to update the configuration). Same would be for GitLab or other services that allow their own hosting.

Thanks for the PoC for Gittea at least I now know what it will look like.

[lmilbaum]

So in this case it seems that the configuration of renovate could be really complicated. As I can see it you need to define registryUrlTemplate for each new instance of gitea being added in Anitya. So this needs another code in Anitya for updating the renovate config each time this happens or some way to notify admin that there is a new project on gitea instance we don't know yet (and the admin needs to update the configuration). Same would be for GitLab or other services that allow their own hosting.

Thanks for the PoC for Gittea at least I now know what it will look like.

True