green-coding-solutions / green-metrics-tool

Measure energy and carbon consumption of software
https://metrics.green-coding.io
GNU Affero General Public License v3.0
143 stars 19 forks source link

API endpoint /compare does not work anymore #737

Closed davidkopp closed 3 months ago

davidkopp commented 3 months ago

Example API request that fails: https://api.green-coding.io/v1/compare?ids=bf22a5c1-670b-4bd2-ba94-ad225cefe7c0,7e40ee3b-733e-4b66-aaba-e1e32a412a28

However, if only one run is provided to the same API endpoint, the request succeeds: https://api.green-coding.io/v1/compare?ids=7e40ee3b-733e-4b66-aaba-e1e32a412a28 https://api.green-coding.io/v1/compare?ids=bf22a5c1-670b-4bd2-ba94-ad225cefe7c0

I assume the PR https://github.com/green-coding-solutions/green-metrics-tool/pull/693 introduced the issue.

ArneTR commented 3 months ago

i will give it a look later today. ty

-------- Original Message -------- On 11 Apr 2024, 15:30, David wrote:

Example API request that fails: https://api.green-coding.io/v1/compare?ids=bf22a5c1-670b-4bd2-ba94-ad225cefe7c0,7e40ee3b-733e-4b66-aaba-e1e32a412a28

However, if only one run is provided to the same API endpoint, the request succeeds: https://api.green-coding.io/v1/compare?ids=7e40ee3b-733e-4b66-aaba-e1e32a412a28 https://api.green-coding.io/v1/compare?ids=bf22a5c1-670b-4bd2-ba94-ad225cefe7c0

I assume the PR #693 introduced the issue.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

ArneTR commented 3 months ago

Here comes the resolve:

The API link you sent actually contains two non-comparable runs. The have the same repo, but different commits and different usage scenarios. GMT has never supported this comparison so far, so I assume you had maybe a wrong URL flying around?

If not, please follow up!

But the reason why the server failed is: We are currently sending out error messages synchronous via e-mail. Our e-mail provider failed and there was no failover, so it blocked by sending the SMTP mail. I have disable e-mails for now and will now address a proper async error logging over the weekend

API calls and comparisons should now work fine again.

davidkopp commented 3 months ago

As always, thanks for the quick fix and the explanation!

Regarding the comparison of the two measurements: I created the URL today and I think based on the available data a comparison could make sense. In the comparisons I make between two different implementations of a system, the usage scenario always differs because of the different compose file. Sometimes also the commit hash differs when I made some changes between measurement runs. If understand it correctly, GMT doesn't support the comparison of different commits and different usage scenarios only to make sure that comparisons are always valid. Would it be possible to allow such comparisons if the user wants to have it? Maybe a warning would make more sense than a strict error.

ArneTR commented 3 months ago

Technically that comparison can be done by the GMT and I understand your case. If the files are not "actually" different it makes life a bit easier to not be policed by the GMT.

What I could see is a parameter appended to the API request that can be configured on the settings page that would allow "all" comparisons (https://metrics.green-coding.io/settings.html)

Sadly I cannot provide such a feature before the conference as we are quite packed atm.

If you need it for yourself what I can provide is a read-only user for our postgres DB. You could then configure it in your local instance of the GMT which would then allow for comparisons locally. The data would still come from our DB and measurements would also have been done on the cluster.

Let me know if that is helpful for you.

davidkopp commented 3 months ago

Sounds good. I have created another issue, so my feature request is not forgotten.

I would like to have a read-only user for the Postgres DB :) Thanks for the offer! If you want you can send me the credentials via email or LinkedIn. Maybe I will run two more measurements on the cluster, so I have the possibility to make a comparison also with the current live version.

davidkopp commented 3 months ago

Thanks for the credentials! Works great 😀