Implement OpenScanHub initial reporting

lbarcziova commented 1 month ago

[ ] in fedmsg repo, process the messages about OSH task states (see https://github.com/openscanhub/openscanhub/pull/299 )
[ ] create new event class for that and implement parsing of these
[ ] create a model for a scan
[ ] store the task in the DB somewhere here and instead of successful status there, report in-progress
[ ] create a handler to report the results for the new event
- obtain the task in our DB, update the state and report it to user
- for now, just forward the URLs from the message (=> no additional processing)
- take into consideration #2515
- by default don't fail the CI, have a look also into action_required conclusion from here, whether that could be used, or use neutral/successful
- report neutral status for failed, cancelled or interrupted builds

Part of #2516

siteshwar commented 1 month ago

by default don't fail the CI, have a look also into action_required conclusion from here, whether that could be used, or use neutral/successful

report neutral status for failed, cancelled or interrupted builds

Please also see Code scanning results check failures.

siteshwar commented 1 month ago

by default don't fail the CI, have a look also into action_required conclusion from here, whether that could be used, or use neutral/successful

If you want to find out if new findings were detected, follow these steps:

Download the added.js file.
Run csgrep added.js and check if output is non-empty.
If output is non-empty, it means there were new findings by the analzyers. Set the CI to action required state.
Otherwise pass the CI.

EDIT: @kdudka Is there any other way to count number of findings in the reports?

kdudka commented 1 month ago

@siteshwar For simple scans, we can check whether the scan-results-summary.txt file is empty or not. Unfortunately, for differential scans, we do not provide such a file that would summarize the added.* results. We can still check whether added.err is empty or not.

siteshwar commented 1 month ago

And just to keep everybody in sync. The messages from OpenScanHub can be seen through:

fedora-messaging --conf /etc/fedora-messaging/fedora.toml consume --routing-key "org.fedoraproject.prod.openscanhub.task.*

kdudka commented 1 month ago

@siteshwar I can see that the URLs provided in Body: of messages about finished scans use the http:// scheme instead of https://. Consequently the scan results URLs, when used directly, return 302 Found instead of the real data.

kdudka commented 1 month ago

The problem may actually be more generic. I got an e-mail notification and the task URL also contained http:// URL. Something must be misconfigured in the Fedora deployment because the internal OSH instances use https:// URLs.

majamassarini commented 4 weeks ago

Hi! I want just to let you know I am starting working on this.

siteshwar commented 4 weeks ago

The problem may actually be more generic. I got an e-mail notification and the task URL also contained http:// URL. Something must be misconfigured in the Fedora deployment because the internal OSH instances use https:// URLs.

I would investigate it separately through https://github.com/openscanhub/fedora-infra/issues/73

siteshwar commented 2 weeks ago

There is a confusion about task state changes in OSH and how they should be reflected in the reporting. I would summarize it again for clarification:

When a task is submitted to OSH, but has not started, it is in FREE state. The status in Packit should say, it is waiting to be started.
When a worker picks up the task and starts running it, it moves into OPEN state. OSH would emit this message with task id in the body.

org.fedoraproject.prod.openscanhub.task.open

Packit reporting should say the task is running.

If the scan fails to finish successfully, there would be one of these messages with task id in the body:

org.fedoraproject.prod.openscanhub.task.cancel
org.fedoraproject.prod.openscanhub.task.fail
org.fedoraproject.prod.openscanhub.task.interrupt

Packit reporting should put the CI in neutral state.

If the scan finishes successfully, you would get org.fedoraproject.prod.openscanhub.task.finish with url to added and removed findings in JSON format along with task id. Follow the steps from previous comment to process these results.

siteshwar commented 2 weeks ago

@lachmanfrantisek @lbarcziova

I am following up from https://github.com/packit/packit-service/pull/2580#discussion_r1804350263

Please tell me exactly the message topics you would like to receive on Packit side and what should be the content of the message body.

lbarcziova commented 2 weeks ago

hi @siteshwar ! I talked about that with @majamassarini and am aligned with the way how she proposed it, so having 2 types of messages: one for start/in-progress task and one for finished, while there should be the state or result attribute describing the actual status.

siteshwar commented 2 weeks ago

https://github.com/openscanhub/openscanhub/pull/301

siteshwar commented 2 weeks ago

openscanhub/openscanhub#301

These changes are in production now. Example messages:

[INFO fedora_messaging.twisted.consumer] Successfully consumed message from topic org.fedoraproject.prod.openscanhub.task.started (message id f6561b12-a1d3-4567-9f72-2ef3b941210a)
[INFO fedora_messaging.twisted.consumer] Consuming message from topic org.fedoraproject.prod.openscanhub.task.started (message id d1a83e8e-c9d0-4a1f-adad-562bc38026a4)
Id: d1a83e8e-c9d0-4a1f-adad-562bc38026a4
Topic: org.fedoraproject.prod.openscanhub.task.started
Headers: {
    "fedora_messaging_schema": "base.message",
    "fedora_messaging_severity": 20,
    "priority": 0,
    "sent-at": "2024-10-18T15:44:43+00:00",
    "x-received-from": [
        {
            "cluster-name": "rabbit@rabbitmq02.iad2.fedoraproject.org",
            "exchange": "amq.topic",
            "redelivered": false,
            "uri": "amqps://rabbitmq01.iad2.fedoraproject.org/%2Fpubsub"
        }
    ]
}
Body: {
    "task_id": 18203
}
[INFO fedora_messaging.twisted.consumer] Successfully consumed message from topic org.fedoraproject.prod.openscanhub.task.started (message id d1a83e8e-c9d0-4a1f-adad-562bc38026a4)
[INFO fedora_messaging.twisted.consumer] Consuming message from topic org.fedoraproject.prod.openscanhub.task.finished (message id 29b7d83b-2a79-46ab-8b9a-5b4f36794daa)
Id: 29b7d83b-2a79-46ab-8b9a-5b4f36794daa
Topic: org.fedoraproject.prod.openscanhub.task.finished
Headers: {
    "fedora_messaging_schema": "base.message",
    "fedora_messaging_severity": 20,
    "priority": 0,
    "sent-at": "2024-10-18T15:45:01+00:00",
    "x-received-from": [
        {
            "cluster-name": "rabbit@rabbitmq02.iad2.fedoraproject.org",
            "exchange": "amq.topic",
            "redelivered": false,
            "uri": "amqps://rabbitmq01.iad2.fedoraproject.org/%2Fpubsub"
        }
    ]
}
Body: {
    "added.js": "http://openscanhub.fedoraproject.org/task/18198/log/added.js?format=raw",
    "fixed.js": "http://openscanhub.fedoraproject.org/task/18198/log/fixed.js?format=raw",
    "scan-results.js": "http://openscanhub.fedoraproject.org/task/18198/log/crun-1.17-1.20241018152736619857.pr1579.39.g30fda6b/scan-results.js?format=raw",
    "status": "success",
    "task_id": 18198
}

packit / packit-service

Implement OpenScanHub initial reporting #2543