subscribe/listen interface documentation

mwasilew commented 6 months ago

The docs (https://kernelci.org/docs/api/#publisher--subscriber-interface) say that Any client code can subscribe to receive events with an API token. I tried calling subscribe interface, but I didn't manage to receive any events. Could you please provide an example on how to use the interface? I would like to schedule tests to our internal devices. These tests are usually hardware specific and require special setup which I believe doesn't belong to kernelci-core. So the events I'm interested in are build completed for selected kernel trees.

nuclearcat commented 6 months ago

Hi! I built simple example, but i did shortcut and didn't used cloudevents to parse events properly, still it should work: https://gist.github.com/nuclearcat/aec542be9b2f0cb7a7ee3b164d664877 Can you try it please? The only issue you might need to wait for a while for a kbuild event, or ping me in IRC, i can trigger manual staging builds

JenySadadia commented 6 months ago

Hello,

Here is a guide to use PubSub API endpoints directly: https://kernelci.org/docs/api/api-details/#pubsub-and-cloudevent
Pipeline services use API Helper functions from kernelci-core to subscribe and receive node events: Subscribe: https://github.com/kernelci/kernelci-pipeline/blob/main/src/tarball.py#L170 Receive node data from event: https://github.com/kernelci/kernelci-pipeline/blob/main/src/tarball.py#L186
Alternatively, you can also use KCI tools: kci event commands: https://github.com/kernelci/kernelci-core/blob/main/kernelci/cli/event.py

Could you please provide me which is your preferred way of receiving events? I assume you want to listen to events outside of pipeline environment.

JenySadadia commented 6 months ago

You may also want to take a look at a service polling for build events: https://github.com/kernelci/kernelci-pipeline/pull/594

mwasilew commented 6 months ago

@JenySadadia I saw the docs, but they're a bit confusing. For example, what is the channel I should subscribe to? The docs say "channel":"abc", but I assume this is just an example. Is there a list of available channels?

I'm not sure whether I will try to reuse the kernelci-pipeline or write something else. The main idea is to receive events from the API and not depend on the pipeline instance you're running.

@nuclearcat I'll try it. Thank you!

JenySadadia commented 6 months ago

@JenySadadia I saw the docs, but they're a bit confusing. For example, what is the channel I should subscribe to? The docs say "channel":"abc", but I assume this is just an example. Is there a list of available channels?

Atm API is publishing events on 2 channels i.e. node and user_group. You may not be interested in user group channel as it's for user management-related events. You should be listening to node channel to receive all node creation and update events.

I'm not sure whether I will try to reuse the kernelci-pipeline or write something else. The main idea is to receive events from the API and not depend on the pipeline instance you're running.

Then I'd recommend using API endpoints directly or kci event CLI tool for this. The latter would be more convenient if you've already setup kci tool from the guide https://kernelci.org/docs/api/early-access/#setting-up-the-kci-tool.

JenySadadia commented 6 months ago

If you want to create a new channel for instance test and send/receive data on that one, here is the kci commands:

Subscribe to `test` channel:
$ ./kci event subscribe test
28629

Poll for events:
$ ./kci event receive 28629 

Run the below command on a different terminal to send JSON data on `test` channel:
$ ./kci event send node-data.json test

$ ./kci event receive 28629
{"id":"66552254324bb8ea21413698","kind":"test","name":"emerg","path":["checkout","kbuild-gcc-10-x86","baseline-x86-intel","dmesg","emerg"],"group":"baseline-x86-intel","parent":"66552254324bb8ea21413697","state":"done","result":"pass","artifacts":null,"data":{"error_code":null,"error_msg":null,"test_source":null,"test_revision":null,"platform":"dell-latitude-5300-8145U-arcada","device":null,"runtime":"lava-collabora","job_id":null,"job_context":null,"regression":null,"kernel_revision":{"tree":"stable-rc","url":"https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git","branch":"linux-6.1.y","commit":"094034c9e476724e93cf4db9afc5feb7a6e9bcd8","describe":"v6.1.92-275-g094034c9e4767","version":{"version":6,"patchlevel":1,"sublevel":92,"extra":"-275-g094034c9e4767","name":null},"patchset":null},"arch":"x86_64","defconfig":"x86_64_defconfig","config_full":"x86_64_defconfig+x86-board","compiler":"gcc-10","kernel_type":"bzimage"},"debug":null,"jobfilter":null,"created":"2024-05-28T00:16:20.800000","updated":"2024-05-28T00:16:20.800000","timeout":"2024-05-28T06:16:20.800000","holdoff":null,"owner":"staging.kernelci.org","submitter":"9790c38bd29c1375fc96ed92b9b4bfab","user_groups":[]}

I hope this helps.

mwasilew commented 6 months ago

@JenySadadia thank you, much appreciated!

JenySadadia commented 5 months ago

Should I close this issue @mwasilew ?

mwasilew commented 5 months ago

Sorry, I forgot to comment. Please close. I think I got all the details I needed.

kernelci / kernelci-api

subscribe/listen interface documentation #537