A simple sample to get job logs

masalinas commented 3 months ago

Describe the bug

I need a simple sample where:

Execute a job (the job can use a minute to be executed aprox)
Get all logs from the job every one second for example
Remove the job executed correctly

Fabric8 Kubernetes Client version

SNAPSHOT

Steps to reproduce

I need a simple sample where:

Execute a job (the job can use a minute to be executed aprox)
Get all logs from the job every one second for example
Remove the job executed correctly

Expected behavior

Get the jobs in batch mode every 1 second for example

Runtime

minikube

Kubernetes API Server version

1.25.3@latest

Environment

Linux

Fabric8 Kubernetes Client Logs

No response

Additional context

No response

rohanKanojia commented 3 months ago

@masalinas : Do you want to schedule your job after a fixed interval? If yes, Do you think CronJob resource would be better suited for this use case?

masalinas commented 3 months ago

My use case in my app is:

From frontend a user select a job from a list and send a requets with some variables to backend to be executed.
A backend microservice get the request and send a command to kubernetes to execute this job with the environment variables send in the request.
During the job execution the microservice must recover any log generated from the job and send to a topic in a websocket server, where the frontend user is subscribed.
When the jobs finalize, the microserive or kubernetes must remove it.

Right now, create the job and watch it to recover the state of the job and remove it when it finalizes is coded by me correctly (you can see in the code pasted), but recover the logs generated by the job during execution using the watchLog method I don't know how to use it in my use case, this is the reason to send this task, do you know how use correctly the watchLog to recover the logs generated by a job during the execution to be send in realtime to any frontend.

This is my sample code realted to my US:

package es.uniovi.edv.avispe.poc_kubernetes_cli;

import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.io.Resource;

import io.fabric8.kubernetes.api.model.batch.v1.Job;
import io.fabric8.kubernetes.api.model.batch.v1.JobBuilder;
import io.fabric8.kubernetes.client.ConfigBuilder;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.fabric8.kubernetes.client.KubernetesClientBuilder;
import io.fabric8.kubernetes.client.Watch;
import io.fabric8.kubernetes.client.Watcher;
import io.fabric8.kubernetes.client.WatcherException;

import lombok.extern.slf4j.Slf4j;

@Slf4j
@SpringBootApplication
public class PocKubernetesCliApplication implements CommandLineRunner {
    @Value("classpath:welcome-deployment.yaml")
    Resource resourceFile;

    public static void main(String[] args) {
        SpringApplication.run(PocKubernetesCliApplication.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        final Job job = new JobBuilder()
                  .withApiVersion("batch/v1")
                    .withNewMetadata()
                        .withName("sample-job")
                    .endMetadata()
                    .withNewSpec()
                        .withTtlSecondsAfterFinished(5) // automatic deleted after 5 seconds 
                        .withNewTemplate()
                            .withNewSpec()                              
                                .addNewContainer()
                                    .withName("sample-container")
                                    .withImage("busybox")
                                    .withArgs("/bin/sh", "-c", "for i in $(seq 10); do echo \"Welcome $i times\"; sleep 1; done")
                                .endContainer()
                                .withRestartPolicy("Never")
                            .endSpec()
                        .endTemplate()
                    .endSpec()
                  .build();

        // get logs and remove after finalized
        final CountDownLatch isWatchClosed = new CountDownLatch(1);  
        try (KubernetesClient client = new KubernetesClientBuilder()
                .withConfig(new ConfigBuilder()
                    .withMasterUrl("https://172.23.0.2:8443")
                    .withOauthToken("sha256~secret")
                    .withNamespace("default")
                    .withCaCertFile("/home/miguel/.minikube/ca.crt")
                    .withClientCertFile("/home/miguel/.minikube/profiles/minikube/client.crt")
                    .withClientKeyFile("/home/miguel/.minikube/profiles/minikube/client.key")
                    .withClientKeyAlgo("RSA")
                    .build())
                .build()) {
             // execute the job
            client.batch().v1().jobs()
                .inNamespace("default")         
                .resource(job)
            .create();      

            // monitorize the job logs and remove after finalization
            Watch watch = client.batch().v1().jobs()
                .inNamespace("default")
                .withName("sample-job")
                .watch(new Watcher<>() {
                    @Override
                    public void eventReceived(Action action, Job resource) {
                        switch (action.name()) {
                            case "ADDED":
                                log.info("{}/{} got added", resource.getMetadata().getNamespace(), resource.getMetadata().getName());                                           

                                break;
                            case "DELETED":
                                log.info("{}/{} got deleted", resource.getMetadata().getNamespace(), resource.getMetadata().getName());

                                break;
                            case "MODIFIED":
                                log.info("{}/{} got modified", resource.getMetadata().getNamespace(), resource.getMetadata().getName());

                                log.info(client.batch().v1().jobs().inNamespace("default").withName("sample-job").getLog());

                                isWatchClosed.countDown();

                                break;
                            default:
                                log.error("Unrecognized event: {}", action.name());
                        }
                    }

                    @Override
                    public void onClose() {
                        log.info("Watch closed");
                        isWatchClosed.countDown();
                    }

                    @Override
                    public void onClose(WatcherException cause) {
                        log.info("Watched closed due to exception ", cause);
                        isWatchClosed.countDown();
                    }                                                         
            });

            // Wait till watch gets closed
            boolean isTerminatedSuccessfully = isWatchClosed.await(1, TimeUnit.MINUTES);

            if (!isTerminatedSuccessfully) {
                log.error("Time out");
            }

            watch.close();

            client.batch().v1().jobs().inNamespace("default").resource(job).delete();
        } catch (InterruptedException interruptedException) {
            log.info( "Thread Interrupted!");
            Thread.currentThread().interrupt();
        }       
    }
}

Thanks

rohanKanojia commented 3 months ago

create the job and watch it to recover the state of the job and remove it when it finalizes is coded by me correctly

@masalinas : Looking at the code, it doesn't look that you're watching Job logs, but the Job resource itself. You need to check for the Job status to verify whether it's finished or not, then invoke getLog() to get logs of the job.

Here is an example of watching Job logs directly using watchLog https://github.com/fabric8io/kubernetes-client/blob/7b728e752d3279d392383ee3bd437f6435ab7fe5/kubernetes-examples/src/main/java/io/fabric8/kubernetes/examples/kubectl/equivalents/PodLogsFollowEquivalent.java#L27

masalinas commented 3 months ago

Yes I understand you, I only watched the state of the job not the logs, and your sample code is the default one included in the library, but I would like to recover the logs during execution not only at the end, is it possible?. Do you have any sample or pattern to show this use case?

Regards

rohanKanojia commented 3 months ago

@masalinas : It should be possible to get logs during execution using watchLog(OutputStream). Instead of System.out you can pass ByteArrayOutputStream and then read that later when you need it.

rohanKanojia commented 3 months ago

@masalinas : Here is an example of watching job logs in real time:

https://github.com/fabric8io/kubernetes-client/blob/2f313928e46b8361494bd023cbef2d1a29833f0d/kubernetes-itests/src/test/java/io/fabric8/kubernetes/JobIT.java#L47-L51

fabric8io / kubernetes-client