nerc-project / operations

Issues related to the operation of the NERC OpenShift environment
2 stars 0 forks source link

Populate "AI projects" (at "/ai-project") page #796

Open dheerajodha opened 3 weeks ago

dheerajodha commented 3 weeks ago

Today the page related to AI projects doesn't contain any data and looks like this:

Image

Our goal is to populate it with expected data.

dheerajodha commented 3 weeks ago

@computate Can you please help me understand what expected data needs to be populated for this page?

computate commented 3 weeks ago

We want to import AI projects using a GPU from ACM Observability using a query like sum by (cluster, exported_namespace) (DCGM_FI_DEV_GPU_UTIL{cluster="nerc-ocp-prod"}). Look at AiClusterEnUSApiServiceImpl.java queryGpuDevicesTotal method for an example. Add similar _clusterName methods from GpuDevice.java into AiProject.java.

Image

computate commented 3 weeks ago

Here is an example of what the AiProject.java class could look like with some asynchronous queries for all the nodes in a project.

package org.mghpcc.aitelemetry.model.project;

import java.math.BigDecimal;
import java.util.List;
import java.util.Optional;
import org.apache.commons.lang3.StringUtils;
import org.computate.search.tool.SearchTool;
import org.computate.search.wrap.Wrap;
import org.computate.vertx.search.list.SearchList;
import org.mghpcc.aitelemetry.model.BaseModel;
import org.mghpcc.aitelemetry.model.gpudevice.GpuDevice;

import io.vertx.core.Promise;
import io.vertx.core.json.JsonArray;
import io.vertx.core.json.JsonObject;
import io.vertx.pgclient.data.Path;
import io.vertx.pgclient.data.Point;
import io.vertx.pgclient.data.Polygon;

/**
 * Order: 8
 * Model: true
 * 
 * Api: true
 * Page: true
 * SuperPage: BaseModelPage
 * Indexed: true
 * Description: A research project using AI and GPUs
 * 
 * ApiTag: AI project
 * ApiUri: /api/ai-project
 * 
 * ApiMethod:
 * Search:
 * GET:
 * PATCH:
 * POST:
 * DELETE:
 * PUTImport:
 * SearchPage:
 * Page: AiProjectPage
 * ApiUri: /ai-project
 * 
 * Role: SiteAdmin
 * 
 * AName: an AI project
 * PluralName: AI projects
 * Icon: <i class="fa-regular fa-school"></i>
 */
public class AiProject extends AiProjectGen<BaseModel> {

  /**
   * {@inheritDoc}
   * DocValues: true
   * Persist: true
   * DisplayName: cluster name
   * Description: The cluster name of this GPU device
   * HtmRow: 3
   * HtmCell: 1
   * HtmColumn: 1
   * HtmRowTitleOpen: GPU device details
   * Facet: true
   **/
  protected void _clusterName(Wrap<String> w) {
  }

  /**
   * {@inheritDoc}
   * DocValues: true
   * Persist: true
   * DisplayName: name
   * Description: The name of this AI project
   * HtmRow: 3
   * HtmCell: 1
   * HtmColumn: 1
   * HtmRowTitle: AI project details
   * Facet: true
   **/
  protected void _projectName(Wrap<String> w) {
  }

  /**
   * {@inheritDoc}
   * DocValues: true
   * Persist: true
   * DisplayName: description
   * Description: A description of this AI project
   * HtmRow: 3
   * HtmCell: 2
   * Facet: true
   * HtmColumn: 2
   **/
  protected void _description(Wrap<String> w) {
  }

  /**
   * {@inheritDoc}
   * DocValues: true
   * Persist: true
   * DisplayName: node names
   * Description: The node names used by this project.
   * HtmRow: 3
   * HtmCell: 2
   * HtmColumn: 2
   * Facet: true
   **/
  protected void _nodeNames(List<String> l) {
  }

  protected void _searchGpuDevices(Promise<SearchList<GpuDevice>> promise) {
    SearchList<GpuDevice> searchList = new SearchList<GpuDevice>();
    searchList.setStore(true);
    searchList.q("*:*");
    searchList.setC(GpuDevice.class);
    searchList.fq(String.format("clusterName_docvalues_string:%s", clusterName));
    searchList.promiseDeepForClass(siteRequest_).onSuccess(a -> {
      promise.complete(searchList);
    }).onFailure(ex -> {
      LOG.error("This did not work", ex);
      promise.fail(ex);
    });
    promise.complete();
  }

  /**
   * {@inheritDoc}
   * DocValues: true
   * Persist: true
   * DisplayName: GPU device numbers
   * Description: The numbers of GPU devices used by this project
   * HtmRow: 3
   * HtmCell: 3
   * HtmColumn: 3
   * Facet: true
   **/
  protected void _gpuDeviceNumbers(List<Integer> l) {
    for (GpuDevice device : searchGpuDevices.getList()) {
      l.add(device.getGpuDeviceNumber());
    }
  }
SaionarGr commented 3 weeks ago

Good 👍