search

bazelbuild / rules_jsonnet

Jsonnet rules for Bazel
https://bazelbuild.github.io/rules_jsonnet/
Apache License 2.0
68 stars 73 forks source link

Error when `yaml_stream = True,` #114

Closed hongkailiu closed 5 years ago

hongkailiu commented 5 years ago 
$ cat prow/monitoring/mixins/prometheus/BUILD.bazel 
load("@io_bazel_rules_jsonnet//jsonnet:jsonnet.bzl", "jsonnet_library", "jsonnet_to_json")

jsonnet_library(
    name = "prom_lib",
    srcs = [
        "prometheus.libsonnet",
        "ci_absent_alerts.libsonnet",
        "prow_monitoring_absent_alerts.libsonnet",
    ],
)

jsonnet_to_json(
    name = "prow_prometheusrule",
    src = "prow_prometheusrule.jsonnet",
    outs = ["prow_prometheusrule.yaml"],
    deps = [":prom_lib"],
    #without this line, json output works fine.
    yaml_stream = True,
    visibility = ["//visibility:public"],
)

$ bazel build //prow/monitoring/mixins/prometheus:prow_prometheusrule --verbose_failures
DEBUG: /home/hongkliu/.cache/bazel/_bazel_hongkliu/ef5e8e918e6947b01dd429bce37b0e43/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:256:5: rbe_default is using checked-in configs 'struct(config_repos = [], create_cc_configs = True, create_java_configs = True, env = {"ABI_LIBC_VERSION": "glibc_2.19", "ABI_VERSION": "clang", "BAZEL_COMPILER": "clang", "BAZEL_HOST_SYSTEM": "i686-unknown-linux-gnu", "BAZEL_TARGET_CPU": "k8", "BAZEL_TARGET_LIBC": "glibc_2.19", "BAZEL_TARGET_SYSTEM": "x86_64-unknown-linux-gnu", "CC": "clang", "CC_TOOLCHAIN_NAME": "linux_gnu_x86"}, java_home = "/usr/lib/jvm/java-8-openjdk-amd64", name = "9.0.0")'
INFO: Analyzed target //prow/monitoring/mixins/prometheus:prow_prometheusrule (1 packages loaded, 6 targets configured).
INFO: Found 1 target...
ERROR: /home/hongkliu/go/src/k8s.io/test-infra/prow/monitoring/mixins/prometheus/BUILD.bazel:12:1: error executing shell command: '/bin/bash -c set -e; bazel-out/host/bin/external/jsonnet/cmd/jsonnet -J . -J bazel-out/k8-fastbuild/bin -J bazel-out/k8-fastbuild/bin -y prow/monitoring/mixins/prometheus/prow_prometheusrule.jsonne...' failed (Exit 1) bash failed: error executing command 
  (cd /home/hongkliu/.cache/bazel/_bazel_hongkliu/ef5e8e918e6947b01dd429bce37b0e43/sandbox/linux-sandbox/18/execroot/io_k8s_test_infra && \
  exec env - \
    PATH=/home/hongkliu/tool/node/bin:/home/hongkliu/go/bin:/home/hongkliu/go/src/k8s.io/test-infra/node_modules/.bin:/home/hongkliu/tool/go1.12.5/bin:/home/hongkliu/tool/node/bin:/home/hongkliu/go/bin:/home/hongkliu/.local/bin:/home/hongkliu/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/hongkliu/tool/go/bin:/home/hongkliu/go/bin:/home/hongkliu/tool/go/bin \
  /bin/bash -c 'set -e; bazel-out/host/bin/external/jsonnet/cmd/jsonnet -J . -J bazel-out/k8-fastbuild/bin -J bazel-out/k8-fastbuild/bin -y prow/monitoring/mixins/prometheus/prow_prometheusrule.jsonnet -o bazel-out/k8-fastbuild/bin/prow/monitoring/mixins/prometheus/prow_prometheusrule.yaml')
Execution platform: @bazel_tools//platforms:host_platform

Use --sandbox_debug to see verbose messages from the sandbox: bash failed: error executing command 
  (cd /home/hongkliu/.cache/bazel/_bazel_hongkliu/ef5e8e918e6947b01dd429bce37b0e43/sandbox/linux-sandbox/18/execroot/io_k8s_test_infra && \
  exec env - \
    PATH=/home/hongkliu/tool/node/bin:/home/hongkliu/go/bin:/home/hongkliu/go/src/k8s.io/test-infra/node_modules/.bin:/home/hongkliu/tool/go1.12.5/bin:/home/hongkliu/tool/node/bin:/home/hongkliu/go/bin:/home/hongkliu/.local/bin:/home/hongkliu/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/hongkliu/tool/go/bin:/home/hongkliu/go/bin:/home/hongkliu/tool/go/bin \
  /bin/bash -c 'set -e; bazel-out/host/bin/external/jsonnet/cmd/jsonnet -J . -J bazel-out/k8-fastbuild/bin -J bazel-out/k8-fastbuild/bin -y prow/monitoring/mixins/prometheus/prow_prometheusrule.jsonnet -o bazel-out/k8-fastbuild/bin/prow/monitoring/mixins/prometheus/prow_prometheusrule.yaml')
Execution platform: @bazel_tools//platforms:host_platform

Use --sandbox_debug to see verbose messages from the sandbox
RUNTIME ERROR: stream mode: top-level object was a object, should be an array whose elements hold the JSON for each document in the stream.
        During manifestation    
Target //prow/monitoring/mixins/prometheus:prow_prometheusrule failed to build
INFO: Elapsed time: 0.279s, Critical Path: 0.08s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

What did I miss? Thanks.

seh commented 5 years ago

Per the error message, std.manifestYamlStream requires an array as input:

stream mode: top-level object was a object, should be an array whose elements hold the JSON for each document in the stream.

Can you show us the outermost layer of your _prowprometheusrule.jsonnet file?

hongkailiu commented 5 years ago

@seh Thanks for your reply.

https://github.com/kubernetes/test-infra/blob/d5f2a5fe3938b54714cab7059e586f4e9bf6f7de/prow/monitoring/mixins/prometheus/prow_prometheusrule.jsonnet

seh commented 5 years ago

I see; it's a top-level JSON object. If you wrap that object in an array, this should work for you.

local alerts = (import 'prometheus.libsonnet').prometheusAlerts;

[
  {
    // ...
  },
]

That will get you a stream of YAML documents. In your case, you'll wind up with just one document.

hongkailiu commented 5 years ago

May I ask why it requires an array?

Tried with wrapping it into array, but the output is not yaml

$ bazel build //prow/cluster/monitoring/mixins/prometheus:prow_prometheusrule
DEBUG: /home/hongkliu/.cache/bazel/_bazel_hongkliu/ef5e8e918e6947b01dd429bce37b0e43/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:256:5: rbe_default is using checked-in configs 'struct(config_repos = [], create_cc_configs = True, create_java_configs = True, env = {"ABI_LIBC_VERSION": "glibc_2.19", "ABI_VERSION": "clang", "BAZEL_COMPILER": "clang", "BAZEL_HOST_SYSTEM": "i686-unknown-linux-gnu", "BAZEL_TARGET_CPU": "k8", "BAZEL_TARGET_LIBC": "glibc_2.19", "BAZEL_TARGET_SYSTEM": "x86_64-unknown-linux-gnu", "CC": "clang", "CC_TOOLCHAIN_NAME": "linux_gnu_x86"}, java_home = "/usr/lib/jvm/java-8-openjdk-amd64", name = "9.0.0")'
INFO: Analyzed target //prow/cluster/monitoring/mixins/prometheus:prow_prometheusrule (29 packages loaded, 230 targets configured).
INFO: Found 1 target...
Target //prow/cluster/monitoring/mixins/prometheus:prow_prometheusrule up-to-date:
  bazel-bin/prow/cluster/monitoring/mixins/prometheus/prow_prometheusrule.yaml
INFO: Elapsed time: 28.199s, Critical Path: 26.65s
INFO: 15 processes: 15 linux-sandbox.
INFO: Build completed successfully, 19 total actions

$ cat bazel-bin/prow/cluster/monitoring/mixins/prometheus/prow_prometheusrule.yaml
---
{
   "apiVersion": "monitoring.coreos.com/v1",
   "kind": "PrometheusRule",
   "metadata": {
      "labels": {
         "prometheus": "prow",
         "role": "alert-rules"
      },
      "name": "prometheus-prow-rules",
      "namespace": "prow-monitoring"
   },
   "spec": {
      "groups": [
         {
            "name": "ci-absent",
            "rules": [
               {
                  "alert": "deckDown",
                  "annotations": {
                     "message": "The service deck has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"deck\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               },
               {
                  "alert": "ghproxyDown",
                  "annotations": {
                     "message": "The service ghproxy has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"ghproxy\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               },
               {
                  "alert": "hookDown",
                  "annotations": {
                     "message": "The service hook has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"hook\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               },
               {
                  "alert": "plankDown",
                  "annotations": {
                     "message": "The service plank has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"plank\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               },
               {
                  "alert": "sinkerDown",
                  "annotations": {
                     "message": "The service sinker has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"sinker\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               },
               {
                  "alert": "tideDown",
                  "annotations": {
                     "message": "The service tide has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"tide\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               }
            ]
         },
         {
            "name": "prow-monitoring-absent",
            "rules": [
               {
                  "alert": "ServiceLostHA",
                  "annotations": {
                     "message": "The service {{ $labels.job }} has at most 1 instance for 5 minutes."
                  },
                  "expr": "sum(up{job=~\"grafana|prometheus|alertmanager\"}) by (job) <= 1\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               },
               {
                  "alert": "alertmanagerDown",
                  "annotations": {
                     "message": "The service alertmanager has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"alertmanager\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               },
               {
                  "alert": "prometheusDown",
                  "annotations": {
                     "message": "The service prometheus has been down for 5 minutes."
                  },
                  "expr": "absent(up{job=\"prometheus\"} == 1)\n",
                  "for": "5m",
                  "labels": {
                     "severity": "slack"
                  }
               }
            ]
         }
      ]
   }
}
...
seh commented 5 years ago

It requires an array because it's anticipating emitting a stream of several documents. You could implement it to accept a single object, and if that object is an array, do what it does normally, and if not, emit a stream of the single object. It's just not implemented that way today.

As for why you wound up with a YAML stream in which the content is still JSON, I'm confused too. @sparkprime, have you seen this before?

Globegitter commented 5 years ago

json is a subset of yaml and I am pretty sure that is just how jsonnet behaves currently. @hongkailiu can this be closed?

hongkailiu commented 5 years ago

Now I see what you mean by json is a subset of yaml: json can be parsed by any yaml parser https://jsonnet.org/learning/getting_started.html#stream

yaml_stream = True does the right thing according to above doc, just not what I wanted though.

Closing the issue.

@Globegitter Any known bazel tool which in be used to convert json to (real) yaml?

Globegitter commented 5 years ago

@hongkailiu nope, but it should be fairly easy to write one. Here for example I have written a json to js converter: https://github.com/ecosia/bazel_rules_nodejs_contrib/tree/master/internal/json_to_js

Globegitter commented 5 years ago

But also may I ask why you need "real" yaml? Is the output file meant to be read by people?

hongkailiu commented 5 years ago

But also may I ask why you need "real" yaml? Is the output file meant to be read by people?

https://github.com/kubernetes/test-infra/pull/14197#issuecomment-528156238

Originally when I opened the issue, I wanted yaml format for debugging as the output is used as input for rule k8s_object. Debugging (when things do not work) with yaml is easier. Now, the whole jsonnet -> json -> k8s pipeline works nicely together. The yaml format is not essential (only nice-to-have).

I also found a workaround with genrule to convert json to yaml (the links above) with https://github.com/brancz/gojsontoyaml