Quoted values transformed to scientific notation number

[seniorquico]

I'm trying to build a set of deployments based on Hull and ran into an interesting YAML quoting/number formatting issue... I defined a variable tag that I reuse in several places. I populate it with a short Git revision hash. Here's an abbreviated version of the config:

hull:
  config:
    general:
      metadata:
        labels:
          common:
            revision: _HT*hull.config.specific.tag
    specific:
      tag: "3964137"
  objects:
    deployment:
      server:
        pod:
          containers:
            api:
              image:
                registry: us.gcr.io
                repository: my-app/my-api
                tag: _HT*hull.config.specific.tag

However, when I render using helm template and attempt to deploy it via Argo CD (which also calls helm template), I get this (abbreviated, again):

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    revision: "3.964137e+06"
spec:
  template:
    metadata:
      labels:
        revision: "3.964137e+06"
    spec:
      containers:
      - image: us.gcr.io/my-app/my-api:3.964137e+06

As shown above, I explicitly quoted the tag value, i.e. tag: "3964137", and I tried explicitly quoting the shortcut, e.g. tag: "_HT*hull.config.specific.tag". It doesn't seem to have any impact, though.

Is there a way to get my variable value respected as a string and not turned into scientific notation?

[gre9ory]

Hi @seniorquico,

thank you for using HULL and your report!

Unfortunately you have hit a hornets nest here. This is a general Helm problem (or I suppose more of a Go problem) and not specific to HULL, It pops up whenever you have an integer number which is greater than 1.000.000 wrapped in a string. Internally it converts the big int into a float64 and produces the dreaded scientific notation. So in your case - if the stars align badly - it is the commit hash being all numbers.

There are many threads about it, this matches your use case I think exactly. But there are lots of other reports of this such as this , this and this with the mother of all threads on the subject being this one having a display of 93 comments (think it is way more though) and 60 users involved. It started in 2016 and is still - from my point of view - open. There are a couple of workarounds mentioned here which may be applicable in your case,

Actually I have hit this too at some point when trying to reference a large number > 1.000.000 in a chart and putting it into an env var. I was similarily surprised to find this is a well known issue and yet unresolved in a proper fashion. When I found it I tried to do some magic in the inner code of HULL to get around it but it seemed hard to impossible since basically the very last toString call always messed it up and produced scientific notation. I dropped the topic back then due to limited time and it not being critical for us in this specific case. Nevertheless it left me quite unsatisfied at the time.

I will try to come up with a suggestion of applying one of the workarounds to your nice example, thanks for that. Or maybe I take another stab at it to solve it in an acceptable way in HULL. Even though it is not HULL's problem I would certainly like HULL to not stumble over it and offer some advantage over plain Helm if possible.

[gre9ory]

After a second deeper look it seems there is not much that can be done reasonably in HULL itself to overcome the issue(s).

The main somewhat related problems seem to be:

1. integer type information is lost from the YAML input when it gets processed in Helm so all integers seem to become float64 for internal processing
2. large numbers > 1.000.000 are represented as float64 in scientific notation when they are being refered to - with or without being wrapped as a string in the first place
3. strings ending with regex e\d+$ are interpreted as float64 in scientific notation by Helm - with or without being wrapped as a string in the first place

Looking at 1., the original type is already lost when starting to work with the values so it is impossible to automatically apply workarounds to treat everything correctly in the process. Something like 1e+07 could have been integer 1000000 or float 1000000.0 originally (which you'll never know) and guessing and auto-magical conversions when processing seem kind of dangerous. Taking 2. into consideration, it gets worse because not only is the value in the string interpeted as a number, it is converted to float64 and represented in scientifc notation.

The simple and convenient - yet mostly sufficient - _HT* to reference the value does not suffice here and you need to apply one of the various workarounds mentioned in the issue threads. You can achieve this putting the workaround code in a full-fleged _HT! transformation.

The following is tested with helm version=3.13.3

• To reference a value that is unquoted integer and potentially > 1.000.000 you can use this transformation where you need the target value as an big integer:

  _HT!{{ (index . "$").Values.hull.config.specific.potentiallyLargeInteger | int64 }}

  For example, if potentiallyLargeInteger is 123456789 this will produce 123456789 without any quotes.

• To reference a value that is unquoted integer and potentially > 1.000.000 you can use this transformation where you need the target value as a string:

  _HT!{{ (index . "$").Values.hull.config.specific.potentiallyLargeInteger | int64 | quote }}

  For example, if potentiallyLargeInteger is 123456789 this will produce "123456789" with quotes.

  You need to use this if you are creating the value of an env var, an annotation or label or image tag and are expecting this kind of input!

• To reference a value that is string and potentially an integer > 1.000.000 or potentially matches the scientific notation regex e\d+$ you can use this transformation where the target value is a string:

  _HT!{{ (index . "$").Values.hull.config.specific.gitCommit | toJson | trimAll "\"" | quote }}

  For example, if gitCommit is "123456789" this will produce "123456789" or if gitCommit is "11724e28" this will produce "11724e28".

  You always need to use quotation marks around the input value, otherwise it may be interpreted as a number in scientific notation by which the wanted string representation is forever gone

The above mentioned and in my opinion best workarounds I took from here.

---

To conclude, I think you should try using the following code to cover all bases since the commit hash may both resemble large integers > 1.000.000 or a general number in scientific notation:

hull:
  config:
    general:
      metadata:
        labels:
          common:
            revision: _HT!{{ (index . "$").Values.hull.config.specific.tag | toJson | trimAll "\"" | quote }}
    specific:
      tag: "3964137"
  objects:
    deployment:
      server:
        pod:
          containers:
            api:
              image:
                registry: us.gcr.io
                repository: my-app/my-api
                tag: _HT!{{ (index . "$").Values.hull.config.specific.tag | toJson | trimAll "\"" | quote }}

Sorry it is that ugly and unintuitive, can't do much about the original problem :(

Hope what I wrote above is correct and proposed solution works for you, if not let me know!

[gre9ory]

@seniorquico Happy new year to you ;)

Are you satisfied with the explanations above? If so let me know, if not please also let me know.

Will leave this open for a couple of days hoping to hear from you before closing,

Thanks!

[seniorquico]

Yes, I understand the limitations. I'll give this workaround a try the next time this problem occurs. Thanks for the follow-up!