.toString(36) not properly/fully implemented

[the-hotmann]

I also encountered this error:

https://stackoverflow.com/a/52524228

I was about to implement this TS function:

function getToken(id: string) {
  return ((Number(id) / 1e15) * Math.PI).toString(36).replace(/(0+|\.)/g, '')
}

in golang and wanted to use JS interpreter as validation. But it does not work as expected.

When I run it for the a twitter ID, it does just generate the first "part" of the result. Everything after and including the dot, is missing (the dot gets removed anyway), but the token after the dot is missing as well.

Expected result:

4dh7jvecqt6

actual result:

4dh

Here what this function does:

1. take a string
2. convert to float64 (maybe float36? - not too sure about this one)
3. floatNum := (num / 1e15) * math.Pi
4. convert float to base36
5. remove all zeros and dots (0 & .)

That is all. But this tool, does not do this properly, it converts the string to an int and converts the int to base36. Which is missing some operations.

[stevenh]

This is actually documented as incomplete here happy to accept PRs

[the-hotmann]

I actually will provide some code, that comes very close to the original JS function. But it still requires some work.

[the-hotmann]

I will not PR just yet, as the current implementation is not correct, and therefore shall not be merged. I guess I have an error somewhere, as sometimes I am right, sometimes I am a little off..

Here the code:

package main

import (
    "fmt"
    "math"
    "strconv"
    "strings"
)

const base36 = "0123456789abcdefghijklmnopqrstuvwxyz" // basically [0-9a-z]

func main() {
    id := "1808927037068898626"
    token := getToken(id)
    fmt.Println("Input:\t\t", id, "\n")
    fmt.Println("Go Token:\t", token)
    fmt.Println("JS Token:\t", "4duwtt5xm9k") // calculated in JS (see function below)
}

func getToken(string string) string {
    // Convert id to a float36
    num, err := strconv.ParseFloat(string, 36)
    if err != nil {
        panic(err)
    }

    // return the token (will be numbers as string)
    return strings.ReplaceAll(strings.ReplaceAll(floatToBase36((num/1e15)*math.Pi), "0", ""), ".", "")
}

// Convert a float64 to base36 as string
func floatToBase36(f float64) string {
    if f == 0 {
        return "0"
    }

    intPart := int(f)
    fracPart := f - float64(intPart)

    intPartBase36 := intToBase36(intPart)
    fracPartBase36 := fractionToBase36(fracPart, 8) // Precision of 8 (here you can change the precision - please test)

    return fmt.Sprintf("%s.%s", intPartBase36, fracPartBase36)
}

// Convert an integer to base36 string
func intToBase36(n int) string {
    if n == 0 {
        return "0"
    }

    // use stringbuilder for perofrmance
    var sb strings.Builder

    for n > 0 {
        remainder := n % 36
        sb.WriteByte(base36[remainder])
        n = n / 36
    }

    // Reverse the string since we constructed it backwards
    result := []rune(sb.String())
    for i, j := 0, len(result)-1; i < j; i, j = i+1, j-1 {
        result[i], result[j] = result[j], result[i]
    }

    return string(result)
}

// Convert a fraction to base36 string
func fractionToBase36(f float64, precision int) string {
    var sb strings.Builder
    for precision > 0 {
        f = f * 36
        digit := int(f)
        sb.WriteByte(base36[digit])
        f = f - float64(digit)
        precision--
    }
    return sb.String()
}

// original JS function from
/**

function getToken(id) {
  token = ((Number(id) / 1e15) * Math.PI).toString(6 ** 2).replace(/(0+|\.)/g, '')
  console.log(token)
  return token
}

**/

Current output:

Input:           1808927037068898626 

Go Token:        4duwtt5xm9j
JS Token:        4duwtt5xm9k

(seems to be an error in the fracPartBase36 function..)

[the-hotmann]

@stevenh Ok lol just found the error. It seems to match now ;)

package main

import (
    "fmt"
    "math"
    "strconv"
    "strings"
)

const base36 = "0123456789abcdefghijklmnopqrstuvwxyz" // [0-9a-z]

func main() {
    id := "1808927037068898626"
    token := getToken(id)
    fmt.Println("Input:\t\t", id, "\n")
    fmt.Println("Go Token:\t", token)
    fmt.Println("JS Token:\t", "4duwtt5xm9k") // Calculated in JS (see function below)
}

func getToken(id string) string {
    // Convert id to a float64
    num, err := strconv.ParseFloat(id, 64)
    if err != nil {
        panic(err)
    }

    // Return the token
    return strings.ReplaceAll(strings.ReplaceAll(floatToBase36((num/1e15)*math.Pi, 8), "0", ""), ".", "")
}

// Convert a float64 to base36 as string
func floatToBase36(f float64, precision int) string {
    intPart := int(f)
    fracPart := f - float64(intPart)

    intPartBase36 := intToBase36(intPart)
    fracPartBase36 := fractionToBase36(fracPart, precision)

    return intPartBase36 + fracPartBase36
}

// Convert an integer to base36 string
func intToBase36(n int) string {
    if n == 0 {
        return "0"
    }

    // Use a string builder for performance
    var sb strings.Builder

    for n > 0 {
        remainder := n % 36
        sb.WriteByte(base36[remainder])
        n = n / 36
    }

    // Reverse the string since we constructed it backwards
    result := []rune(sb.String())
    for i, j := 0, len(result)-1; i < j; i, j = i+1, j-1 {
        result[i], result[j] = result[j], result[i]
    }

    return string(result)
}

// Convert a fraction to base36 string with rounding
func fractionToBase36(f float64, precision int) string {
    var sb strings.Builder
    for i := 0; i < precision+1; i++ {
        f *= 36
        digit := int(f)
        sb.WriteByte(base36[digit])
        f -= float64(digit)
    }

    // Round up the last digit if necessary
    result := sb.String()
    if len(result) > precision {
        if result[precision] != '0' { // If the next digit after precision is not zero, round up - this is to match JS behavior
            rounded, _ := strconv.ParseInt(result[:precision], 36, 64)
            rounded++
            result = intToBase36(int(rounded))
        } else {
            result = result[:precision]
        }
    }

    return result
}

// Original JS function:
/**
function getToken(id) {
  token = ((Number(id) / 1e15) * Math.PI).toString(6 ** 2).replace(/(0+|\.)/g, '')
  console.log(token)
  return token
}
**/

Result:

Input:           1808927037068898626 

Go Token:        4duwtt5xm9k
JS Token:        4duwtt5xm9k

@all feel free to look over it and improve it performance and structure wise ;) This is the first working function that follow the logical implementation of the original JS implementation.

This was from a (react/tsx I guess) JS snippet, that I found on GitHub that implemented .toString(36) as part of its Twitter/X-ID to Token conversion part.

The input of this function takes a Twitter/X-ID and gives you a token which lets you access some data of the post.

[the-hotmann]

Here a nicer implementation with .toString36WithPrecision(precision int) and .toString36() (default precision is 8, as used by JS)

package main

import (
    "fmt"
    "math"
    "strconv"
    "strings"
)

const base36 = "0123456789abcdefghijklmnopqrstuvwxyz" // [0-9a-z]
const defaultPrecision = 8

type MyFloat64 float64

func main() {
    id := "1808927037068898626"
    token := getToken(id)
    fmt.Println("Input:\t\t", id, "\n")
    fmt.Println("Go Token:\t", token)
    fmt.Println("JS Token:\t", "4duwtt5xm9k") // Calculated in JS (see function below)
}

func getToken(id string) string {
    // Convert id to a float64
    num, err := strconv.ParseFloat(id, 64)
    if err != nil {
        panic(err)
    }

    // Calculate the token
    preCalc := MyFloat64((num / 1e15) * math.Pi)
    floatToString36 := preCalc.toString36()

    // cleanup zeros and dots
    token := strings.ReplaceAll(strings.ReplaceAll(floatToString36, "0", ""), ".", "")

    // Return the token
    return token
}

// Method to convert MyFloat64 to base-36 string with default precision
func (f MyFloat64) toString36() string {
    return f.toString36WithPrecision(defaultPrecision)
}

// Convert a float64 to base36 as string with precision (number of digits after the decimal point)
func (f MyFloat64) toString36WithPrecision(precision int) string {
    intPart := int(f)
    fracPart := float64(f) - float64(intPart)

    intPartBase36 := intToBase36(intPart)
    fracPartBase36 := fractionToBase36(fracPart, precision)

    return intPartBase36 + fracPartBase36
}

// Convert an integer to base36 string
func intToBase36(n int) string {
    if n == 0 {
        return "0"
    }

    // Use a string builder for performance
    var sb strings.Builder

    for n > 0 {
        remainder := n % 36
        sb.WriteByte(base36[remainder])
        n = n / 36
    }

    // Reverse the string since we constructed it backwards
    result := []rune(sb.String())
    for i, j := 0, len(result)-1; i < j; i, j = i+1, j-1 {
        result[i], result[j] = result[j], result[i]
    }

    return string(result)
}

// Convert a fraction to base36 string with rounding
func fractionToBase36(f float64, precision int) string {
    var sb strings.Builder
    for i := 0; i < precision+1; i++ {
        f *= 36
        digit := int(f)
        sb.WriteByte(base36[digit])
        f -= float64(digit)
    }

    // Round up the last digit if necessary
    result := sb.String()
    if len(result) > precision {
        if result[precision] != '0' { // If the next digit after precision is not zero, round up - this is to match JS behavior
            rounded, _ := strconv.ParseInt(result[:precision], 36, 64)
            rounded++
            result = intToBase36(int(rounded))
        } else {
            result = result[:precision]
        }
    }

    return result
}

// Original JS function:
/**
function getToken(id) {
  token = ((Number(id) / 1e15) * Math.PI).toString(6 ** 2).replace(/(0+|\.)/g, '')
  console.log(token)
  return token
}
**/

Please notice that this works fine for toString(36), but might not work as well for other lengths, as this was made by me for toString(36) explicitly. But with not much effort it very likely can be adjusted to the general floatToString() function that JS provides.

[the-hotmann]

https://gist.github.com/the-hotmann/a745c76c806128db68700685105a20e8

[stevenh]

Cool feel free to get a PR, easier to look at there

[the-hotmann]

Currently it is just a replacement for toString(36) from JS ;) I will, as soon as I have implemented it as a replacement for toString() from JS. Or after giving up and accepting, that I just can make .toString(36) work.

But as they all have different implementations I guess there needs to be a differenciator. This implementation therefore would be for toString(36) the others currently would be missing.

Also: I overworked it again. No precision is neede anymore.