feat(component): rust component model support

chrisdickinson commented 11 months ago

upgrade to wasmtime and wasmtime_wasi to ">= 17.0.0, < 18.0.0"
- 17 is the wasi preview 2 supporting release
the component model necessitated splitting the adapter setup into two parts: first creating the adapter so it can be associated with the state, then telling the linker how to get from the incoming state to the adapter. WASI also has this split.
change the c bindings in observe-api:
1. "_" is an illegal character! switch to "-". (This is a breaking change!)
2. change the module name from "dylibso_observe" to "dylibso:observe/api"
The component model representation of our format enum doesn't match our core module format – Statsd is 0 in the component model version.

Issue: https://github.com/dylibso/observe-sdk/issues/142

G4Vi commented 10 months ago

https://github.com/dylibso/observe-sdk/blob/5a8d5ac20f123c1f3a94e9b0defdbdb3d570a5d3/rust/src/wasm_instr.rs#L9 should be bumped to 4 to match the change in wasm-instr as changing the naming is a breaking change. This change probably needs to be done on the other language sdks.

The pointers in the observe api were intentionally passed as uint64's in order to have the same api with wasm with 64 bit linear memory. @chrisdickinson what are your thoughts on wasm64 support?

chrisdickinson commented 10 months ago

@chrisdickinson what are your thoughts on wasm64 support?

I ended up aligning our API with the component model's canonical ABI, which is defined in terms of 32-bit pointers and extents. It looks like they're tracking the memory64 proposal here and here; but both issues haven't had activity in about a year.

This makes the observe-api-crate-punning approach a little more appealing compared to the component::bindgen approach, to be honest. At least with the crate we could hide the difference so that users wouldn't have to recompile their applications when memory64 support lands in the component model. I'd want to get a read from someone closer to the component model implementation on how likely ABI changes to accommodate memory64 are, though.

G4Vi commented 10 months ago

Thanks for the thorough explanation on the pointer passing change! That sounds reasonable.

G4Vi commented 8 months ago

@mhmd-azeez Could you please try out 0b43202 with the modules and adapters used on https://github.com/dylibso/observe-sdk/pull/152 ? I just finished rebasing this on top of #152 and manually reapplying my changes to the go sdk and want to be sure it's still working as intended. The only change when running modules with the dylibso_observe namespace should be a deprecation warning printed to stdout:

log.Println("Module uses deprecated namespace \"dylibso_observe\"!\n" +
                    "Please consider reinstrumenting with newer wasm-instr or Observe API!")

mhmd-azeez commented 7 months ago

@G4Vi sorry, just saw your comment. It seems to be working correctly:

Old modules:

PS D:\dylibso\observe-sdk\go\bin\otelstdout> go run . "D:\x\observe-sdk\observe-api\test\rust_guest.wasm" | jq
2024/01/23 13:37:47 Module uses deprecated namespace "dylibso_observe"!
Please consider reinstrumenting with newer wasm-instr or Observe API!
2024/01/23 13:37:47 metric: worlds.helloed:1|c
2024/01/23 13:37:47 Hello World 1
2024/01/23 13:37:47 metric: worlds.helloed:1|c
2024/01/23 13:37:47 Hello World 2
2024/01/23 13:37:47 metric: worlds.helloed:1|c
2024/01/23 13:37:47 Hello World 3
{
  "resource_spans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "Value": {
                "StringValue": "golang"
              }
            }
          }
        ]
      },
      "scope_spans": [
        {
          "spans": [
            {
              "trace_id": "IBIj3rV2n1slwYUSDAYlhQ==",
              "span_id": "wY/dyqn58kk=",
              "name": "main",
              "kind": 1,
              "start_time_unix_nano": 1706006267423033100,
              "end_time_unix_nano": 1706006267423033100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-main"
                    }
                  }
                },
                {
                  "key": "user_id",
                  "value": {
                    "Value": {
                      "StringValue": "123"
                    }
                  }
                },
                {
                  "key": "world",
                  "value": {
                    "Value": {
                      "StringValue": "hello"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "IBIj3rV2n1slwYUSDAYlhQ==",
              "span_id": "FaQZ7o/bA2E=",
              "parent_span_id": "wY/dyqn58kk=",
              "name": "run",
              "kind": 1,
              "start_time_unix_nano": 1706006267423033100,
              "end_time_unix_nano": 1706006267423033100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-run"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "IBIj3rV2n1slwYUSDAYlhQ==",
              "span_id": "PtKcp19WCwo=",
              "parent_span_id": "FaQZ7o/bA2E=",
              "name": "log_something",
              "kind": 1,
              "start_time_unix_nano": 1706006267423033100,
              "end_time_unix_nano": 1706006267423033100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-log_something"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "IBIj3rV2n1slwYUSDAYlhQ==",
              "span_id": "NjHWDxzMqEg=",
              "parent_span_id": "FaQZ7o/bA2E=",
              "name": "log_something",
              "kind": 1,
              "start_time_unix_nano": 1706006267423033100,
              "end_time_unix_nano": 1706006267423033100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-log_something"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "IBIj3rV2n1slwYUSDAYlhQ==",
              "span_id": "JyCk0E2FLwM=",
              "parent_span_id": "FaQZ7o/bA2E=",
              "name": "log_something",
              "kind": 1,
              "start_time_unix_nano": 1706006267423033100,
              "end_time_unix_nano": 1706006267423033100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-log_something"
                    }
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}
PS D:\dylibso\observe-sdk\go\bin\otelstdout> cd ..\stdout\
PS D:\dylibso\observe-sdk\go\bin\stdout> go run . "D:\x\observe-sdk\observe-api\test\rust_guest.wasm"
2024/01/23 13:38:19 Module uses deprecated namespace "dylibso_observe"!
Please consider reinstrumenting with newer wasm-instr or Observe API!
2024/01/23 13:38:19 metric: worlds.helloed:1|c
2024/01/23 13:38:19 Hello World 1
2024/01/23 13:38:19 metric: worlds.helloed:1|c
2024/01/23 13:38:19 Hello World 2
2024/01/23 13:38:19 metric: worlds.helloed:1|c
2024/01/23 13:38:19 Hello World 3
2024/01/23 13:38:19  Call to main took 502µs
2024/01/23 13:38:19  Span tags: [user_id:123 world:hello]
2024/01/23 13:38:19    Call to run took 502µs
2024/01/23 13:38:19      Call to log_something took 502µs
2024/01/23 13:38:19      Call to log_something took 0s
2024/01/23 13:38:19      Call to log_something took 0s

New modules:

PS D:\dylibso\observe-sdk\go\bin\stdout> go run . "D:\dylibso\observe-sdk\observe-api\test\rust_guest.wasm"
2024/01/23 13:39:17 metric: worlds.helloed:1|c
2024/01/23 13:39:17 Hello World 1
2024/01/23 13:39:17 metric: worlds.helloed:1|c
2024/01/23 13:39:17 Hello World 2
2024/01/23 13:39:17 metric: worlds.helloed:1|c
2024/01/23 13:39:17 Hello World 3
2024/01/23 13:39:17  Call to main took 0s
2024/01/23 13:39:17  Span tags: [user_id:123 world:hello]
2024/01/23 13:39:17    Call to run took 0s
2024/01/23 13:39:17      Call to log_something took 0s
2024/01/23 13:39:17      Call to log_something took 0s
2024/01/23 13:39:17      Call to log_something took 0s
PS D:\dylibso\observe-sdk\go\bin\stdout> cd ..\otelstdout\
PS D:\dylibso\observe-sdk\go\bin\otelstdout> go run . "D:\dylibso\observe-sdk\observe-api\test\rust_guest.wasm" | jq
2024/01/23 13:39:31 metric: worlds.helloed:1|c
2024/01/23 13:39:31 Hello World 1
2024/01/23 13:39:31 metric: worlds.helloed:1|c
2024/01/23 13:39:31 Hello World 2
2024/01/23 13:39:31 metric: worlds.helloed:1|c
2024/01/23 13:39:31 Hello World 3
{
  "resource_spans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "Value": {
                "StringValue": "golang"
              }
            }
          }
        ]
      },
      "scope_spans": [
        {
          "spans": [
            {
              "trace_id": "KTuK5VqbIhxZtGZYGhrjBQ==",
              "span_id": "hAtU6Is1GG0=",
              "name": "main",
              "kind": 1,
              "start_time_unix_nano": 1706006371741159100,
              "end_time_unix_nano": 1706006371741159100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-main"
                    }
                  }
                },
                {
                  "key": "user_id",
                  "value": {
                    "Value": {
                      "StringValue": "123"
                    }
                  }
                },
                {
                  "key": "world",
                  "value": {
                    "Value": {
                      "StringValue": "hello"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "KTuK5VqbIhxZtGZYGhrjBQ==",
              "span_id": "NDcJDoLD1j8=",
              "parent_span_id": "hAtU6Is1GG0=",
              "name": "run",
              "kind": 1,
              "start_time_unix_nano": 1706006371741159100,
              "end_time_unix_nano": 1706006371741159100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-run"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "KTuK5VqbIhxZtGZYGhrjBQ==",
              "span_id": "PfvaR+U8Q3U=",
              "parent_span_id": "NDcJDoLD1j8=",
              "name": "log_something",
              "kind": 1,
              "start_time_unix_nano": 1706006371741159100,
              "end_time_unix_nano": 1706006371741159100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-log_something"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "KTuK5VqbIhxZtGZYGhrjBQ==",
              "span_id": "mWvWM3e/Wxw=",
              "parent_span_id": "NDcJDoLD1j8=",
              "name": "log_something",
              "kind": 1,
              "start_time_unix_nano": 1706006371741159100,
              "end_time_unix_nano": 1706006371741159100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-log_something"
                    }
                  }
                }
              ]
            },
            {
              "trace_id": "KTuK5VqbIhxZtGZYGhrjBQ==",
              "span_id": "v4kawKvAK0k=",
              "parent_span_id": "NDcJDoLD1j8=",
              "name": "log_something",
              "kind": 1,
              "start_time_unix_nano": 1706006371741159100,
              "end_time_unix_nano": 1706006371741159100,
              "attributes": [
                {
                  "key": "function-name",
                  "value": {
                    "Value": {
                      "StringValue": "function-call-log_something"
                    }
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

dylibso / observe-sdk

feat(component): rust component model support #128