Slow speed in obtaining results.

[akaler727]

Hi there Hello, I'm using the m3e-base model and the task is text-encoding. The results I'm getting back are very slow, taking about 15 seconds to return. I see the main reason is in this code below ` // Encode performs the Bert encoding.

  func (e *Encoder) Encode(xs []ag.Node,t time.Time) []ag.Node {
    return e.Layers.Forward(xs...)
}

`

` // Forward operates on a slice of StandardModel connecting outputs to inputs sequentially for each module following, // finally returning its output.

  func (ml ModuleList[T]) Forward(xs ...ag.Node) []ag.Node {
      for _, m := range ml {
          xs = m.Forward(xs...)
      }
  }

`

My request value is: POST /v1/encode { "input": "The author believes that industrial ecology is to study the flow of materials and energy in industrial and consumer activities, understand these relationships and their influences on natural systems, and then take action to achieve a desired state while other states do not occur. The author also believes that a complete consideration of industrial ecology should include the full range of economic activities, such as mining, agriculture, forestry, manufacturing, and consumer behavior. It also includes the demand side and supply side of economic behaviors and their subsequent impacts on natural systems at all times and spatial scales.", "parameters": {}, "pooling_strategy":1 }

Could you please tell me why it's so slow?

[matteo-grella]

What is your OS and architecture (amd64 vs arm64)?

[akaler727]

Mine is amd64.

[matteo-grella]

Hello @akaler727 would you mind to update to the latest version and give it a try?