PubSub trigger `json: cannot unmarshal object`

[nelsonpina]

Hi!

I'm trying to deploy a Go Cloud Function, with PubSub trigger, but I'm always hitting this error:

{
 insertId:  "000000-0e31e000-5a9c-41f7-b40e-5af2eba5cdab"   
 labels: {
  execution_id:  ""    
 }
 logName:  "projects/infrasnukture/logs/cloudfunctions.googleapis.com%2Fcloud-functions"   
 receiveTimestamp:  "2018-07-26T11:27:35.175212742Z"   
 resource: {
  labels: {
   function_name:  "test-go1"     
   project_id:  "infrasnukture"     
   region:  "europe-west1"     
  }
  type:  "cloud_function"    
 }
 severity:  "ERROR"   
 textPayload:  "Failed to decode event: json: cannot unmarshal object into Go struct field EventContext.resource of type string
"   
 timestamp:  "2018-07-26T11:27:28.982Z"   
}

This is the payload of the PubSub message:

{
"data":{
    "data":"dGVzdA==",
    "attributes":{
      "age":"22"
    },
    "@type":"type.googleapis.com/google.pubsub.v1.PubsubMessage"
  }
}

And my main.go looks like this:

package main

import (
    "flag"
    "net/http"

    "./events"
    "./nodego"
)

func main() {
    flag.Parse()

    http.HandleFunc(nodego.PubSubTrigger, events.Handler(func(event *events.Event) error {
        nodego.InfoLogger.Printf("PubSub triggered Go function!")

        msg, err := event.PubSubMessage()
        nodego.InfoLogger.Printf("Your message: %s", msg.Data)
        if err != nil {
            return err
        }
        return nil
    }))

    nodego.TakeOver()
}

BTW, if I run it locally on my machine, it works fine.

Appreciate some help!

[iangudger]

@ssttevee, can you take a look?

[nelsonpina]

I dug a bit deeper and found out the following:

the error with decoding the event can be avoided if instead of trying to decode the event we decode event.Data.

@ events.go / line: 152

var event Event
if err := json.NewDecoder(r.Body).Decode(&event.Data); err != nil {
    nodego.ErrorLogger.Print("Failed to decode event: ", err)
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
}

the event object seems still to be interpreted differently locally and in the cloud. With the above fix, if my POST body looks like this:

{
    "data":"dGVzdA=="
}

the output of the Your message: infoLogger in the main bellow is:

func main() {
    flag.Parse()

    http.HandleFunc(nodego.PubSubTrigger, events.Handler(func(event *events.Event) error {
        nodego.InfoLogger.Printf("PubSub triggered Go function!")
        nodego.InfoLogger.Printf("Your message: %s", event.Data)

        msg, err := event.PubSubMessage()
        if err != nil {
            return err
        }
        nodego.InfoLogger.Printf("Your message: %s", msg.Data)

        return nil
    }))

    nodego.TakeOver()
}

locally

Your message: {
    "data":"dGVzdA=="
}

deployed cloud function

Your message: {
    "data": {
        "data": "dGVzdA=="
    },
    "context": {
        "eventId": "3b8ddd8e-82e1-4c48-aec6-6519bcf7c98a",
        "resource": {
            "service": "pubsub.googleapis.com",
            "name": "projects/infrasnukture/topics/apexes-telemetry"
        },
        "eventType": "google.pubsub.topic.publish",
        "timestamp": "2018-07-27T11:05:26.462Z"
    }
}

Which then fails to retrieve the PubSub message at msg, err := event.PubSubMessage().

any idea why we have different behaviour between local and deployed envs?

[ssttevee]

There seems to be a disparity between what's documented on google's website and what is actually being sent: https://cloud.google.com/functions/docs/writing/background#function_parameters. It says that the event.context.resource value is supposed to be a string, but the resource value in the output you've pasted above is not quite a string.

You might also run into another issue if you're using the testing tab of the deployed function page because google likes wrapping your test message with some context information. When testing locally, if your actual message is:

{
    "data": "dGVzdA=="
}

you'll have to wrap it up like this:

{
    "data": {
        "data": "dGVzdA=="
    }
}

to simulate the context wrapper.

[ssttevee]

@iangudger, should the documentation be consider truth or should this be accounted for in the events package?

[nelsonpina]

FYI, these are the changes I have made to get it going for PubSub events:

@ events.go / line: 90

func (e *Event) PubSubMessage() (*PubSubMessage, error) {

    type EventData struct {
        Data json.RawMessage `json:"data"`
    }

    var eventData EventData
    if err := json.Unmarshal(e.Data, &eventData); err != nil {
        return nil, err
    }

    var msg pubsub.PubsubMessage
    if err := json.Unmarshal(eventData.Data, &msg); err != nil {
        return nil, err
    }
    ....

@ events.go / line: 152

    if err := json.NewDecoder(r.Body).Decode(&event.Data); err != nil {
    ....

I had to decode events.Data to get it to pass HTTP body decoding, and then I added a new structure to retrieve "data:" from the event payload to be able to unmarshal the event.Data.

@ssttevee the documentation regarding the event object is indeed a bit confusing and it doesn't seem consistent.

Note: the changes I made may break the event handling for different event types, (HTTP, etc..)

[iangudger]

@ssttevee It doesn't surprise me too much that the documentation isn't fully correct. Feel free to send a PR to fix this case.