getdozer / dozer

Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.
https://getdozer.io
GNU Affero General Public License v3.0
1.48k stars 120 forks source link

GRPC query method doesnt work #1379

Closed karolisg closed 1 year ago

karolisg commented 1 year ago

Describe the bug

If table contains any number, GRPC query method fails with error like Response message parsing error: invalid wire type 7 at offset 217

Screenshot 2023-04-04 at 16 08 52

This started to happen after feat: u128 and i128 Field and FieldType (#1315) (611912ee486d742773c5c43a515b4b3392bac6f4)

Environment

To Reproduce

Steps to reproduce the behavior:

  1. Share your dozer-config.yaml
    
    app_name: flight-microservices
    app_max_map_size: 737280

flags: grpc_web: true

api: api_security: !Jwt ABCD

connections:

sql: |

select f.arrival_airport as airport, a.coordinates as coordinates, COUNT(t.ticket_no) as tickets

INTO airports_count

from tickets t

join ticket_flights tf on t.ticket_no = tf.ticket_no

join flights f on tf.flight_id = f.flight_id

join airports a on f.arrival_airport = a.airport_code

group by f.arrival_airport, a.coordinates;

#

select extract(HOUR FROM f.window_start) as start, count(f.window_start) as dep_count

INTO departures_count

from TUMBLE(flights, scheduled_departure, '4 HOURS') f

group by extract(HOUR FROM f.window_start)

sources:

endpoints:

chloeminkyung commented 1 year ago

Tried to replicate this bug but failed - following is the steps that I took

Scenario 1

  1. created master token by running dozer api generate-token
  2. run gRPC common service query with no filter

Scenario 2

  1. created specific token by calling getAuthToken of AuthGrpcService with access_filter
    {"access_filter": "{\"Custom\":{\"flights\":{\"filter\":{\"flight_id\":{\"$eq\":32658}},\"fields\":[]}}}"}
  2. run gRPC flights service query

Both of the scenario worked fine with the config provided

karolisg commented 1 year ago

Looks like it was postman cache issue.