vaexio / vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
https://vaex.io
MIT License
8.23k stars 590 forks source link

How to read from vaex server with requests with pyton #2369

Closed luksobczyk closed 1 year ago

luksobczyk commented 1 year ago

hello,

I'm quite aware it seems like a simple problem, but can't find a solution. So, I've started a vaex server as follows (running on local machine)

[05/04/23 11:34:24] INFO master_data: http://0.0.0.0:8080/dataset/master_data for REST or ws://0.0.0.0:8080/master_data for websocket fastapi.py:452 INFO position_history_data: http://0.0.0.0:8080/dataset/position_history_data for REST or ws://0.0.0.0:8080/position_history_data for websocket fastapi.py:452 ←[32mINFO←[0m: Started server process [←[36m13276←[0m] ←[32mINFO←[0m: Waiting for application startup. ←[32mINFO←[0m: Application startup complete. ←[32mINFO←[0m: Uvicorn running on ←[1mhttp://0.0.0.0:8080←[0m (Press CTRL+C to quit)

I would like to read data with requests - i'm getting a 200 response but, accurate row count, but no data? What am I doing wrong?

Code:

import requests import pandas as pd import json

set the URL for the REST API endpoint

url = "http://127.0.0.1:8080/dataset/master_data"

set the headers

headers = {"Content-Type": "application/json"}

make the request

response = requests.get(url)

parse the response

if response.status_code == 200: data = response.json() print(json.dumps(data, indent=4, sort_keys=False))

else: print(f"Error: {response.status_code} - {response.reason}")

Response:

{ "id": "master_data", "row_count": 3706, "schema": { "": "int64", "CURRENT_ORDER_ID": "int64", "CLIENT_SYMBOL": "string", "CURRENT_SUBORDER_ID": "int64", "MAIN_ORDER_ID": "int64", "MAIN_ORDER_NUMBER": "string", "SUB_ORDER_NUMBER": "string", "ORDER_TYPE": "string", "ORDER_TMS_ROUTE": "string", "ORDER_DIRECTION": "string", "ORDER_LEVEL": "string", "SUB_ORDER_FILEGROUP": "string", "ORDER_STATUS": "string", "TELEMATICS_STATUS": "string", "TELEMATICS_STATUS_DESCRIPTION": "string", "MODIFY_USER_NAME": "string", "MODIFY_TIME": "float64", "CARGO_DESCRIPTION": "string", "CS_REPRESENTATIVE": "string", "MAIN_ORDER_COORDINATOR": "string", "SUB_ORDER_COORDINATOR": "string", "EVENT_TYPE": "string", "REF_ODB": "string", "EVENT_ID": "int64", "EVENT_ID_MAIN_ORDER": "int64", "VEHICLE_LOCATION_KEY": "string", "EVENT_GEO_ZONE_HOUR_CORRECTION": "float64", "DISPOSITION_HOUR_BEGIN": "time32[s]", "DISPOSITION_HOUR_END": "time32[s]", "PLANNED_HOUR_BEGIN": "time32[s]", "PLANNED_HOUR_END": "time32[s]", "PLANNED_ORDER_DAY_START": "date32[day]", "PLANNED_ORDER_DAY_END": "date32[day]", "EVENT_DELAY_REASON": "string", "MAIN_ORDER_TRUCK_PLATES": "string", "MAIN_ORDER_TRUCK_SIDE_NO": "string", "SUB_ORDER_TRUCK_PLATES": "string", "SUB_ORDER_TRUCK_SIDE_NO": "string", "TRAILER_ID": "float64", "TRAILER_PLATES": "string", "TRAILER_SIDE_NO": "string", "TRACKED_VEHICLE": "string", "TELEMATICS_DEVICE_NAME": "string", "TELEMATICS_CUSTOMER_ZONE": "string", "TELEMATICS_TRIP": "string", "TELEMATICS_GPS_LON": "float64", "TELEMATICS_GPS_LAT": "float64", "TELEMATICS_DRIVE_MODE": "string", "TELEMATICS_GPS_DIRECTION": "float64", "TELEMATICS_GPS_AGE": "string", "TELEMATICS_STATE_ERROR": "string", "TELEMATICS_STATE_CONNECTION": "string", "TELEMATICS_AVG_SPEED": "float64", "TELEMATICS_GPS_CURRENT_COUNTRY": "string", "TELEMATICS_GPS_CURRENT_CITY": "string", "TELEMATICS_GPS_CURRENT_POSTAL_CODE": "string", "LOCATION_ID": "int64", "EVENT_COUNTRY": "string", "EVENT_CITY": "string", "EVENT_ADDRESS": "string", "EVENT_LOCATION_LAT": "float64", "EVENT_LOCATION_LON": "float64", "KMS_TO_EVENT_LOCATION": "float64", "NR_ZLECENIA_K": "int8", "EVENT_COMPLETED": "string", "ORDER_COMPLETED": "string", "PREVIOUS_EVENT_NO": "float64", "PREVIOUS_EVENT_COMPLETED": "string", "NUMBER_OF_EVENTS": "int64", "NUMBER_OF_EVENTS_NO_END_TIME": "int64", "IS_LATE": "int64", "LATENESS": "float64", "SnapshotTime": "datetime64[ns]", "DAYS_DELTA": "int64", "COFOR": "string", "LOCATION": "string", "EMAIL_CONTACT_KEY": "string", "DRIVER_1": "string", "DRIVER_MOBILE_1": "string", "DRIVER_2": "string", "DRIVER_MOBILE_2": "string", "USR_FLD_EXT_ORDER_ID": "float64", "USR_FLD_EXT_ORDER_NUM": "int8", "USR_FLD_EXT_LOAD_ID": "float64", "USR_FLD_EXT_LOAD_NUM": "int64", "USR_FLD_REF1": "string", "USR_FLD_REF2": "string", "USR_FLD_EXT_LAST_UPDATE": "int8", "ORDER_ROUTE": "string", "FLOW": "string", "SNG_RT": "string", "REF_4": "int64", "LOAD_MAT": "int64", "LOAD_EMT": "int64" } }

Process finished with exit code 0