search

kaiko-ai / typedspark

Column-wise type annotations for pyspark DataFrames
Apache License 2.0
65 stars 4 forks source link

Autocomplete structtypes #72

Closed nanne-aben closed 1 year ago

nanne-aben commented 1 year ago

This PR provides autocomplete for structtype columns, for example:

from pyspark.sql import SparkSession
from pyspark.sql.types import IntegerType
from typedspark import Column, Schema
from typedspark._core.datatypes import StructType

spark = SparkSession.builder.getOrCreate()

class A(Schema):
    a: Column[IntegerType]

class B(Schema):
    a: Column[IntegerType]
    b: Column[StructType[A]]

B.b.dtype.schema.a