Closed michalzelechowski-getindata closed 3 years ago
Supporting kedro projects which use pyspark. With this version, dataproc is the only supported spark infrastructure, but the overall design allows extensions, f.e. to AWS Glue. There's also the way to declare custom operators via configuration.
Keep in mind:
Supporting kedro projects which use pyspark. With this version, dataproc is the only supported spark infrastructure, but the overall design allows extensions, f.e. to AWS Glue. There's also the way to declare custom operators via configuration.
Keep in mind: