MrPowers / chispa

PySpark test helper methods with beautiful error messages
https://mrpowers.github.io/chispa/
MIT License
595 stars 65 forks source link

Add tree string #95

Open jeffbrennan opened 6 months ago

jeffbrennan commented 6 months ago

addresses #88

See comment below

jeffbrennan commented 6 months ago

hey @MrPowers, this is ready for review. I based the output on how pyspark prints schemas with the printSchema() function. Verified that this approach works with multiple levels of nested structs, schemas of different lengths, ignore_nullable T/F, and ignore_metadata T/F

Standard example:

image

Different lengths:

image

Multiple nested structs:

image

Some thoughts for additional features

kleemaj commented 1 week ago

This fix is super helpful! Any chance it can get merged? Happy to try resolving the merge conflicts if you need help.

jeffbrennan commented 1 week ago

This fix is super helpful! Any chance it can get merged? Happy to try resolving the merge conflicts if you need help.

thanks for putting this back on my radar! I'll work on resolving the merge conflicts

jeffbrennan commented 1 week ago

@fpgmaas @SemyonSinchenko could one of y'all please review the changes after my recent updates?

For the merge conflicts, most of the differences were formatting/line length related and I took the incoming version from main in those instances.

I also experienced some pytest failures after the merge that I have addressed in the subsequent commits.

Let me know if anything should be changed!

fpgmaas commented 1 week ago

@jeffbrennan build is failing due to formatting issues. Could you run the pre-commit hooks locally? Sorry, we should reflect this in CONTRIBUTING.md

jeffbrennan commented 1 week ago

@jeffbrennan build is failing due to formatting issues. Could you run the pre-commit hooks locally? Sorry, we should reflect this in CONTRIBUTING.md

@fpgmaas passing locally after my recent changes

image