How to Save PySpark Dataframe to a Single Output File

Save a Spark dataframe to a single output csv file

August 27 2023  ·  6 min

How to save the output of PySpark DataFrame 'show' to a variable

There is no obvious way to save the nicely formatted DataFrame show() string to a variable. But here is how you can do it

Unit testing PySpark code using Pytest

How to unit test PySpark code using Pytest

October 29 2022  ·  12 min