Pandas UDFs in Apache Spark
One of the reasons I’ve preferred Scala for working with Spark, is the ability to define complex logic in a UDF without as big of a performance penalty as Python UDFs. (Aside from the performance risk of UDFs in general.)