Quantcast
Channel: Wrong data types when reading data with spark - Stack Overflow
Browsing index pages (2 articles)

Wrong data types when reading data with spark

if I try to specify the schema when reading a file using pyspark, the read method doesn't fail when the schema is incorrect. For example:schema =...

View Article


Answer by Kafels for Wrong data types when reading data with spark

You can provide mode=PERMISSIVE|FAILFAST|DROPMALFORMED:Scenario 1: Store unprocessed data into another column:schema =...

View Article

Browsing index pages (2 articles)


Latest Images