Reading and Saving a File to DB using Spark

Reading and storing in a file in Hive using

  • Spark – 3.5.0
  • Scala – 2.13.12
object dfone extends App {

  val spark = SparkSession.builder()
    .appName("test")
    .config("spark.master", "local")
    .enableHiveSupport()
    .getOrCreate()

  val orderDf = spark.read.csv("path.csv")


  spark.sql("create database if not exists ecommerce")

  orderDf.write
    .format("csv")
    .mode(SaveMode.Overwrite)
    .saveAsTable("ecommerce.orders")

  spark.catalog.listTables("retail").show()
}

Leave a comment