[Spark] Spark on Yarn

---

Spark on Yarn

• Hadoop-Spark Cluster on Docker(M1)
• Web URL
• YARN on Spark test

---

Hadoop-Spark Cluster on Docker(M1)

• 🔗 Hadoop-Spark Cluster

---

Web URL

• 🔗 Namenode: localhost:9870
• 🔗 ResourceManager: localhost:8088

YARN on Spark test

$SPARK_HOME/bin/spark-submit \
--master yarn \
--deploy-mode client \
$SPARK_HOME/examples/src/main/python/pi.py 10

ref

• 🔗 SPARK ON YARN 공식문서
• 🔗 Cloudera 블로그
• 🔗 SPARK on YARN Cluster 참고
• 🔗 YARN 개념 참고1
• 🔗 YARN 개념 참고2
• 🔗 Medium 참고