[Apache Airflow] Airflow Installation on Docker

---

Airflow on Docker

• Airflow-Docker Installation
• Docker Airflow CLI

---

Airflow-Docker Installation

# make folder and cd
mkdir airflow-docker
cd airflow-docker

# docker-compose.yaml file download(check version)
curl -LfO 'https://airflow.apache.org/docs/apache-airflow/2.1.4/docker-compose.yaml'

# make folders for dags & plugins & logs
mkdir ./dags ./plugins ./logs

# On Linux & Mac OS, the quick-start needs to know your host user id and needs to have group id set to 0
echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env

# .env 파일 * 파일 안의 다음과 같은 내용 생김
# AIRFLOW_UID=501
# AIRFLOW_GID=0

# run database migrations and create the first user account
docker-compose up airflow-init

# cf. airflow-docker은 requires at least 4GB memory
# 잘 설치된 경우 아래와 같은 msg 나옴 
# airflow-init_1       | 2.1.4
# airflow-docker_airflow-init_1 exited with code 0

# run airflow
docker-compose up

# end airflow
docker-compose down

---

Docker Airflow CLI

방법1

curl -LfO 'https://airflow.apache.org/docs/apache-airflow/2.1.4/airflow.sh'
chmod +x airflow.sh

# 컨테이너 bash 환경으로 접속
./airflow.sh bash

방법2

docker container exec -it airflow_airflow-worker_1 bash

방법3

# create user
docker-compose run airflow-worker users create -u zsu -p abcd -f jisu -l park -r Admin -e zsu58@icloud.com

# airflow 명령어 사용하기 위한 venv 구성
# airflow 설치한 폴더로 cd
python3 -m venv airflow1
source airflow1/bin/activate

# install airflow
pip3 install apache-airflow==2.1.4

ref

• 🔗 공식 문서
• 🔗 참고 블로그
• 🔗 가상환경 관련