Engineering Notes

Writing index

SCOPE / CONTAINERS / DISTRIBUTED SYSTEMS / ORCHESTRATION / DATA PIPELINES

1. 01Article

   Why My Spark Container Keeps Exiting — Docker PID 1 and the Daemon TrapOpens in a new tab

   A debugging investigation into why Spark containers exit immediately, how Docker ties container lifetime to PID 1, and why background daemon scripts fail inside containers.

   Topics

   • Docker
   • Spark
   • PID 1
   • Container Runtime

   Platform

   DEV Community

   Read on DEVOpens in a new tabLinkedIn discussionOpens in a new tab
2. 02Article

   Debugging a Multi-Container Airflow Pipeline: Kafka Network Isolation and the YAML Indentation TrapOpens in a new tab

   A practical breakdown of cross-project Docker networking, Kafka advertised listeners, Airflow orchestration failures, and structurally valid but incorrect YAML.

   Topics

   • Airflow
   • Kafka
   • Docker Networking
   • dbt
   • YAML

   Platform

   DEV Community

   Read on DEVOpens in a new tab