RAW TO RELIABLEby Lee YaoData Engineer & Analytics Systems Builder
Menu

01

Hero

RAW TO RELIABLEby Lee Yao

Data Engineer & Analytics Systems Builder

I turn fragmented inputs into validated systems and decision-ready data.

Reliable pipelines, tested models, and analytics-ready outputs built with Python, SQL, Snowflake, dbt, and modern orchestration.

Raw inputs → Validated systems → Trusted decisions

Based in New York.

Python / SQL / Snowflake / dbt / Airflow

View Selected WorkView Resume
RAW TO RELIABLE / SYSTEM PATH / FIG. 01
Raw inputs become validated systems, trusted outputs, and usable decisions.

02

Currently Building

Systems in motion.

A current ledger of platform work, product development, and focused technical study.

  1. 01In Development

    SEC Financial Data Platform

    Quarterly SEC filing data transformed into validated, modeled, and monitored financial datasets.

  2. 02Active Build

    Job Search OS

    A local-first workflow for job ingestion, evidence-based scoring, duplicate detection, and application tracking.

  3. 03Current Study

    Snowflake Engineering Practice

    Deepening Snowflake platform knowledge and preparing for the SnowPro Core certification.

03

Selected Work

RAW TO RELIABLE / ARCHITECTURE AS EVIDENCE

Selected Systems, documented.

Three data engineering studies organized around flow, modeling, quality, and useful outputs.

01

Batch Data Platform

DATASET: 2024 Q1

PROJECT 01 / Batch Data Platform

SEC Financial Data Platform

An end-to-end data platform for ingesting, validating, modeling, and monitoring SEC company financial records.

PythonS3SparkSnowflakedbtAirflowGreat ExpectationsPandasSQL

Focus: Financial ingestion · Validation · Modeling · Monitoring

SEC PLATFORM / ARCHITECTURE / FIG. 02
Layered ingestion, validation, and financial modeling flow.
02

Streaming Architecture

2024

PROJECT 02 / Streaming Architecture

CMS Healthcare Streaming Pipeline

A streaming-oriented healthcare data pipeline using Kafka, Spark, Airflow, AWS S3, and Snowflake to produce validated analytics-ready datasets.

KafkaSparkAirflowAWS S3Snowflake

Focus: Streaming ingestion · Distributed processing · Data quality

View on GitHub
CMS PIPELINE / STREAMING / FIG. 03
Streaming-oriented processing into governed analytics datasets.
03

Analytics Engineering

2024

PROJECT 03 / Analytics Engineering

Olist Commerce Analytics Platform

An analytics engineering platform that transforms more than 100,000 Olist e-commerce records into modeled datasets for revenue, seller performance, delivery, customer satisfaction, and operational reporting.

PythonSnowflakedbtPrefectPower BICortex LLMNeo4j

Focus: Dimensional modeling · Seller performance · Customer analytics

OLIST ANALYTICS / MODELING / FIG. 04
Dimensional models organized for e-commerce reporting.
View on GitHub

04

Job Search OS

RAW TO RELIABLE / LOCAL-FIRST WORKFLOW / HUMAN REVIEW

Job Search OS

A local-first job-search workflow for importing job descriptions, reviewing duplicates, scoring role fit against stored evidence, and tracking applications.

JOB SEARCH OS / INTERFACE / FIG. 05
A restrained workspace for ingestion, evidence review, and application tracking.

One workflow, with human review at every step.

Human review remains part of every decision. The system does not automatically submit job applications.

View Case Study

05

Engineering Notes

RAW TO RELIABLE / DEBUGGING / SYSTEMS / DOCUMENTATION

Debugging systems, documenting the reasons.

Practical notes on data pipelines, Docker, Spark, Kafka, orchestration, and the failures that reveal how systems actually work.

View All Writing

06

Let’s Connect

RAW TO RELIABLE / OPEN CHANNEL / NEW YORK

Have a data engineering role, technical problem, or project worth discussing?