中文
Principal Data Engineer · Taiwan · Open to remote

John Tung童曉瑜 · Hsiao-Yu Tung

I've run production on GCP, AWS, and Azure, and shipped real workloads on Airflow — but most of what I do lately is taking something built too heavy and turning it back into a query that runs on a schedule. I pick tools to fit the problem. My background is Chinese text mining and public-opinion analysis; I led a Data Team for 2.5 years — coding throughout — then chose to step back from management to a Principal IC role.

代表作Selected Work
$300 → <$1monthly cost · 1/300

Inherited pipeline rewrite

Took over a revenue pipeline running on Composer + Apps Script and replaced the whole thing with BigQuery External Tables + Scheduled Queries. Same outputs, same SLAs, monthly spend down to ~1/300 of the original. The lesson I keep applying: most "pipelines" don't need an orchestrator — they need a query that runs on a schedule.

15 jobsCloud Run · scheduled

Multi-platform social ingestion

Facebook, Instagram, Threads, and YouTube ingested into BigQuery — 15+ Cloud Run Jobs on staggered schedules, with GA4 site/app traffic added on top. Raw data is append-only, so every fetch's full history is kept (not just the latest state); dbt then shapes it into clean, query-ready tables. Powers the dashboards editorial and marketing open every morning.

2 appsStreamlit · OIDC + ACL

Two production operating dashboards

Built two Streamlit dashboards on Cloud Run with Google login — one for group-wide digital revenue, one for social-media performance. A self-serve permission system — 3-tier roles plus a change audit log — lets managers add and remove members themselves, no redeploy and no engineer in the loop; event tracking feeds product- and feature-adoption analysis.

60d · 0 manualtoken auto-rotation

Threads OAuth 2.0, fully automated

Dual-account OAuth across 8 permission scopes, with a weekly Cloud Run Job that rotates the 60-day token through Secret Manager. Upgraded the old "alert on-call when the token's about to expire, then ask someone to re-authorize by hand" into something fully unattended.

0 backendfrontend-only · no ops

Self-serve OAuth for non-engineers

A pure-frontend authorization tool on Cloudflare Pages. Social-team editors run the whole API authorization flow themselves — no ticket, no pinging an engineer. Static hosting, no backend, near-zero maintenance.

Gemini 2.5BigQuery AI.GENERATE · shipped

LLM-augmented data pipelines

Gemini 2.5 Flash called in-warehouse via BigQuery (AI.GENERATE) — no separate inference service to run. Two things: first, ~33k comments and ~3.2k posts auto-classified by sentiment and topic into two query-ready tables, computing only new rows (old ones aren't re-run, keeping cost down); topic labels match the newsroom's own section taxonomy rather than one generic "finance" bucket (which would swallow 70% of everything), with a cleanup pass to absorb model noise. Second, on-dashboard "AI insights" that read only the current period's numbers to draft a sentiment summary, flag negative hotspots, and suggest an editorial next step — tied to the real data by a strict no-fabrication rule. Both share a cache, so the AI calls stay cheap and flat no matter how many people use it.

768d + evalembeddings · LLM-as-judge

Semantic clustering + RAG retrieval foundation

Vertex embeddings (768-dim multilingual) plus KMEANS group the negative comments, and an LLM names each theme — turning "20% negative" into "here's what they're actually angry about." On top of that, a comment retrieval layer with a weekly LLM-as-judge eval: two-tier quality thresholds, and the job fails if retrieval quality drops, so a code change or a model/data drift can't quietly degrade it. The embedding + retrieval half of RAG, shipped and guarded — the editorial semantic-search archive is the next layer.

4 jobsGA4 Data API · shipped

GA4 web-analytics ingestion

Pulling the group's site and app traffic into the platform through the GA4 Data API — 4 Cloud Run Jobs plus a story-traffic ranking table. The Viewer grant non-engineers approve also runs through a self-serve page on Cloudflare Pages — no engineer in the loop.

SoV + statsbenchmark · cross-BU

Competitor benchmarking + cross-BU analysis

Competitor data pulled via official public APIs (YouTube + Instagram) for topic share-of-voice. External factors — the stock-market index and the national business-cycle signal — are joined to revenue and run through Mann-Kendall trend tests, Sen's slope, seasonal decomposition, and lead-lag correlation. So a number isn't just "up" — it's "up with the tailwind," "up against the trend" (the part that's actually us), or "dragged by the macro."

3 xls → PDFReportLab · auto-typeset

Newspaper-layout PDF generator

A Streamlit + ReportLab tool that parses stock-market data files and auto-typesets them into a print-ready monthly-revenue newspaper page (the actual layout), replacing what used to be done by hand.

25h / 49hfreshness warn / error

Silent failure detection

Data-freshness monitoring (dbt source freshness — warn at 25h stale, error at 49h) plus Cloud Monitoring alerts that scan job-failure logs — so a broken upstream API or a crashed job can't sit unnoticed for days.

+28 / −1diff · merged

openclaw gateway deadlock fixOSS · 2026-05

Tracked down a deadlock (between a SIGUSR1 listener and a dynamic import) that left the gateway unable to restart after a package upgrade — it had failed silently for six days. Confirmed it with production logs and a live reproduction, then sent a surgical one-line fix (PR #84890, merged).

正在做In Flight
Editorial semantic search
Semantic search over the archive — the embedding + retrieval foundation is already live (comment clustering + RAG retrieval); extending it to historical articles, evaluating BigQuery Vector Search vs. self-hosted Elasticsearch so editors can search by meaning.
Migration wrap-up
Retiring the old orchestration — the new architecture is already producing; now decommissioning the last revenue pipeline off the old Composer DAG / Apps Script.
側專案Side Projects
Taiwan real-floor-area transaction database — dbt-duckdb cleans the government transaction record; one shared query core serves it through both a REST API (FastAPI) and an MCP interface (FastMCP), with API keys / usage quotas, self-serve signup, an OAuth dashboard, and a Cloudflare edge guard; a LightGBM property-valuation model (median error ~12%) is built and held ready until there's demand. Live at api / mcp.realping.tw. Exploring data licensing and API / MCP subscriptions as revenue.
技術Stack
Data engineering
BigQuery · dbt (raw / staging / marts) · Cloud Run Jobs + Cloud Scheduler · Airflow · PySpark / Spark · Apache Iceberg · Fluentd · crawler ETL (Scrapy / Selenium)
Cloud
GCP (Cloud Run · Scheduler · Secret Manager · Monitoring · GKE · Compute Engine · Cloud Storage) · AWS (Data Pipeline · S3 · cross-cloud ETL) · Azure
AI / GenAI / LLMOps
Gemini via BigQuery AI.GENERATE · grounded generation · Vertex embeddings + KMEANS · RAG retrieval + LLM-as-judge eval · classical ML (XGBoost / RandomForest / SVM / LightGBM) · Chinese text mining / NLP
Analysis / viz
Streamlit (Cloud Run + OIDC + ACL) · Looker Studio / Data Studio · Superset · Mann-Kendall / Sen's slope / lead-lag · SEO (Sitemap / json-ld)
DevOps / CI-CD
Docker · Cloud Build · GitHub Actions · Ansible · Kubernetes · Poetry / pytest / ruff · Scrum · Jira · GitLab
Languages / database
Python · SQL · BigQuery / MySQL / SQLite / MariaDB · MongoDB · Redis · Elasticsearch
寫作Writing
開放合作Open To
Full-time
Fully-remote full-time, EOR included.
Contract / freelance
Data-platform and pipeline builds on GCP · Airflow · dbt; LLM / RAG adoption.
Advisory / side
Concurrent advisory and technical consulting.