Menu
ESC

Nhập từ khóa để tìm kiếm

↑↓ Di chuyển
Enter Mở
ESC Đóng

Đang tải...

Bài 51 — Performance Optimization + Cost Management

Data-Driven Organization Bài 51/60

Data Platform Cost

Cost driver

  • Compute — query time × cost/sec (Snowflake credits).
  • Storage — TB stored × $/TB-month.
  • Egress — cross-region transfer.
  • Vendor seat — Looker license per user.
  • Pipeline run — Fivetran rows synced.

Optimization

  • Query: avoid SELECT *, partition, cluster, materialized view.
  • Storage: lifecycle policy — move cold to cheap tier.
  • Compute: right-size warehouse, auto-suspend.
  • Caching: result cache for repeat query.
  • Pipeline: incremental dbt models, batch larger.

Cost monitoring

  • Tag query by team/project.
  • Dashboard cost by team monthly.
  • Alert on anomaly (10× spike).
  • Quota per team — kick when exceeded.

FinOps practice

  • Showback (visibility) → Chargeback (bill them).
  • Right-size review quarterly.
  • Reserved capacity for predictable workload.
  • Spot/preemptible for batch.

VN concern

  • BigQuery on-demand cheap entry, expensive scale → switch to slots.
  • Snowflake credits hide usage — must monitor.
  • USD billing — depreciation VND impact.

Anti-pattern

  • "Cost is engineering problem only" — business consumer needs to know cost of their query.
  • Ignore cost until shock — invest visibility upfront.

Saving examples

  • Cluster by date → 70% query cost cut for time-series.
  • Materialized view for 100x repeat query → 95% cut.
  • Compress before load → storage 30-50% saved.