REMOVE PIICLEAN + WEIGHTAGGREGATELIFECYCLERawraw-bucketTOOLNiFi · raw writeFORMATCSV · ParquetWRITERData engineerQUARTERLY SURVEYQ1 dump with PIIverbatim exportAnonymisedanon-bucketTOOLSQL insertFORMATIceberg · partitionedWRITERData engineerQUARTERLY SURVEYno namesstable household IDStagingstaging-bucketTOOLSQL · notebooksFORMATIceberg · cleanedWRITERData scientistQUARTERLY SURVEYweighted recordsharmonised codesAggregatedagg-bucketTOOLSQL insertFORMATIceberg · indicatorsWRITERData scientistQUARTERLY SURVEYunemployment rateparticipationArchivearchive-bucketTOOLlifecycle policyFORMATcold tier · immutableWRITERData adminQUARTERLY SURVEY5+ years retained五层 Medallion 架构同一份调查数据集的原始、脱敏、暂存、聚合和归档五个层级。每一层是同一数据在不同质量级别的形态,每张卡说明它存什么、谁写入、如何写入。聚合层是焦点,因为业务读的就是它,归档的推进用虚线。在编辑器中打开