SE Radio 741: Sahil Walia on Schema Changes and Data Migrations

On this episode

About this episode

Sahil Walia, a senior technical architect at Snowflake, joins host Robert Blumen for a conversation about schema migration and data migration. Together, they discuss the basics of schema: do all data sets have schema?; what drives schema change?; are data migration and schema migration the same thing? Must they happen at the same time? They then talk about the challenges and failure modes of schema migration, including challenges at scale, migrating without downtime, the concept of "safety" in data migrations, and safety patterns for data migration - double writes, read repair, replication, ending on the Apache Iceberg case study.