So you've started using Apache Iceberg and you're loving the features it offers. But have you thought about the cost and performance implications of using Iceberg? In this post we'll look at the new optimization features in AWS Glue that help you manage Iceberg tables.
Explore the newly supported AWS Glue table statistics feature for Apache Iceberg tables. This post provides a high-level overview of how to generate column-level aggregation statistics to significantly accelerate query performance. Follow along to set up your environment, create and load an Iceberg table, and generate valuable statistics that can optimize query execution.
IoT Core, Kinesis Data Streams, Firehose, IoT Rules and Lambda... the list of services that AWS Solution architects will recommend you use to build your IoT data ingestion pipelines. But what if I told you that you could replace all of these services with a single S3 bucket? Without sacrificing any abilities to process, analyze, or visualize your data.
In this post we'll take a look at how to use the new automatic compaction feature in AWS Glue, and how it can help you optimize your Iceberg tables. Use my handy helper script to create an Iceberg table, load sample data into it, and test out this new feature in a hands-on way.
A quick look at how to use partition projections in Amazon Athena accompanied by some hands on examples that cover the basics through to more advanced use cases.
Resolving ICEBERG_VACUUM_MORE_RUNS_NEEDED errors by using AWS Step Functions to retry vacuuming Amazon Athena Iceberg tables until they are clean.