From 20-21 February, Bojan Sapunov from the Data Team participated in an invite-only, two-day Accelerated Data Lab Workshop intended exclusively for a selected audience consisting of Data Engineers, Cloud Engineers, DBAs, DevOps and Solution Architects from predominantly large enterprises. The sessions deep-dived into the design, development, and architecture of Amazon Aurora with a large focus on how to perform fast migration from the various on-premise databases to the managed cloud DBs (Aurora or RDS).
The session was a valuable opportunity to get hands-on experience through a series of lab exercises led by AWS experts with sample use cases. He got a chance to test Schema Conversion Tool (SCT) and Data Migration Service (DMS) and migrate big databases from MS SQL Server on the stand-alone server to Aurora. He also tried the latest functionalities from Aurora like Aurora Serverless, Backtrack, Autoscaling read replicas, Performance testing, and Insights, Failover, etc and he got a chance to test.
In the end, there was an introduced to the roadmap for Amazon Aurora for 2020.
More detailed agenda covered:
Aurora in-depth overview
Managed database service compatible with MySql or Postgres.
5X the throughput of standard MySQL and 3X the throughput of standard PostgreSQL
Scalability
High-performance auditing
Storage nodes
Parallel query processing
Availability, cloning, failover
Performance: Replicas in multiple AZ and AWS Regions
Security: encryption, row-level security, IAM Authentication integration,
Serverless option and Data API
Focus on Schema Conversion Tool (SCT) and Data Migration Service (DMS)
Migration options: SQL Server, MySQL, Oracle, PostgreSQL, SAP ASE, Azure DB, DB2, MongoDB
Using Schema Conversion Tool
Best practices on troubleshooting DMS tasks
How Amazon migrated 7500 Oracle DBs with 75PB of data on Aurora, DynamoDB and RDS with little or no downtime and what it meant in terms of performance, infrastructure costs and reduction of database administration overhead.
Serverless Aurora
Auto Scaling, Cloning, Backtrace and Failover
Performance tuning and monitoring
Aurora Roadmap for 2020:
Postgres: Max storage 128/256 TB
Postgres: Support for SQL 11
Postgres: Major version upgrade Q2
Postgres: S3 Export in csv (2020)
Postgres: Extract table or schema from snapshot and export in S3 parquet format
Postgres: Enhanced Partition Management
Postgres: Lambda functions from SQL - Q1
Postgres: TimescaleDB extensions
Postgres: Background Job and scheduling
MySQL: Multimaster single region 4 node support
MySQL: Global database
MySQL: Cross account cloning
MySQL: Active Directory Integration
MySQL: Storage tagging
MySQL: Zero downtime restart 5.6 and 5.7
MySQL: Data Activity Stream
MySQL: Support for Sync Lambda
MySQL: Instance R5.24XL
Serverless: Global database
Serverless: Parallel Query expansion
Serverless: Backtrack
Serverless: Fast DDL, Hash joins
Serverless: Long-running transactions
Serverless: Improved wake time performance <10s