ETL

Question 1

A company needs to transform CSV files to Parquet format stored in S3, apply data cleansing and automatically catalog the schema. Which service provides this ETL capability with LOWEST operational overhead?

Accepted Answer

AWS Glue with ETL jobs and automatic Data Catalog

Answer

EMR cluster with custom Spark scripts

Answer

Lambda functions processing S3 files

Answer

EC2 with Python scripts running transformations

Question 2

An organization stores data in multiple formats (JSON, Parquet, CSV) in S3. The analytics team needs to run ad-hoc SQL queries without provisioning infrastructure. Which service enables this with HIGHEST cost-effectiveness?

Accepted Answer

Amazon Athena with serverless queries on S3

Answer

Permanently provisioned Redshift cluster

Answer

EMR cluster with Presto for interactive queries

Answer

RDS PostgreSQL with Foreign Data Wrapper to S3

Question 3

An application generates streaming logs that must be transformed, compressed and delivered to S3 in near real-time without managing servers. Which service provides this capability with LOWEST complexity?

Accepted Answer

Amazon Kinesis Data Firehose with integrated Lambda transformation

Answer

Kinesis Data Streams + Lambda + S3 (manual pipeline)

Answer

EMR cluster with Spark Streaming to S3

Answer

EC2 Auto Scaling processing logs and writing to S3

Question 4

A company has S3 datasets in multiple formats without cataloging. The team needs to automatically discover schemas and populate a metadata catalog. Which Glue component performs this with HIGHEST automation?

Accepted Answer

AWS Glue Crawlers that automatically scan and catalog

Answer

Glue ETL jobs with manual cataloging script

Answer

Manually executed Athena CREATE TABLE statements

Answer

Lambda functions parsing files and updating Glue Data Catalog API

Question 5

A team needs to run complex big data processing with specific frameworks (Hadoop, Spark, Hive, Presto) with full control over cluster configuration. Which service is MOST SUITABLE for this requirement?

Accepted Answer

Amazon EMR (Elastic MapReduce) with customizable clusters

Answer

AWS Glue with limited PySpark jobs

Answer

Lambda with big data libraries

Answer

Athena with standard SQL queries

Question 6

An organization needs to process lightweight events (< 1MB) in real-time and execute simple transformations before storing in S3. Which solution is MOST cost-effective for this volume?

Accepted Answer

Lambda functions triggered by S3 events or Kinesis

Answer

Permanent EMR cluster processing events

Answer

Glue ETL jobs with minimum DPU

Answer

EC2 t3.micro running processing scripts

Question 7

An S3 data lake contains sensitive PII (personally identifiable information). The company needs granular access control at table/column/row level. Which service provides this with HIGHEST centralized security?

Accepted Answer

AWS Lake Formation with fine-grained permissions

Answer

S3 Bucket Policies with specific prefixes

Answer

IAM Policies with conditions on object keys

Answer

Glue Data Catalog with resource policies

Question 8

Athena queries on S3 are consuming too much time and scan costs. Data is in unpartitioned CSV. Which optimizations SIGNIFICANTLY reduce costs? (Select TWO)

Accepted Answer

Answer

Convert data to columnar Parquet format with compression

Answer

Partition data by frequent query attributes (date, region)

Answer

Increase query timeout in Athena

Answer

Replicate data to multiple S3 buckets

Answer

Change storage class to S3 Glacier

Question 9

A Glue ETL job needs to read data from on-premises JDBC database behind firewall. Which configuration allows Glue to access this data with HIGHEST security?

Accepted Answer

Configure Glue with JDBC connection using VPN/Direct Connect to on-premises

Answer

Expose on-premises database to public internet with IP whitelist

Answer

Replicate database to RDS and connect Glue to RDS

Answer

Export data to files and manually upload to S3

Question 10

A company needs to run SQL analysis on historical data in S3 and recent data in Redshift without duplicating data. Which feature allows querying both without moving data?

Accepted Answer

Redshift Spectrum to query S3 directly from Redshift

Answer

Load S3 data to Redshift with COPY command

Answer

Use Athena separately for S3 and manual queries on Redshift

Answer

Glue ETL job consolidating S3 and Redshift to new location