A data engineer needs to ingest messages from devices that publish messages to a “lighting” topic and to a “heating” topic. Which AWS IoT Core configuration might they use to store lighting data for weekly reporting?
The correct answer and explanation is:
Correct Answer:
Create an AWS IoT rule that writes data from the “lighting” topic to an Amazon S3 bucket.
Explanation (300 words):
AWS IoT Core enables devices to publish messages to MQTT topics. To process or store these messages, you can create IoT rules, which are powerful mechanisms for filtering and routing data from topics to other AWS services.
In this case, the data engineer wants to ingest messages from devices that publish to “lighting” and “heating” topics, with a specific need to store lighting data for weekly reporting. Since reporting often involves aggregation and analysis of structured or semi-structured data, the best destination is typically Amazon S3, which is a durable, scalable, and cost-effective object storage service well-suited for analytics workloads.
To achieve this, the engineer would:
- Create an AWS IoT rule that triggers when a message is published to the
"lighting"topic. - Use an SQL-based rule query statement like:
SELECT * FROM 'lighting'This filters the messages coming only from the"lighting"topic. - Set the rule action to send the incoming message to Amazon S3. The rule will need:
- An S3 bucket configured to receive the data.
- A proper IAM role that grants permission to write to the bucket.
Storing messages in S3 allows integration with reporting and analytics tools such as Amazon Athena, AWS Glue, and Amazon QuickSight. Weekly reports can be generated by querying this S3-stored data on a schedule.
This configuration separates the concerns: the IoT Core rule handles message ingestion and filtering, while Amazon S3 stores data durably and cost-effectively for downstream analytics.
Meanwhile, a similar rule could be created for the "heating" topic, if necessary, targeting a different destination or processing logic.