Event-driven architecture is one of the patterns in software development for decoupled and distributed services. I am not going to deep dive into what is it or not. But I will try to share some initial info for an existing cloud service in AWS that might empower our event-driven solutions.
Those who follow my posts probably know that I am more interested in Azure. But because learning new things is fun(😍) and because of some other “real-life” requirements, time to time I need to work with other cloud providers. Let’s try to understand our options in the cloud world for building good solutions.
Amazon EventBridge is a fully managed, serverless and scalable event routing service (a.k.a service-bus). It provides easy ways to connect with some applications and with other AWS services. Briefly, it provides us to build solutions with event-driven architectures in the cloud without thinking too deeply. Filtering, transforming message, routing the messages, re-try or archive things are all managed and easy within AWS EventBridge.
Let me share some simple scenarios to make it a little bit clearer why we can need AWS EventBridge for who have never used it before.
For example, sometimes in an e-commerce web site, some items are out-of-stock. When items are out-of-stock, automatically a button appears. And we click that button which is letting us know when items are in stock again. In that case when items are store’s inventory again, the system might notify us automatically.
This is all happens automatically within some flow which is triggered by an action/data with or without human interaction in almost real time. With AWS EventBridge it is efficient, reliably and easy to build this kind of or more complex solutions.
AWS EventBridge has some core blocks.
Event Buses are the services that receive events from some sources and stream them for rules. By default, there is a “default” event bus. There is possible to create some other custom buses, but there are some limitations for them. For example, if you are going to have scheduled rule, it can only be run in “default” event bus. Basically, custom event buses might be preferable for custom applications.
With some additional permissions for Event Buses, it is possible to have events from some other AWS accounts and from some another region.
Rules are like decision units which matches the incoming events to defined pattern. When a rule is defined, there are some settings needed to be done. Some matching pattern is defined so that incoming events can be filtered. For example, a rule can be defined with an AWS S3(a.k.a storage service) pattern so if a S3 is set to publish some events, this rule can match this event according to defined pattern. And with some advanced context filtering it is possible to filter according to the resource name.
It might be a little irrelevant with events, but it is also possible to create scheduled rule with AWS EventBridge. With some scheduled time, settings, it is possible to trigger some other resources. Now there is a new AWS EventBridge Scheduler, it is basically the same with current things, but a little bit advanced in background. But it’s obvious that “naming things” is hard.😁😁😁
Targets are like the execution units for the events’ data. They are defined in Rules. So, if a rule match the event data, it triggers a resource. A resource can be another AWS EventBridge event bus or AWS resource. For example, when a file is uploaded into the AWS S3, another AWS services like AWS Lambda (a.k.a serverless function), EC2(a.k.a VM) can be triggered. There are lots of options for targets in AWS EventBridge rules. A rule can contain more than one target. If the rule matches the event data, the targets are executed in parallel.
As I described above; AWS EventBridge is fully managed event routing service, so that some additional features are provided as built in. For example, with some simple settings it is possible to define re-try policies. Number of hours (default is 24h) to keep unprocessed events or max. re-try number (default is 185) like of settings can be done within Rules.
And within AWS EventBridge, it is possible to archive and re-sent events when it’s needed. It might be thought as an “Event Store” in some ways.
All events need some defined structures to have a reliable data. AWS EventBridge has schema registry support by default. All AWS services events schemas are defined there. So, it is very easy to adopt these schemas for custom built applications. And it is possible to create our own schemas for our own custom events. So, event buses can ingest those events.
Let’s go over with AWS Console to make these a little bit clearer.
For all these AWS resources, I have also a GitHub repo. to play with it. Terraform and related test artifacts are also there. It’s my own playground, it might be a little bit messy but feel free to play with me. 😁🧑🏻💻
As I explain before; we have “default” event bus as built in. We can have additional service buses for our custom events. And we can pass some events from “default” event bus to our custom event bus.
In above picture you will notice that “Schema Discovery” is started, which means that event bus is able to process events. So, if the event bus is not initiated it cannot process the events.
Let’s continue with a rule. When we check the rules from the left menu the following screen will welcome us.
We can see all rules by “Event bus”. And when we click the “Create Rule”, the following wizard will help us to create a rule.
As you can also notice we can define our rule to have an event pattern or not. If we choose “Schedule” type of for the rule, if it will be cron typed rule. But so far let’s move with “Rule with an event pattern”. So, when we fill the required fields and click “Next” the following UI will welcome us.
This UI will be a little bit detailed than the previous one. In here we choose the event source for our rule.
Sample Event part is not related with the rule definition. But it helps us to see existing event schemas and validate our defined event pattern as below.
Within the above section, we are creating the event pattern that is matched by the rule. So when an event is passed into the rule, this pattern will be checked. If it is matched the rule will be invoked and attached targets are triggered.
With pattern form it is more user-friendly to create the pattern. We can choose event source service and some event types and properties.
For example, the following pattern is for AWS S3. So, when ObjectCreated and ObjectDeleted events are published within AWS S3 Notification Events for bucket name “some-bucket-name”, that event is going to match with this pattern and rule is invoked.
To send notifications to AWS EventBridge from AWS S3, there is an setting which is needed to be enabled in AWS S3 properties. So please be aware of this setting.
When we are done with rule pattern we can click Next and continue with target for the rule.
For above target, it is an example for AWS Batch. As target definition, for AWS Batch we just need to define job definition, job queue and job name.
For some other AWS resource there are going to be some other fields. And there are lots of AWS resources exists as targets.
It is also possible to have some other custom event bus as target for the rule.
With some additional settings for the rule, we can configure event input. We can do some configuration like transformation. For example, we can do change some event field or use static values for our requirements.
It is also possible to configure re-try policy and configure dead-letter-queue for unprocessed events. It is good to have these features by default.
So when we done with these settings we can complete the wizard with some tags and review. And now our rule is ready…
With these simple configurations when a new file is created in S3 bucket, an event is going to published and AWS EventBridge’s event bus will get that event. Our just defined rule is going to process the event. If the event is matched with our defined event pattern, the target in the rule is going to be triggered. This was a very simple scenario but good scenarios for event-driven architectures can be implemented with AWS EventBridge.
I just tried to create some ideas about AWS EventBridge, and I hope it helps within your journeys…At least I hope, now you have had more questions in your mind.
Happy coding, see you in next post…