Alerts are a key part of any quality security program. Prior to joining SafeBase, I had worked extensively with both Microsoft Azure and Amazon Web Services at prior jobs. Working at much larger organizations in the past meant that I had a significantly larger budget with which I could spend on security tooling. In particular, I had gotten very familiar with using the native intrusion detection and misconfiguration tools offered by both Azure and AWS.
Microsoft Azure
For Azure, customers can take advantage Microsoft Security Center. It flags things like suspicious logins, unencrypted Azure SQL databases, users with MFA disabled, and more. A good amount of this information is available in the free tier. The paid Azure Defender plan includes several additional features such as Just-In-Time access and endpoint protection for servers. Pricing for the paid plan increases as you create more resources in your environment. While I was on the security team at Jet.com, our security operations center used this tool extensively.
Although Security Center is quite comprehensive, there are still some alerts that a security team might want to manually create for their own risk appetite and circumstances. Fortunately for my team at Jet.com, we actually had a dedicated 24/7 security operations center team that exported all audit logs to a Splunk instance and created a variety of custom alerts on our behalf.
AWS
For AWS, customers can use GuardDuty. GuardDuty is a fully configured, machine based learning IDS for AWS that creates alerts for events such as your EC2 instances communicating with known malicious IP addresses, console logins from suspicious locations, and cryptocurrency mining activity. GuardDuty pretty much requires no configuration in terms of resources monitored and alerts to generate. For this reason, and its relative low cost, it is extremely popular with startups and growth stage companies that want a hands-off intrusion detection system that doesn't require constant maintenance.
I myself used GuardDuty while I was at SeatGeek and found it to be a great product for understaffed security teams. However, despite GuardDuty's intended hands-off design, I wanted to add my own alerts based on rules for Cloudtrail that were unique to my company, such as a notification when a certain user was added to an admin IAM group. Unlike at Jet.com, I didn't have a managed SOC and needed to create and manage these rules myself. A quick Google search showed that this seemed to be a pretty common project. I was able to leverage some of the many tutorials found online to easily deploy a Lambda function to accomplish this.
Google Cloud
Based on my history with Azure and GuardDuty, I was expecting a similar type of offering from Google Cloud. Sure enough, Google Cloud offers something called Security Command Center. Like GuardDuty and Microsoft Security Center, it offers recommendations on fixing misconfigurations, identifies anomalies, and more. Note that like with other AWS offerings, the monthly charge for GuardDuty increases as you create more resources in your environment. There is a 30-day free trial, but no permanent free tier.
As with GuardDuty, I found the tool to be quite good, but also wanted to add my own manually created alerts to supplement the features and alerts available in the Standard tier and output them to Slack. Part of the reason is that the Premium tier, which is required for Event Threat Detection, starts at $25,000. That's quite a price point for a startup like SafeBase.
gSlack
After a few Google searches, I surprisingly didn't find a whole lot of tools that sent Google Cloud audit logs to Slack other than gSlack, which turns out was a great tool for my needs.
gSlack is an open source tool that uses a combination of a Slack bot, a Firebase Cloud Function, and Google Cloud Audit Logs to generate customizable Slack alerts. When you deploy this tool in Firebase, the Makefile uses Pub/Sub to export admin activity logs automatically to Firebase. Rules are configured in the form of Javascript expressions that export a boolean that triggers a customizable Slack message if the expression results in true.
Some potential events that you may want to create dedicated alerts for using gSlack include:
- Audit log for a specific service was disabled
- New service account created
- Privileged IAM policy attached to a user
- New virtual machine created
After a day or two of tinkering and reading through various audit logs, I was able to create several rules that send alerts to a private Slack channel when certain events trigger. I also added Data Access logs to the gSlack Makefile in addition to the Admin Activity logs that are read by default in order to create alerts when Admin Activity logs themselves are accessed.
Limitations
Note that while this tool is great, there are some limitations that you should be aware of before you decide to try it out:
- It's an older repo and was created when Slack still supported legacy API tokens for external apps. For now they still work but there is a strong chance that Slack will disable support for these legacy apps in the near future.
- Some audit logs maybe more complex than others, with useful information often being displayed in non-fixed length JSON arrays. You might have to output quite a bit of text to display the desired information.
- For more complex rules, such as rate limits, or triggers during certain hours of the day, you might need to extend gSlack yourself to allow for this. This could theoretically be done by adding a few new optional parameters to the rule documents.
- Pricing for Firebase is based on number of function invocations. For the SafeBase, environment, the Free Spark plan is sufficient, but more complex environments with lots of GCP activity may result in monthly charges.
Despite these shortcomings, overall I found gSlack to be a great and low-cost addition to our security tool stack and highly recommend it for startup folks looking to increase their security visibility in GCP.
I'd like to conclude this post by giving a shout out to the team at DoiT International for setting up this awesome repo!
