Slack (and Microsoft Teams) revolutionized the way organizations collaborate efficiently, especially in the work-from-home era. At the same time, its adoption enables technical personas such as software engineers and IT and DevOps personnel to share AWS keys over Slack channels with little or no governance. This blog describes the root causes of this issue and demonstrates a fully automated and “on-demand” solution.
What’s the problem?
Slack channels are public by default
Slack allows users to create channels and set their visibility to either private (invite-only members) or public (anyone in the Slack workspace can join at any time, excluding guests). The Slack channel creation wizard sets the channel to public by default unless the user specifies otherwise.
From Slack’s official documentation:
“Public channels promote transparency and inclusivity. Any member of your workspace (but not guests) can view and join a public channel, giving everyone access to the same shared information. Messages or files posted in a public channel can be searched by other members of your workspace.”
“Private channels are for conversations that should not be open to all members. People must be added to a private channel by someone who's already a member of the channel. Messages or files posted in a private channel can only be searched by members of that channel.”
In reality, organizations have hundreds if not thousands of public Slack channels that their workspace members use daily to push the business forward.
Files in public channels are accessible by all workspace members
Any file uploaded by any user to a public channel is now accessible and searchable by all members of the workspace (excluding guests). If I’m a frustrated employee or an attacker taking over an employee’s Slack account, I can easily scrape sensitive information from the company’s Slack workspace by simply searching for files based on sensitive keywords or PII.
Slack’s data retention is forever by default
From Slack’s official documentation:
“By default, Slack will retain all messages and files (including audio and video clips) for the lifetime of your workspace. If you’d like, you can adjust your retention settings to automatically delete messages and files after a set amount of time. You can also allow members to edit the message retention settings for individual channels and direct messages (DMs).”
Slack offers minimal granularity around data retention policies:
Enterprise Grid customers can create specific data retention policies for specific channels, but that requires maintaining these configurations on an ongoing basis as new channels are being created all the time.
A broad application of Slack’s data retention settings can harm business enablement in many ways. Some data will be deleted that should be kept indefinitely, such as:
- Important files used for internal collaboration
- General company guidelines that should be public to all members
- Customer-related information used to drive the business
Using only Slack’s native controls, it’s not possible to retain any of this data without also retaining everything else. As a result, any file uploaded to Slack will remain indefinitely by default, thus unnecessarily overexposing sensitive company information to potential threats.
Employees upload AWS keys to public Slack channels
In practice, employees from R&D, IT, and DevOps departments upload AWS keys to public Slack channels. As a result, these issues coalesce into a significant threat model:
- AWS keys are available to all Slack workspace members (excluding guests)
- By default, these AWS keys remain available forever
- Suspicious employees or attackers taking over employees’ Slack accounts can easily search, download, and use these AWS keys
- From there, unauthorized individuals can access your AWS production environment
The danger of exposed AWS keys
This attack model was involved in a number of high profile data breaches, such as the Twitter breach and more recently with Okta.
The danger of exposed AWS keys
When a bad actor discovers your AWS keys, what’s their next move? It depends on the privileges that those keys confer, but none of the possibilities are good for your business or data security.
Possible outcomes of having your AWS keys obtained by a cybercriminal include:
Data stolen from your AWS environment
Armed with your AWS keys from within your Slack channels, bad actors will be able to access, view, and download your company’s most business-critical information.
A major target would be your customers’ PII (Personal Identifiable Information), which includes details such as their full names, addresses, dates of birth, social security card numbers, and more. This information can easily be used for nefarious purposes, such as identity theft and credit card fraud.
It’s important to note that your customers’ PII being leaked will likely land your company in serious trouble. In most jurisdictions, PII is strictly protected by the law, so you could potentially be facing severe issues when it comes to regulatory compliance, such as legal penalties and fines.
At the very least, you’ll be facing a massive loss of trust from your clients and investors if they see that bad actors were able to obtain customer PII. Such a breach is extremely damaging for your company’s reputation.
Your strategic business information and IP (Intellectual Property) are also an appealing target for bad actors.
This critical data could serve to give your competitors a major leg up, as they can gain the inside scoop regarding your product development process or your future business plans.
For example, they could look at what you’re developing in an attempt to replicate it, or release it before you can. They may also study what your product’s weaknesses are and create their own product which specifically covers those pain points.
The same goes for your business plans, such as your marketing calendar. With this inside info, your competitor could easily steal your ideas or beat you to the punch when you have significant events or milestones on the horizon.
Cryptojacking
Beyond theft of data, bad actors who have obtained your AWS keys could potentially exploit your system for other purposes. For example, they could use your computing power to mine for cryptocurrency, a practice known by the term cryptojacking.
In recent years, bad actors were able to obtain companies’ AWS credentials from their GitHub accounts in just a matter of minutes. They then used those keys to compromise AWS accounts, which were repurposed for mining Monero.
Changes to other access permissions
Once in your AWS environment, bad actors can do whatever they wish regarding access permissions for your company’s most sensitive assets. They could change, grant or delete access permissions for various users - sometimes without you ever becoming aware of this occurring.
This paves the way for them to persistently penetrate your environment, as they can retain access to your assets under the radar. Bad actors can create copies of crucial data, so that you never notice unusual users being given access to specific assets.
All of these potential outcomes illustrate why Amazon is so detailed about how to secure AWS keys, and how to mitigate potential problems if you do end up exposing your AWS keys.
In general, the best practice is to store AWS keys in a dedicated and secure environment. Slack or equivalents are not considered to be a secure environment – these tools are meant to drive communication, security is not a primary focus. Consider the downstream impact of exchanging sensitive credentials over Slack – what's the outcome when those credentials become leaked? There are better platforms to store those keys, such as a Secret Manager, where the solutions are designed for sensitive keys storage in terms of security and control.
How can DoControl help?
On-demand remediation
Once integrated with Slack (any pricing tier is supported), the DoControl platform consolidates all metadata around Slack files (among other objects) and keeps it up to date based on ongoing user interactions (file uploaded, etc). As such, the Assets page clearly exposes all Slack files in a single place enabling quick searching, filtering, exporting, etc.
From there, you can simply take 3 actions:
- Search for “.pem” files
- Select them all
- Click to delete
This will trigger a security workflow deleting all the files and generating logs in DoControl for compliance evidence purposes.
Automated remediation
DoControl’s integration with Slack is essentially a Slack Bot in Slack channels so that any user interaction in such channels will generate a webhook event pushed to DoControl by Slack in near real-time.
These webhook events can trigger security workflows within DoControl based on many various Slack event types:
The event we’re interested in is “File shared” which essentially means a user has uploaded a file to a Slack channel.
Once the event trigger is selected, we can then define a condition to narrow down this workflow to be triggered only when users upload .pem files (representing AWS keys):
Then, we can add several workflow steps:
- Delete the file to remediate the issue automatically
- Notify the security team via Slack webhook with fully customizable message
- Open an incident on Datadog (or any other SIEM/SOAR via https API call)
- Open a ticket on Jira (or any other ITSM via https API call)
The end result: we’ve created a security workflow that automatically identifies and deletes AWS keys from Slack!
Conclusion
The intention of this blog post is to raise awareness on this critical issue that so many organizations face worldwide, and explain our approach to preventing these types of activities that might seem insignificant, but can cause significant harm to the business. Slack is an outstanding collaboration platform that is here to stay. AWS keys will forever be used to access production. Let’s not mix between the two and prevent data breaches all together.
If you’re interested in seeing this in action, we’re more than happy to demonstrate.
FAQ
How should I keep my AWS keys secure?
There are a number of steps you can take to keep your AWS keys safe from bad actors.
- Don’t generate access keys for your AWS root users. Instead, Amazon recommends that you create an administrative user, within the AWS IAM Identity Center, in order to facilitate daily administrative tasks and operations.
If you have already created access keys for root users, Amazon recommends that you remove those keys. - Have a strictly controlled environment, like a Secret Manager, available for sharing AWS keys. Having this secure medium available will help ensure that access keys are not shared via insecure mediums, such as public Slack channels.
- Implement a SSPM solution that will detect and mitigate sharing of AWS keys in Slack channels or other SaaS applications. Even if you have a Secret Manager or other secure environment, people may slip up and share AWS keys elsewhere. Having a SSPM solution that can operate in near real-time to remediate inappropriate sharing of AWS keys is important for quick detection and response.
Why shouldn’t I keep AWS keys in Slack channels?
AWS keys provide bad actors with a way into your company’s sensitive data and business-critical data. Keeping them in public Slack channels greatly increases the risk that a cybercriminal will be able to improperly obtain them, granting them access to your business’ internal systems and data. Because AWS keys give threat actors an easy way to obtain sensitive data, they should only be stored within highly secure environments.
What are the risks of exposed AWS keys?
There are numerous risks that come along with storing AWS keys in an exposed, insecure environment. Bad actors can use your AWS keys to access various parts of your AWS environment, which means they can potentially view, copy, download, or even delete sensitive, business-critical data and information.
This could cause everything from an embarrassing public data leak that will harm your brand reputation and damage consumer and investor trust, to sparking a regulatory investigation to see if your organization was in compliance with data protection regulations.
In short, it’s definitely a huge risk to leave your AWS keys exposed, due to the increased likelihood of experiencing a data breach.
How do I stop my employees from sharing AWS keys in Slack channels?
Employee education and training is critical in this scenario. Your teams likely have no idea how risky it is to share AWS keys in Slack channels, and are doing so in order to streamline and speed up their workflows and business operations. It’s crucial that you educate your employees regarding exactly how high-risk this practice is, as well as offer them secure alternatives (such as Secret Managers) where they can store AWS keys.
Can Slack automatically stop AWS keys from being exposed?
The short answer to this question is no. Unfortunately, Slack cannot automatically prevent an AWS key from being posted within a channel. However, you could potentially use a bot to help quickly remedy this situation. You can ask the bot to recognize when AWS keys are posted (e.g. when a user shares a .pem file). From there, the bot can inform you or the user who posted that they are creating a security risk by sharing the key. DoControl’s Slack integration works on this principle, with the added ability to automatically delete the AWS key once detected and take other remediation actions.
