Code42-Tips-From-the-Trenches-Red-Teams-and-Blue-Teams

Tips From the Trenches: Red Teams and Blue Teams

In my most recent post, I wrote about the important role proactive threat hunting plays in a mature security program. Equally important to a well-designed program and closely related to hunting for threats is having a robust red team testing plan. Having a creative and dynamic red team in place helps to “sharpen the knife” and ensure that your security tools are correctly configured to do what they are supposed to do — which is to detect malicious activity before it has advanced too far in your environment.

“ It is much more challenging to build and maintain defensible systems than infiltrate them. This is one of the reasons why red team exercises are so important. ”

Red teams and blue teams

A red team’s mandate can range from assessing the security of either an application, an IT infrastructure or even a physical environment. For this post, I am referring specifically to general infrastructure testing, where the goal is to gain access to sensitive data by (almost) any means necessary, evaluate how far an attacker can go, and determine whether your security tools can detect or protect against the malicious actions. The red team attackers will approach the environment as if they are an outside attacker.

While your red team assumes the role of the attacker, your blue team acts the defender. It’s the blue team that deploys and manages the enterprise’s defenses. While the red team performs their “attack” exercises, there are many things your blue team can learn about the effectiveness of your company’s defenses — where the shortfalls are and where the most important changes need to be made.

Defining success

Before conducting a red team test, it helps to decide on a few definitions:

1. Define your targets: Without specifying what the critical assets are in your environment — and therefore what actual data an actual attacker would try to steal — your testing efforts will not be as valuable as they could be. Time and resources are always limited, so make sure your red team attempts to gain access to the most valuable data in your organization. This will provide you the greatest insights and biggest benefits when it comes to increasing defensive capabilities.

2. Define the scope: Along with identifying the data targets, it is essential to define the scope of the test. Are production systems fair game or will testing only be done against non-production systems? Is the social engineering of employees allowed? Are real-world malware, rootkits or remote access trojans permitted? Clearly specifying the scope is always important so that there aren’t misunderstandings later on.

How tightly you scope the exercise includes tradeoffs. Looser restrictions make for a more realistic test. No attacker will play by rules. They will try to breach your data using any means necessary. However, opening up production systems to the red team exercise could interrupt key business processes. Every organization has a different risk tolerance for these tests. I believe that the more realistic the red team test is, the more valuable the findings will be for your company.

Once you define your scope, make sure the appropriate stakeholders are notified, but not everybody! Telegraphing the test ahead of time won’t lead to realistic results.

3. Define the rules of engagement: With the scope of the test and data targets well defined, both the red team and the blue team should have a clear understanding of the rules for the exercise. For example, if production systems are in scope, should the defenders treat alarms differently if they positively identify an activity as part of the test? What are the criteria for containment, isolation and remediation for red team actions? As with scope, the more realistic you can make the rules, the more accurate the test will be, but at the potential cost of increased business interruption.

Making final preparations

Don’t end the test too quickly. A real attacker who targets your organization may spend weeks or even months performing reconnaissance, testing your systems and gathering information about your environment before they strike. A one-day red team engagement won’t be able to replicate such a determined attacker. Giving the red team the time and resources to mount a realistic attack will make for more meaningful results.

It’s also important to precisely define what success means. Often a red team attacker will gain access to targeted resources. This should not be seen as a failure on the part of the blue team. Instead, success should be defined as the red team identifying gaps and areas where the organization can improve security defenses and response processes — ultimately removing unneeded access to systems that attackers could abuse. A test that ends too early because the attacker was “caught,” doesn’t provide much in the way of meaningful insights into your security posture. An excellent red team test is a test that is comprehensive.

It’s important to note that defenders have the harder job, as the countless daily news stories about breaches illustrate. It is much more challenging to build and maintain defensible systems than infiltrate them. This is one of the reasons why red team exercises are so important.

Completing the test

Once the test is complete, the red team should share the strategies they used to compromise systems, and gain access or evade detection with the blue team. Of course, the red team should be documenting all of this during the test. Armed with this information, the blue team can determine how to harden the environment and create a bigger challenge for the red team during the next exercise.

We have a fantastic red team here at Code42. The team has conducted multiple tests of our infrastructure, and we have always found the results to be incredibly valuable. Any organization, no matter the size, can gain much more than they risk by performing red team testing.

As always, happy threat hunting!

Code42 Tips From the Trenches- Threat-Hunting Weapons

Tips From the Trenches: Threat-Hunting Weapons

When it comes to cybersecurity, too many enterprises remain on a reactive footing. This ends up being a drag on their efforts because, rather than getting ahead of the threats that target their systems, they spend too much of their time reacting to security alerts and incidents within their environments.

While being able to react to attacks quickly is important for any security team, it’s also important to get out in front of potential risks to identify threats lurking within your systems before they become active.

In this post, we’ll explain how threat hunting within one’s environment can help to break that reactive cycle and improve the effectiveness of any security program.

“ You don’t need a large security organization or any special security tools to start to proactively threat hunt; any security team can start threat hunting, and often using the tools they already have. ”

Threat hunting defined

Before going forward, let’s first take a step back and define what we mean by threat hunting. Essentially, threat hunting is the proactive search for evidence of undetected malicious activity or compromise. These threats can include anything from remote-access tools beaconing to an attacker’s command and control server to malicious actions of an employee or other trusted insider.

Threat hunting is essential for effective security for many reasons. First, defensive security technologies such as intrusion detection/prevention systems and anti-malware software will never successfully identify and block all malware or attacks. Some things are just going to get through. Second, by finding malware and threats that made it past your defenses, you’ll be able to more effectively secure your systems and make your environment much harder for attackers to exploit. Finally, getting adept at finding threats in your environment will improve your organization’s overall ability to respond to threats and, as a result, over time dramatically improve your security posture.

Your arsenal

Because threat hunting entails looking for things that have yet to trigger alerts — if they ever would trigger alerts, to begin with — it is important to look deeper for evidence of compromise. Fortunately, you don’t need a large security organization or any special security tools to start to proactively threat hunt; any security team can start threat hunting, and often using the tools they already have.

For instance, many of the data sources used in threat hunting will be found in firewall, proxy and endpoint logs. While these sources of data probably aren’t alerting on anything malicious, they still hold a considerable amount of security data that can point to potential indicators that an environment has been breached under their radar.

Other readily available tools are helpful for threat analysis, such as Bro (https://www.bro.org/), RITA (https://github.com/activecm/rita), or OSQuery (https://osquery.io/). These tools will help provide additional visibility into network and endpoint data that could provide insights into potential compromise. With these tools, teams can monitor internal network activity, such as virus outbreaks and lateral movements of data. Monitoring East-West network traffic in addition to what is moving through the firewall provides critical insights to the overall health of your network.

The investigation capabilities of Code42 Next-Gen Data Loss Protection (DLP) can be extremely helpful for threat hunting, for determining how widespread a file is distributed in the environment, and to give information about file lifecycle, all of which provide context around whether a file is business-related or suspicious. For example, with Code42 Next-Gen DLP, you can search by MD5 hash or SHA-256 to find all instances of a sensitive file in your organization, or determine if known malware has been detected in your organization.

New tools and new ways of thinking may seem overwhelming at first. However, threat hunting doesn’t have to be all-consuming. You can start with committing a modest amount of time to the hunt, and incrementally build your threat hunting capability over weeks and months to find malicious files and unusual activity. Also, as a direct benefit to your security program you will be able to eliminate noise in your environment, better tune your security tools, find areas of vulnerability and harden those areas, and enhance your security posture at your own pace.

Now, get hunting.

Tips-From-the-Trenches--Enhancing-Phishing-Response-Investigations

Tips From the Trenches: Enhancing Phishing Response Investigations

In an earlier blog post, I explained how the Code42 security team is using security orchestration, automation and response (SOAR) tools to make our team more efficient. Today, I’d like to dive a little deeper and give you an example of how we’re combining a SOAR tool with the Code42 Forensic File Search API — part of the Code42 Next-Gen Data Loss Protection (DLP) product —  to streamline phishing response investigations.

A typical phishing response playbook — with a boost

Below is a screenshot of a relatively simple phishing response playbook that we created using Phantom (a SOAR tool) and the Code42 Forensic File Search API:

We based this playbook on a phishing template built into the Phantom solution. It includes many of the actions that would normally be applied as a response to a suspicious email — actions that investigate and geolocate IP addresses, and conduct reputation searches for IPs and domains. We added a couple of helper actions (“deproofpoint url” and “domain reputation”) to normalize URLs and assist with case management.

You may have noticed one unusual action. We added “hunt file” via the Code42 Forensic File Search API. If a suspicious email has an attachment, this action will search our entire environment by file hash for other copies of that attachment.

“ Combining the speed of Code42 Next-Gen DLP with the automation of SOAR tools can cut response times significantly. ”

What Code42 Next-Gen DLP can tell us

Applying Code42 Next-Gen DLP to our playbook shortens investigation time. The “hunt file” action allows us to quickly see if there are multiple copies of a malicious file in our environment. If that proves to be true, it is quick evidence that there may be a widespread email campaign against our users. On the other hand, the search may show that the file has a long internal history in file locations and on endpoints. This history would suggest that the file exists as part of normal operating procedure and that we may be dealing with a false alarm. Either way, together the Code42 Next-Gen DLP API and its investigation capability give us additional file context so our security team can make smarter, and more informed and confident decisions about what to do next.

Applying Code42 Next-Gen DLP to other threat investigations

This type of “hunt file” action does not need to be limited to investigating suspected phishing emails. In fact, it could be applied to any security event that involves a file — such as an anti-virus alert, an EDR alert or even IDS/IPS alerts that trigger on file events. Using Code42 Next-Gen DLP, security staff can determine in seconds where else that file exists in the environment and if any further action is necessary.

Combining the speed of Code42 Next-Gen DLP with the automation of SOAR tools can cut response times significantly. That’s something any security team can appreciate.

As always, happy threat hunting!

Architecting-IAM-for-AWS-with-Okta-Code42_Blog

Tips From the Trenches: Architecting IAM for AWS with Okta

In the last year, Code42 made the decision to more fully embrace a cloud-based strategy for our operations. We found that working with a cloud services provider opens up a world of possibilities that we could leverage to grow and enhance our product offerings. This is the story of how we implemented identity and access management (IAM) in our Amazon Web Services (AWS) environment.

IAM guiding principles

Once the decision was made to move forward with AWS, our developers were hungry to start testing the newly available services. Before they could start, we needed two things: an AWS account and a mechanism for them to log in. Standing up an account was easy. Implementing an IAM solution that met all of our security needs was more challenging. We were given the directive to architect and implement a solution that met the requirements of our developers, operations team and security team.

We started by agreeing on three guiding principles as we thought through our options:

1.) Production cloud access/administration credentials need to be separate from day-to-day user credentials. This was a requirement from our security team that aligns with existing production access patterns. Leveraging a separate user account (including two-factor authentication) for production access decreases the likelihood of the account being phished or hijacked. This secondary user account wouldn’t be used to access websites, email or for any other day-to-day activity. This wouldn’t make credential harvesting impossible, but it would reduce the likelihood of an attacker easily gaining production access by targeting a user. The attacker would need to adopt advanced compromise and recon methods, which would provide our security analysts extra time to detect the attack.

2.) There will be no local AWS users besides the enforced root user, who will have two-factor authentication; all users will come through Okta. Local AWS users have some significant limitations and become unwieldy as a company grows beyond a few small accounts. We were expecting to have dozens, if not hundreds, of unique accounts.  This could lead to our developers having a unique user in each of the AWS environments. These user accounts would each have their own password and two-factor authentication. In addition to a poor end-user experience, identity lifecycle management would become a daunting and manual task. Imagine logging into more than 100 AWS environments to check if a departing team member has an account. Even if we automated the process, it would still be a major headache.

Our other option was to provide developers with one local AWS user with permissions to assume role in the different accounts. This would be difficult to manage in its own way as we tried to map which users could connect to which accounts and with which permissions. Instead of being a lifecycle challenge for the IAM team, it would become a permissioning and access challenge.

Fortunately for us, Code42 has fully embraced Okta as our cloud identity broker. Employees are comfortable using the Okta portal and all users are required to enroll in two-factor authentication. We leverage Active Directory (AD) as a source of truth for Okta, which helps simplify user and permission management. By connecting Okta to each of our AWS accounts, users can leverage the same AD credentials across all AWS accounts — and we don’t need to make any changes to the existing IAM lifecycle process. Permissioning is still a challenge, but it can be managed centrally with our existing IAM systems. I will describe in greater detail exactly how we achieved this later in the post.

3.) Developers will have the flexibility to create their own service roles, but will be required to apply a “deny” policy, which limits access to key resources (CloudTrail, IAM, security, etc.).  As we were creating these principles, it became clear that the IAM team would not have the bandwidth to be the gatekeepers of all roles and policies (how access is granted in AWS). Developers would need to be empowered to create their own service roles, while we maintained control over the user access roles via Okta. Letting go of this oversight was very difficult. If not properly managed, it could have opened us up to the risk of malicious, over-permissioned or accidental modification of key security services.

Our initial solution to this problem was to create a “deny” policy that would prevent services and users from interacting with some key security services. For example, there should never be a need within an application or microservice to create a new IAM user or a new SAML provider. We notified all users that this deny policy must be attached to all roles created and we used an external system to report any roles that didn’t have this deny policy attached.

Recently, AWS released a new IAM feature called permission boundaries. The intent of permission boundaries is similar to that of our deny policy.  By using permission boundaries we can control the maximum permissions users can grant to the IAM roles that they create.  We are planning to roll this out in lieu of the deny policy in the very near future.

Example of a role found without the deny policy attached

Implementing Okta and AWS

When thinking through connecting Okta and AWS, we were presented with two very different architectural designs: hub and spoke and direct connect. The hub and spokedesign leverages an AWS landing account that is connected to Okta. Once logged in to this account, users can switch roles into other AWS accounts that are authorized. The direct connect design we implemented creates a new Okta application icon for each AWS account. Users access their accounts by visiting their Okta homepage and selecting the account they want to use.

Power users tend to prefer the hub and spoke model, as this allows them to quickly jump from account to account without re-logging in or grabbing a new API token. The more casual users prefer to have all accounts presented on one page. They aren’t swapping among accounts, and it isn’t fair to ask them to memorize account numbers (or even exact short names) so they can execute an assume role command. In addition to user experience, we considered how easy it was to automate management once a new account has been created. The two approaches each have merit, so we decided to implement both.

When a new account is created, it is bootstrapped to leverage the hub and spoke landing account. Automation can immediately start working with the account, and certain power users get the access they need without any IAM intervention. The IAM team can revisit the account when convenient and stand up the direct connection to Okta. New Okta features, currently in beta, will improve this direct connect process.

One final thing I would like to touch on is how we leverage the Okta launcher to get API tokens in AWS. One of the benefits of having local users in AWS is that each user is given their own API key. While this is a benefit to end users, these keys are very rarely rotated and could present a significant security risk (such as an accidental public GitHub upload). To address this, Okta has created a java applet that generates a temporary AWS API key. The repo can be found here. Like many other companies, we have created wrappers for this script to make things as easy as possible for our end users. After cloning the repo, a user can type the command “okta -e $ENV_NAME” and the script will reach out to Okta and generate an API key for that specific AWS account.  The users do need to know the exact environment name for this script to work, but most power users that need API access will have this information.

No matter where your company is on the path to leveraging a cloud service provider, IAM is a foundational component that needs to be in place for a successful and secure journey. If possible, try to leverage your existing technologies to help improve user experience and adoption. I hope the principles we shared here help you think through your own requirements.

Code42-Tips-from-the-Trenches-Searching-Files-in-the-Cloud

Tips From the Trenches: Searching Files in the Cloud

In a few of my previous blogs, I shared some examples of ways the Code42 security team uses Code42 Forensic File Search to find interesting files — macro-enabled Microsoft Office files, known malicious MD5 hashes and so on. Now that the search capabilities of our newest product have been extended beyond endpoints to include cloud services, such as Google Drive and Microsoft OneDrive, I’d like to look at how we’re using this broadened visibility in our investigations.

“ Because we can now use Code42 Forensic File Search to search for files and file activity across both endpoints and Google Drive, we can be more certain of the locations of sensitive files when we are doing file movement investigations. ”

Finding files – and tracking file movement – in the cloud

Code42 uses Google Drive as a cloud collaboration platform. Because we can now use Code42 Forensic File Search to search for files and file activity across both endpoints and Google Drive, we can be more certain of the locations of sensitive files when we are doing file movement investigations. We combine Code42 Forensic File Search with the Code42 File Exfiltration Detection solution to execute an advanced search — using a given MD5 hash — to find files that have been moved to a USB drive. This allows us to quickly build a complete picture of where a file exists in our environment — and how it may have moved from someone’s laptop to the cloud and back.

What files are shared externally?

Using the latest version of Code42 Forensic File Search, we can also search files based on their sharing status. For example, in a matter of a few seconds, we can search for all Google Drive documents that are shared with non-Code42 users. This shows us all documents that have been intentionally or inadvertently shared outside of the company. A deeper look at this list helps us identify any information that has been shared inappropriately. As with all searches within Code42 Forensic File Search, these investigations take only a few seconds to complete.

Here’s a hypothetical example: Let’s say the organization was pursuing an M&A opportunity and we wanted to make sure that confidential evaluation documents weren’t being shared improperly. We could use Code42 Forensic File Search to pull up a list of all documents shared externally. Should that list contain one of the confidential M&A evaluation documents, we could look more closely to determine if any inappropriate sharing occurred.

Continually finding new use cases

Code42’s ffs-tools repository on GitHub now includes several new searches that take advantage of our new cloud capabilities. You can find them all here.

Like most organizations, we use many cloud services to perform our day-to-day work. That’s why in the near future, we plan to expand the search capabilities of Code42 Forensic File Search across even more cloud services — giving you even greater visibility into the ideas and data your organization creates, no matter where they live and move.

Happy threat hunting!

Tips from the Trenches: Multi-Tier Logging

Tips From the Trenches: Multi-Tier Logging

Here’s a stat to make your head spin: Gartner says that a medium-sized enterprise creates 20,000 messages of operational data in activity logs every second. That adds up to 500 million messages — more than 150 GB of data — every day. In other words, as security professionals, we all have logs. A lot of logs. So, how do we know if our log collection strategy is effectively meeting our logging requirements? Unfortunately, a one-size-fits-all logging solution doesn’t exist, so many leading security teams have adopted a multi-tier logging approach. There are three steps to implementing a multi-tier logging strategy:

“ A one-size-fits-all logging solution doesn’t exist, so many leading security teams have adopted a multi-tier logging approach. ”

1. Analyze your logging requirements

A multi-tier logging strategy starts with analyzing your logging requirements. Here’s a simple checklist that I’ve used for this:

Who requires access to the organization’s logs?

  • Which teams require access?
  • Is there unnecessary duplication of logs?
  • Can we consolidate logs and logging budgets across departments?

What logging solutions do we currently have in place?

  • What is the current health of our logging systems?
  • Are we receiving all required logs?
  • Have we included all required log source types?
    • Do we need public cloud, private cloud, hybrid cloud and/or SaaS logs?
  • How many events per second (EPS) are we receiving?
  • How much log storage (in gigabytes) are we using now?
  • What are our logs of interest?
    • Create alerts and/or reports to monitor for each.

What time zone strategy will you use for logging? 

  • How many locations are in different time zones across the organization?
  • Will you use a single time zone or multiple time zone logging strategy?

How much storage capacity will be needed for logging for the next 3-5 years?

Do we have a log baseline in place?

  • Where are our logs stored now?
  • Where should they be stored in the future?

Are we collecting logs for troubleshooting, security analysis and/or compliance?

  • What are our compliance requirements?
    • Do we have log storage redundancy requirements?
    • What are our log retention requirements?
    • Do we have log retention requirements defined in official policy?
  • What logs do we really need to keep?
    • Identify those that are useful.
    • Drop those that are not.

2. Digest log information

After all of this information is gathered, it’s time to digest it. It’s important to align your logging infrastructure to log type and retention needs — so you don’t end up inserting a large amount of unstructured data that you will need to be able to quickly search in an SQL database, for example. Most organizations have multiple clouds, many different devices that generate different log types and separate required analysis methods. In other words, one solution usually does not meet all logging needs.

3. Implement multi-tier logging

If, after analyzing your logging requirements, you find that one logging strategy does not meet all of your requirements, consider this tiered logging flow:

Code42 Tiered Logging Flow Example

In this example logging flow, there are three different logging flow types and five different log repositories. There are SIEM logs, application logs and system log flow types. The repositories are the SIEM database, ELK (elasticsearch, logstash and kibana) stack, two long-term syslog archival servers and cloud storage. The repositories each have a unique role:

  • The SIEM correlates logs with known threats.
  • The ELK stack retains approximately 30-60 days of logs for very fast searching capabilities.
  • The two syslog archival servers store the last three to seven years of syslog and application logs for historical and regulatory purposes. One syslog archival server is used for processing logs, the other is a limited-touch, master log repository.
  • Cloud storage also stores the last three to seven years of logs for historical and regulatory purposes.

Simplify your log activity

This is just one quick example of an innovative solution to simplifying log activity. Regardless of whether multi-tier logging is the right solution for your organization, the most critical step is making sure you have a clearly defined logging strategy and an accurate baseline of your current logging state. This basic analysis gives you the understanding and insights you need to simplify log activity — making it easier to accomplish the complex logging goals of your organization.

Code42 13 Tips for Situational Awareness

Tips From the Trenches: 13 Situational Awareness Questions

A key aspect of responding to security events is situational awareness: knowing what is happening in your environment and why. Standard data security tools like firewalls, proxies, email filters, anti-virus reports and SIEM alerts are all common sources of data for situational awareness. However, it’s also important to have visibility into business operations. Only with a holistic view of your entire organization can you have true situational awareness.

For example, as a software company, writing and deploying software is a significant and complex part of our business operations. Naturally, this work is supported by development, test and staging environments, which are used by our engineers to create and test product features. Security teams need to be aware of all non-production environments in their organizations. Open non-production environments (or environments that re-use credentials between production and non-production systems) can be a vulnerability that attackers can exploit.

“ No matter what business your organization is in, you should know where your important data can be found as well as what activities are normal and what is not normal. ”

Asking questions is the key to knowledge. Here are 13 questions I have used to help paint a full view of internal operations at Code42. They are divided into four separate categories based on major categories of concern for most organizations. I hope they will help you improve your situational awareness and overall data security.

Development Environments:

  1. Where are your development environments?
  2. Do you have the appropriate level of logging in those environments?
  3. How is access handled and are there controls that prevent the reuse of credentials across environments?
  4. Are there forgotten dev environments that need to be cleaned up?

Build Process:

  1. Where is your code built?
  2. Where is your code stored?
  3. If somebody maliciously inserted code into your environment, would you be able to detect who, when and what?
  4. Where are your build/CICD servers?

Deployments:

  1. Do you know what your typical deploy schedule is?
  2. Are you involved in the change management process and other governance bodies so you know when major changes are occurring in your environment?

Decommissioning:

  1. What systems and environments are going away?
  2. Is there a plan to keep information such as logs from those environments after the environment itself goes away, in accordance with your data retention policies?
  3. Will any infrastructure be reused, and if so, has it been processed properly?

While these questions are specific to software development and deployment, the data security issues they raise are relevant to businesses of all types. No matter what business your organization is in, you should know where your important data can be found as well as what activities are normal and what is not normal. Ensuring that tools are in place to answer these questions is vital.

Here’s one tool I use to answer these questions in our environment: Code42 Forensic File Search. It provides the visibility I need into all activity in our organization. With it, we can quickly and accurately take stock of data movement, data security risks and countless other activities. It makes it easier and faster to know what is happening in our environment and why. It provides the situational awareness that is critical for any modern organization.

Until next time, happy threat hunting!

Why Local Deduplication Is the Key to Faster Restores

Tips From the Trenches: Hunting Endpoint Threats at Scale

A big part of “walking the talk” about proactive data security here at Code42 is our “Red Team vs. Blue Team” internal simulations. Today, I’d like to share a few ways I’ve used the Code42 Forensic File Search API to give me completely new threat-hunting capabilities during these exercises.

Endpoint devices are still one of the big blind spots for the modern threat hunter. It’s often nearly impossible to search files on endpoints that are offline or were reimaged due to an incident. This is one reason I’m so excited about the Code42 Forensic File Search API: it doesn’t suffer from this limitation; it truly sees every version of every file on all endpoints, whether online or offline. And since we use our backup product, we also have every file that ever existed.

“ Leveraging Code42 Forensic File Search, I’m able to identify potentially unwanted applications that have slipped past antivirus and other traditional security tools. ”

Locating EXE files in download directories

Leveraging Code42 Forensic File Search, I’m able to identify potentially unwanted applications that have slipped past antivirus and other traditional security tools. To find these previously undetected threats, I’m forwarding output from the Code42 Forensic File Search API (hashes) to the VirusTotal Mass API for further enrichment. Here are some of the high-value searches I’ve used within Code42 Forensic File Search, along with the corresponding JSON files for reproducing the searches in your environment:

  • Search all macro-enabled Word documents
  • Search all DLL files in download directories
  • Search all Dylib files
  • Search all DMG files in download directories

Parameters for customizing FFS search results

Once you have your raw JSON results, here are a few parameters I’ve found useful in customizing Code42 Forensic File Search queries:

  • fileName:The fileName parameter can take a wildcard with a file extension at the end to list all DLL files in this example:   {“operator”:”IS”,”term”:”fileName”,”value”:”*.dll”},
  • filePath:Another useful parameter for searches is the filePath parameter, especially when you are searching for filetypes typically found in specific locations. The example below captures the Windows download directory of all users, as well as all paths below the downloads directory — hence the two wildcards: {“operator”:”IS”,”term”:”filePath”,”value”:”c:/users//Downloads/“}

Hash-check best practice

After you have configured your JSON file, the Code42 Forensic File Search search results should look something like this: 

Python ./ffs_search.py –username –search_type raw –in_file ./hunt.json –out_filter md5 | awk ‘!seen[$0]++’ | tr -d ‘”, []’ | sed ‘/^\s*$/d’

With an output that appears below:

Code42 Security Tips from Trenches Hash-check

Piping the results to awk and tr simply removes duplicate MD5 hashes and cleans up the JSON output, so you avoid the cost of submitting the same MD5 hash to a service like VirusTotal multiple times. Once we have the hashed file results, we can search those hashes across any threat intel or data enrichment tool.

One quick note: The public VirusTotal API key is rate-limited to four queries a minute. I would recommend using a private API key, since searching across hundreds of unique hashes can take quite a long time.

Code42 Security Tips from Trenches Hash-check 2

In our case, we leveraged Virustotal-api-hashcheck to give us a user-friendly view of the hashes we’re seeking. There are many VirusTotal API tools on GitHub and you can use whichever one suits your use case.

Finding malicious files—examining your exposure

In my example, while searching for Excel documents, we uncovered one malicious result that ties back to a document lure that contained a zero-day exploit being used in a targeted attack as discovered by icebrg. You can read more about the specifics of the file on their website.

Code42 Security Tips from the Trenches Hash Analysis 3

I then took the VirusTotal results and searched back in FFS to determine the extent of our exposure. Fortunately, the malicious file was only on two researchers’ systems, and we confirmed that they had been using the file for analysis and demonstration purposes.

Code42 Security Tips from Trenches Forensic File Search

Leveraging Code42 Backup + Restore for file analysis

I’ve also leveraged Code42 to recover unknown files for automated (sandbox) or manual analysis. In the previous example, there was one Excel document that VirusTotal didn’t recognize:

Code42 Security Tips from Trenches Backup Restore

Instead of uploading a potentially sensitive file to VirusTotal, I can do initial triage and analysis by recovering the file with the Code42 application and uploading it to my sandbox analysis tool. Below is a screenshot of the XLSM file running in a sandbox:

Code42 Security Tips from Trenches Virus Total

After doing initial triage and analysis, the file looks safe and not sensitive. At this point, the file could be uploaded to VirusTotal or kept private.

I hope this article has given you a few ideas of how you can use the Code42 Forensic File Search tool to gain powerful new threat-hunting capabilities in defending your organization. Since I first began using the tool, I’ve continually discovered new ways to gain greater visibility in detecting threats. I hope you’re as excited as I am about the current and future ways that security teams can leverage Code42 Forensic File Search internally to enhance security at scale.

Happy threat hunting!

Tips From the Trenches: Choosing a Security Orchestration Tool

Tips From the Trenches: Choosing a Security Orchestration Tool

Like most of our customers, we here at Code42 are constantly looking to enhance our efficiencies when it comes to security. As we use more technology in our environment, that means more log sources, more events and potentially more alerts. It also means we have more opportunities to gather information from disparate sources and put together a more complete picture of the events we do investigate.

Five ways security orchestration tools can help

To help simplify and automate those activities, we are turning towards security orchestration tools. There are many reasons to invest in an orchestration tool. But for us, the following five items are the most important:

  1. Case management: As our team has grown, delegating work and tracking who is working on what becomes increasingly important. An orchestration tool can ideally function as that single workspace for assigning, managing and closing tasks.
  2. Metrics: Closely related to the first item on our list, better management of workload can improve visibility into key metrics like SLAs, as well as make it easier to identify bottlenecks and improve efficiency in analyst workflows.
  3. Integration: We’re constantly testing and adding new security tools, so it’s critically important that an orchestration tool easily integrates with tools we not only are using now but also may add in the future. The less time we have to spend developing integrations, the more time we have for investigating anomalies.
  4. Automation: Of course, automation is the name of the game when it comes to an orchestration tool. Automation allows our team to dream up new ways to streamline data collection and enrichment. Automation also can find connections that we may miss when manually reviewing data.
  5. Value: Analyst time is always in short supply. When a tool does the first four things on this list well, it means our security team can spend less time on low-value work—and more time on important analysis tasks. The more a tool allows us to focus on analysis, the more value it brings to our team.

A page out of the Code42 security orchestration playbook

The right orchestration tool also will allow us to leverage our own Code42 application in exciting new ways. Here’s just one example from the Code42 orchestration playbook:

  • Step 1 – Automatically locate files: To determine the scope of an event and show us how many endpoints have a suspicious attachment, we can search for a specific MD5 hash using Code42 Forensic File Search.
  • Step 2 – Restore deleted files: In situations in which the original file has already been deleted, Code42 Backup + Restore allows us to automatically restore that file.
  • Step 3 – Investigate suspicious files: With all the suspicious files identified (and restored, if necessary), we can now conduct analysis via an orchestration tool—such as running it in a sandbox. Best of all, because we didn’t spend hours or days manually locating and restoring files, we can focus all our time on the critical analysis.

This really is just the tip of the iceberg when it comes to use cases for security orchestration tools—whether it’s leveraging Code42 functionality or any of our many other security tools. As we continue our investigation into security orchestration tools, we’ll share more useful integrations and some automation playbook ideas.

Stay tuned for more updates—and as always, happy threat hunting!

Finding Files in the Wild: From Months to Hours

Every day, your organization faces a variety of data security challenges. Many come from outside your organization, but a significant number also come from within. There are countless reasons why someone may take sensitive data from your organization, many of which are purely opportunistic. For example, what if a file with sensitive financial information is mistakenly emailed to an entire company? That may prove too tempting an opportunity for some. How can your organization respond when this happens? In this post, I’ll discuss how the response process often works today—and how it can be streamlined with Code42 Forensic File Search.

A true story

Here’s a true story of an IT team working through just such a challenge: At this organization, the HR team used Microsoft Excel for management of financial information such as bonus structures and payout schedules. By mistake, a member of the team sent an email containing an Excel file with compensation information for the entire staff to the whole company, instead of the select few who were supposed to receive it. Over 6,000 employees worldwide received the email.

Fortunately, the most sensitive information was contained on a hidden tab in the Excel file, and most employees never even opened the file. The IT team was able to recall the email, but the legal team needed to know who in the company had downloaded and opened it, in case the information within was ever used in a lawsuit. The IT and Security teams were tasked with finding every copy of the file in the organization.

A painful two-month process

While recalling the email cut the number of potential endpoints to search to around 1,000, the IT team still had to search all those devices—many of which belonged to individuals at the organization’s international offices. The IT team used a Windows file searching utility to crawl the user endpoints in question, searching for the name of the file. However, Outlook’s email client can scramble names of files, so the IT team also had to scan for any Excel file in the Temp folder of each machine, and open those files to visually confirm that it wasn’t the file in question.

Each scan would take between one and eight hours, depending on the size of the drive—and the scan could only be run when the target endpoint was online. If a laptop was closed during the scan, the process would have to be restarted. If a device was located in an international office, the IT team would have to work nights in order to run the scan during that office’s working hours.

The process was a tremendous hit to productivity. The IT team tasked fully half its staff to running the scans. Two of the organization’s five security team members were tasked with overseeing the process. Even the legal team’s productivity was affected. Since the IT team had to open every version of the file to verify the sensitive financial data within, the legal team had to draw up non-disclosure agreements for every person working on the project.

All told, the search for the mistakenly distributed financial file took the organization two months, and the IT team estimated that they had only recovered 80 percent of the instances of the file.

“ With Code42 Forensic File Search, administrators can search and investigate file activity and events across all endpoints in an organization in seconds. ”

A better way: Code42 Forensic File Search

Fortunately, there is a better method for locating critical files in an organization. With Code42 Forensic File Search, administrators can search and investigate file activity and events across all endpoints in an organization in seconds. In the case of this Excel file, the IT team could have used Code42 Forensic File Search to search for the MD5 hash of the file. By searching for the MD5 instead of the file name, Code42 Forensic File Search would locate all instances of the file across all endpoints, including versions that had been renamed in the Temp folder or renamed to intentionally disguise the file. This single search would find all copies of the file, even on endpoints that are offline.

The feature video demonstrates Code42 Forensic File Search in action. The IT team member that shared this story is confident that it would have played out very differently with Code42 Forensic File Search. “Had we had Code42 Forensic File Search deployed, that project was probably done in a couple hours,” he said. “We would have cut two months to a couple hours.”

Facebook Twitter Google LinkedIn YouTube