10-Step Checklist for an Effective Phishing Testing Program

“We don’t feel guilty, really,” a Vietnamese hacking couple shared with BBC News. “We prefer to have a legal job here in Vietnam but the wage is average $300 per month.” The couple had just hacked into the Intercontinental Hotel Group and attempted to deploy ransomware. This is why an effective phishing testing program is crucial for reducing your risk of ransomware and other top cyberattacks.

It all started with a “booby-trapped email,” – a phishing email which the hackers sent to an IHG employee. The employee fell for the ruse and opened the email attachment, which installed malware on the company’s internal network and opened a backdoor for the attackers. The result? IHG hotels around the world (including the Holiday Inn) were disrupted, customers were unable to book stays, and hotel owners suffered huge losses.

Phishing & Vishing Attacks on the Rise

According to the FBI, phishing was the top crime type by victim count in 2022, directly linked to over $52 million in reported losses (undoubtedly an underestimate, given that many attacks, such as ransomware, begin with phishing emails but are classified differently).  Voice phishing (or “vishing”) attacks are on the rise, too, increasing 625% between Q1 2021 and Q2 2022, according to a recent report by PhishLabs and Agari.

Both phishing and vishing are two common types of social engineering attacks, in which adversaries attempt to trick victims into taking an action that will give them unauthorized access to your technology resources. Attackers today use social engineering techniques, including phishing and vishing, to deploy ransomware, capture usernames and passwords, and steal valuable data.

How can you effectively defend against phishing and vishing attacks? Phishing testing is a key exercise which strengthens your “human firewall.” In these tests, security professionals design and launch simulated phishing emails and attacker phone calls, track your users’ response, identify gaps, and provide guidance for policy changes and additional training. All organizations should have a phishing testing program that also includes vishing tests to train users to resist attacks and assess your resistance.   

To watch a phishing and vishing test in action, see LMG’s feature on the TODAY show!

10-Step Checklist for an Effective Phishing Testing Program

1. Raise Awareness Beforehand

Before testing anyone in your organization, ensure that the whole organization has been notified that social engineering testing will be a component of their routine cybersecurity training (this doesn’t mean announcing the precise testing date). Employees should have a clear understanding that testing will be conducted, they should be provided with frequent communications and effective awareness training, and the testing should be designed to test performance based on written, well-communicated policies and procedures.

It’s important to raise employee awareness before testing, so that employees stand a chance of passing the test. On-demand awareness training is a very effective means of educating employees and keeping security top-of-mind. (See LMG’s on-demand awareness training if you would like a fully managed solution.)

You may also wish to publicize tips for identifying suspicious activities. For example, attackers who are conducting social engineering attacks may refuse to give contact information or give bogus contact information. Live callers or in-person attackers may rush, seem nervous, or make small mistakes such as mispronouncing the company name. Check out LMG’s popular tip sheets, including “How to Spot a Phishing Email,” for handy resources!

2. Define the Goals of Your Test

Make sure that everyone from IT staff to the executive management team is on the same page by clearly defining your test goals. Typical goals include:

• • Gain an accurate understanding of organizational vulnerability to social engineering.
  • Educate employees and raise awareness.
  • Identify groups and topics that may require additional training.
  • Fulfill compliance requirements.
  • Find gaps in policies.

Often, organizations test specific policies that relate to authentication or authorization. In certain situations, humans are the sole gatekeepers with the ability to grant or deny access to high-value assets such as computer systems or sensitive information. For example, will employees give out sensitive information over the phone without identifying the caller?

3. Select Your Method for Phishing Testing

Phishing tests can be conducted using an automated platform, a dedicated team that crafts emails specifically for your organization, or some combination of both! For example, LMG’s automated KnowBe4 phishing solution can be configured to send phishing emails to randomly selected employees on an ongoing basis, and you can receive monthly or quarterly performance reports. This system also includes a “report phishing” button for your users’ email, so they can report actual phishing attacks easily, which are then used to inform your testing program.

At the same time, it can be very valuable to have an experienced security professional design and execute a phishing test for your team. This way, you can simulate a spear-phishing or whaling attack or test specific processes and procedures.

When it comes to voice phishing, you’ll want to have a bank of phone callers conduct the test. Consider the languages of your employee pool, as well as their locations and time zones, when selecting callers and scenarios.

4. Select Your Test Time(s) and Length

When selecting your phishing testing (or vishing testing) times, carefully consider factors such as your employees’ standard work hours, time zones, peak business hours, and more. Remember that recipients often share suspicious emails and call information with each other, as well as a central IT or security team. As a result, a phishing or vishing scenario can typically only run for a fairly short period of time in order to produce accurate response statistics. For the same reason, phishing testing work best when only a subset of employees are targeted with the same scenario, so that individual responses can be assessed before a whole group is alerted.

5. Choose A Scenario

Certain scenarios are more likely to trick your employees into falling for a phishing or voice phishing test. Consider whether your goal is to simulate a targeted attack or a more general ruse.

One benefit of automated platforms such as LMG’s KnowBe4 solution is that it includes phishing testing templates based on actual phishing emails reported by users. For customized, manual tests, an experienced professional can help you select a scenario with an appropriate level of difficulty for your staff.

When you test, consider the pretext, urgency and “bait.” Hackers often send messages that include widely familiar names and logos, such as Microsoft, Amazon, or shipping companies. In targeted attacks, the hackers may spoof messages from an executive at your company or a vendor. Be careful and avoid pretexts that are unwise or illegal—for example, in many jurisdictions it is illegal to pose as a federal agent.

Typically, effective phishing testing scenarios include a sense of urgency, such as “Your account will be disabled within 2 hours if you don’t respond!” Hackers may also include “bait”—an enticing prize or valuable asset to lure your employees into responding. How do you choose a good “bait” for a test? In fly fishing, the fly you choose is dictated by where you’re fishing, what species you’re fishing for, what they like to eat, and what food they expect to be available. Similarly, when choosing bait for your social engineering test, consider the location of your targets, their demographics, the items that are popular, the time of year, and the reasonable availability of the item. It’s often a good idea to choose an item which you can later use as a real prize, so that there is positive reinforcement.

6. Set Up the Test

When you set up your phishing test, consider data collection needs, email/Caller ID spoofing, allow listing, and other key factors.

Collecting accurate statistics is critical for maximizing the value of a phishing testing program. At the end of the test, you want to know which users fell for the scam so that you can conduct appropriate follow-on training and reduce risk. It can be very helpful to review detailed responses such as entries in forms and comments from phone conversations in order to pinpoint problem areas and develop solutions.

Always test before the test! You don’t want any surprises on the day of the test. Before conducting a phishing test, make sure that the test emails will get through the spam filters. If they can’t, coordinate with technical staff to configure allow lists. Remember, a social engineering test is designed to test human responses, not your technical defenses. Hackers can work full time to bypass your spam filters, and you want to simulate that during phishing testing.

For phone tests, ensure that your testing team’s Caller ID spoofing systems work appropriately, and that your target phone numbers are correct. By the time you launch, you should know that the test emails will get through filters, that caller ID spoofing works as expected, and that your ability to track responses is up and running.

7. Launch!

Make sure that the testers plan the launch time in advance, and that your IT/security team is appropriate informed (if you intend for them to handle responses in a certain way). It is also important to designate an emergency contact for your testing team, so that if something goes wrong—for example, the test phishing emails get caught in spam filters—there is someone available who can address any issues.

During an email phishing test, expect to receive the majority of clicks and responses within ten minutes of the email being sent. You may also receive visits from targets who were not on the original list. This can happen when employees forward the phishing message. During vishing tests, callers may receive challenges such as “who do you work for?” and “is there a number I can call you back at?” It’s likely that the first few callers will be more amenable to the scenario than later targets who may have already been alerted. Throughout the test, managers, IT support staff and perhaps even executives will receive reports of a phishing emails and phone calls. Keep track of traffic on internal email lists and watch for discussion of suspicious activity.

For phone phishing, you have to decide when to stop making calls. Typically calls should be made until the targeted sample size has been contacted. You may also reach a point when the test is clearly “busted,” at which point it may be appropriate to end sooner, depending on your team’s preferences.

8. Analyze the Results

Analyzing results is fun! Modern phishing test reports typically include information about users that opened the email or clicked on link, corresponding dates and times, and any data they entered into a phishing web site. You can use this data, combined with supplemental information about targets, to determine how employees performed based on location, department, role, or other factors.

Advanced phishing systems such as LMG’s KnowBe4 platform can produce performance reports based on role and geography, compare your users’ performance to industry peers, and even automatically schedule follow-up training for employees that fell for a phishing ruse.

For phone phishing, callers typically take careful notes of user pickups, responses, and results, including statistics that may be broken down by department, geographic region, or other variable.

9. Communicate

It’s important to effectively communicate the phishing testing results with management, HR, and other leadership as appropriate. Typically, managers should review results of employee performance, so that they can respond appropriately by reinforcing good reactions or responding to risky behavior.

Often, executive leadership and sometimes even the board will review high-level results of phishing tests. Visual graphics such as charts and statistics can give your leadership an understanding of the current risk at a glance.

10. Conduct Follow-Up

Remember that when employees fail social awareness training, it is generally not because they are “bad employees” or ignorant. The employees that fail can even be the most diligent employees, who are trying to be as helpful as possible—the best team players! Often, phishing and vishing tests can uncover deficiencies in procedures, such as caller authentication processes. These can be addressed through organization-wide process changes and updated training.

In other cases, the weakness may be due to an individual user that made a mistake or didn’t understand the policy. These issues can usually be addressed through extra training, discussion, and clear incentives.  Many organizations also offer a reward for employees that perform well, such as entry into a raffle or another perk. This can help to ensure that routine phishing tests are seen as a positive opportunity to perform well and protect your organization.

Make sure that after each test, you budget time to develop a follow-up plan and implement it in a timely manner.

Conclusion

Now more than ever, hackers rely on phishing and vishing to bypass your technical defenses and break into your organization. Conducting routine phishing testing is a great way to train your users and accurately assess your risk. Want help? LMG’s expert team can take care of your phishing and vishing tests and provide accurate reporting—contact us today if you need assistance.