Network Inspection System Enhances Security for Microsoft Security Essentials and the TMG Firewall

by [Published on 2 Feb. 2011 / Last Updated on 2 Feb. 2011]

This article discusses network traffic inspection, specifically Microsoft’s Network Inspection System (NIS) that is now found in both TMG firewall and Microsoft Security Essentials. NIS provides an exceptional level of security for networks that contain Windows servers and client systems.


You have probably heard the old chestnut that says “the only way to secure your network is to pull all the cables out of their Ethernet jacks”. While such an answer is given a bit tongue in cheek, and that wouldn’t work in today’s wireless world anyway, the idea is that if one computer is able to communicate with another computer (or computing device, as tablets and smart phones are making up an increasing percentage of the network connected devices compared to PCs) there is always a chance that one of the machines is going to compromise the other machine. Of course, you don’t necessarily need a wired Ethernet network or even wi-fi to accomplish the compromise, as exploits can also be carried from one device to another using removable media (CDs, DVDs, floppies, USB keys, etc). As for wireless, given the ever-increasing use of wi-fi and cell phone carrier-based “G” (3G, 4G) networks, the future of network attacks is likely to be “over the air”.

Regardless of the medium over which the attacks take place, however, it’s clear that we all need protection against such attacks. In the earlier days of networking, the miscreants and malcontents used simple denial of service attacks to create havoc and interrupt business activities. As the Internet bad guys got more sophisticated (and maybe a bit more egotistical) they began to spend their “valuable” time defacing web sites. Then, when the Internet became more mainstream and more actual business started taking place over the global network, criminal types (individuals and organizations) were able to take advantage of the opportunities to make money off their crimes, and co-opt the skills of hackers to quietly compromise devices so that they could steal data (including businesses’ trade secrets and individuals’ identities) without detection. As networking itself has grown more complex, so have the threats against which we need to protect ourselves.

Network Traffic Inspection

There are a couple of ways you can protect yourself against today’s more sophisticated attackers. One effective method is to put technologies on both the network and on the host operating systems that can inspect the nature of the traffic destined for the host that needs the protection. Network based approaches are typically located on network firewalls that lie on the edge of the network, such as the Forefront Threat Management Gateway (TMG) firewall, and that works well for many purposes.

The problem with network based protections (when used alone) is that they are typically focused on a particular “choke point” on the network – commonly at the network edge or right behind the network edge. For example, the TMG firewall is often placed behind a traditional so-called hardware firewall – this configuration allows the dedicated firewall device to do some basic and simplistic processing of old-style network layer attacks, and this reduces the processing overhead required of the TMG firewall, which is capable of doing much more sophisticated protection than the “hardware” firewall.

Since you can’t cost-effectively put TMG firewalls at every juncture on your network, it’s important that you also put network layer protection on the host operating systems themselves. This gives you protection against both Internet based attacks (typically initiated by the user who tries to access malicious content, since Internet hosts rarely can initiate connections to private network hosts without the hosts being previously compromised) and attacks sourced from other hosts on the same corporate network, where those other hosts have been compromised and then automatically seek to spread an infection or other type of system exploit.

Network Inspection System – Application Layer Inspection

This is where the Network Inspection System (NIS) comes in. NIS is Microsoft’s response to the growing threat of network based attacks. NIS was first introduced with the Forefront Threat Management Gateway (TMG) firewall to enable sophisticated network based IDS/IPS at the edge of the corporate network. Recently, Microsoft extended the significant protection enabled by having NIS on the TMG firewall by including NIS with the most recently released version of Microsoft Security Essentials (version 2.0).

With an increasing number of application layer attacks hitting the scene and new ones being released on a regular basis, Microsoft Research designed the Generic Application-level Protocol Analyzer (GAPA).GAPA includes a protocol specification language and an inspection engine that operates on network streams and captures. GAPA makes it possible to create network protocol parsers faster and reduces the development time required to create the parsers, and these parsers are used extensively by the NIS.

One of the key problems networks security professionals have to deal with is that attackers usually create and launch exploits for disclosed vulnerabilities more quickly than application vendors can deploy security updates. In addition to the time it takes to develop the security updates, you have to factor in the time it takes for most administrators to test these security fixes before deploying them, to realize that networks often go unprotected for a significant amount of time after an exploit becomes known.

This delay leaves computers vulnerable to attacks and exploitation, during a period when the bad guys know all about the exploit and are scrambling to take advantage of it before it’s patched. The Network Inspection System reduces these windows of vulnerability between disclosures and patch deployment from weeks to a few hours. That is a significant improvement and can make the difference between a network that goes on working and one brought down to its knees by a zero day attack.

The vulnerability research and the signature development are done by Microsoft’s Malware Protection Center (MMPC). For security bulletins that fix publicly-unknown vulnerabilities, NIS helps provide immediate protection shortly after the details of the vulnerability become publicly known. The MMPC also rapidly responds to zero day incidents by releasing NIS signatures for them as soon as they are known. At this time, NIS signatures help detect exploits of vulnerabilities in Microsoft products only. While this might be interpreted as a limitation when implemented with the TMG firewall (since the TMG firewall is intended to protect the entire network), it isn’t a problem at all when the NIS is included with Microsoft Security Essentials, since MSE can only be installed on Windows computers.

Types of NIS Signatures

The Network Inspection System uses three types of signatures when performing its IDS/IPS functions:

  • Vulnerability-based. These signatures will detect most variants of exploits against a given vulnerability.
  • Exploit-based. These signatures will detect a specific exploit of a given vulnerability.
  • Policy-based. These are signatures that are generally used for auditing purposes and are developed when neither vulnerability nor an exploit-based signature can be written.

We don’t know at this time which of these signature types is used with Microsoft Security Essentials, as there is no public information available on this currently. We do know that the TMG firewall uses all three types of signatures and they are enabled by default. Note that while the policy-based signatures might not seem to be useful in that they are not providing IPS protection, they do provide IDS functionality so that you’re aware of the possible compromise of a system on your network and you can then initiate your incident response plan as needed.

Application Layer Protocols Supported by NIS

The Network Inspection System can analyze a number of application layer protocols for potential exploit code. While there are literally thousands of application layer protocols in use today, only a small handful represent a significant percentage of all network traffic. Because there are time constraints for any endeavor of this kind, Microsoft has focused on the following popular protocols:

  • HTTP
  • DNS
  • SMB
  • SMB2
  • NetBIOS
  • SMTP
  • POP3
  • IMAP
  • MIME

After reviewing that list of protocols, I think you can agree that these are the most commonly used – and most commonly abused – protocols used on the Internet and on intranets today.

Microsoft evaluates the need for supporting additional protocols on a continuous basis and will add that support as necessary if there is an exploit using some other protocol that needs to be protected against. If additional protocols are enabled, the support for that protocol will be included when the signature update takes place. In most cases, support for a new protocol is added because there is a significant vulnerability or exploit that uses that protocol, and therefore in most cases, the MMPC will also release a signature that uses that new protocol. If you are using a TMG firewall, you can see which protocols the signatures use by grouping the NIS signatures by protocol. The TMG firewall obviously gives you more information and more control, as a network administrator, whereas MSE is designed for the use of consumers and very small businesses and thus is intended to work more transparently.


The Network Inspection System is a network level IDS/IPS system that uses the GAPA language to enable fast development of NIS signatures. NIS inspects network traffic for a collection of the most commonly used protocols on both the Internet and the intranet, and assesses that traffic for potentially malicious code. NIS is currently available with the TMG firewall, where it inspects traffic to and from the Internet, and with Microsoft Security Essentials, where it inspects traffic moving into or out of Windows hosts. NIS depends on the Windows Filtering Platform, which means it’s available for Windows Server 2008 and above and Windows Vista and above. NIS focuses on Windows based vulnerabilities, which makes it the ideal IDS/IPS for Windows hosts. All these features enable both the TMG firewall and Microsoft Security Essentials to provide an exceptional level of security for networks that contain Windows servers and client systems.

See Also

The Author — Deb Shinder

Deb Shinder avatar

Debra Littlejohn Shinder, MCSE, MVP (Security) is a technology consultant, trainer and writer who has authored a number of books on computer operating systems, networking, and security.