Automating Code Review Tools – CAT.NET – Part II

The other day in Part I of this series I said I was going to write up some more articles about automating code review tools in your build process. Today I spent some time looking over CAT.NET and figuring out how it fits into my build process. CAT.NET is a static code analysis tools that analyzes data flow through a .NET library or executable and tries to identify common security problems such as cross site scripting (XSS) and SQL Injection (SQLi) vulnerabilities. My process is not perfect but it does the job well enough. Remember, you should always augment automated tools with a human review to catch things automation cannot.

CAT.NET Installation

The first thing to do is download the CAT.NET MSI installer. Then either unpack it or do what I did and install it, copy the files from C:\Program Files\Microsoft\CAT.NET to your project directory, and uninstall. I like to keep all tools contained within the svn repositories for a project so they can be utilized on any development computer without having to set it up first.

One funny thing with CAT.NET (CISG take note) is that it has to live in a specific path within the project repository. When it was put in {root.dir}\tools\CAT.NET it failed to find the config files, default rules files, etc. It was looking for them at Microsoft\CAT.NET so I broke with my standards (laziness really) and moved it to {root.dir}\Microsoft\CAT.NET. The tool is much happier there and still available in the build file.

Basics of The Project

The demo project is a website that takes in a cat’s name, tail length, and color and then repeats them back to the page (“You have entered: <name>, <tail length>, <color>”). When the user fills in the textboxes and clicks submit, the page creates a CatContext object, fills it with the values taken from the page, and passes it to a CatEngine object. The CatEngine.GetMyCat() method returns the values and they are put in ASP Labels on the page – lblName.Text = cEngine.GetMyCat().Name, etc etc. The CatEngine and CatContext classes are defined in the ClassLibrary1 project (original eh?) and the website code (Default.aspx) is in the WebApplication1 project.

The important thing to note with this project is that the CatEngine does not do any cleaning or validation on the input or output. This is a clear cut example of a Cross Site Scripting vulnerability (and possibly a SQLi vulnerability if the input were eventually put in a database). Both of these situations are caught with CAT.NET, so it is a good tool to have in your build process.

Project Repository Layout

The following is the layout of the project repository (in the root.dir):

• ClassLibrary1\
  • bin\
  • CatContext.cs – A value object for holding information about a cat (name, tail length, and color)
  • CatEngine.cs – The “business object” that takes in a CatContext object and has a method – CatContext GetMyCat() that returns the object
  • ClassLibrary1.csproj
• Microsoft\CAT.NET\
  • Config\
  • Rules\
  • CatNetCmd.exe
  • Other CAT.NET files (from the download) *
• tools\
  • nant\
    • nant.exe
    • Other nant files
• WebApplication1\
  • Default.aspx – The page with the XSS vulnerability in it
  • Default.aspx.cs – The code behind for the page
  • web.config
  • WebApplication1.csproj
• build.bat – Easy way to run nant – (Tools\nant\NAnt.exe -buildfile:default.build %* -nologo)
• default.build – The build file for the solution
• Solution1.sln

The Build File

The build file is a typical NAnt build file with the following properties and targets

• Properties
  • build.dir – Directory to put the output from the csc task (C# Source Compiler)
  • dist.dir – Directory to put all file that are necessary for a deployment of the application to a webserver (also where the local IIS is configured to look when manually testing the site)
  • artifacts.dir – Directory to hold reports generated from NUnit, CAT.NET, FxCop, and other tools that generate XML and HTML reports
• Targets
  • init – Creates the build.dir and dist.dir
  • clean – Deletes build.dir, dist.dir, and artifacts.dir for a clean environment
  • compile.catengine – Compiles the ClassLibrary1 project into ${build.dir}\ClassLibrary1.dll
  • compile.web – Compiles the website code behind files into ${build.dir}\WebApplication1.dll
    • This target depends on compile.catengine
  • build – Runs all the necessary targets to build the libraries and put them in the build directory
    • Depends on clean, init, compile.catengine, compile.web (in that order)
  • dist – Depends on ‘build’ and copies the libraries and the website pages (Default.aspx) to the ${dist.dir} for deployment
    • Depends on ‘catnet’
  • catnet – Runs the CAT.NET code analysis tool on the libraries and checks for errors. If any are found the build if failed and the distribution package never completes.

The ‘catnet’ Target

This is the bread and butter of the CAT.NET automation. You can get a sense of the command line options we use by running CatNetCmd.exe /?. Here is the code:

<target name=”catnet” description=”Runs the CAT.NET static code analysis tool on a set of libraries”>
<exec program=”Microsoft\CAT.NET\CatNetCmd.exe” workingdir=”.” output=”catnetout.log”>
<arg value=”/file:${build.dir}\ClassLibrary1.dll” />
<arg value=”/file:${build.dir}\WebApplication1.dll” />
<arg value=”/report:${artifacts.dir}\CatNetReport.xml” />
<arg value=”/reportxsloutput:${artifacts.dir}\CatNetReport.html” />
</exec>

<!– Check for errors since the command doesn’t throw error codes –>
<loadfile file=”catnetout.log” property=”catnetout” />
<fail if=”${string::contains(catnetout, ‘ERROR’)}” message=”There was an error running CAT.NET Static Analysis” />
<!– Clean up if we didn’t error out –>
<delete file=”catnetout.log” />

<xmlpeek file=”${artifacts.dir}\CatNetReport.xml”
property=”hasErrors”
xpath=”//Rule[TotalResults>0]/Results/Result/ProblemDescription” />

<fail if=”${string::get-length(hasErrors) &gt; 0}” message=”CAT.NET found at least one problem: ${hasErrors}” />
</target>

This target runs the CAT.NET tool over the libraries produced from the ‘build’ target. It uses the default rule set and outputs the reports to ${artifacts.dir}\CatNetReport.xml and CatNetReport.html. Because the CAT.NET command line tool does not return an error code (CISG again), the build file sends the output to catnet.log file, read it in, and check for the word ‘ERROR’. If that is found, the build is failed because something went wrong with the command itself.

If the command completes successfully it cleans up the standard output file (delete catnet.log) and then performs an XMLPeek on the CatNetReport.xml file. The xmlpeek xpath looks for a <TotalResults>#</TotalResults> where # is greater than zero. Unfortunately the xmlpeek command only pulls the first record it comes across that matches the xpath, but that is enough to know that the build needs to be failed. If even one error is reported from this utility, the build should stop anyway and a manual check of the report shoud be performed. The CI process should include some way of notifying developers of errors like this but for this is an example those targets were not included.

Analyzing the Output

When the build is run (‘build dist’), the output of the catnet target looks like the following:

catnet:

[exec] Application Assurance
[exec] Static Analysis Tool
[exec] 1.0.3272.32340
[exec]
[exec] 12/31/2008 4:13:38 PM:Info : Starting analysis [2 modules]
[exec] 12/31/2008 4:13:38 PM:Info : Analyzing module ClassLibrary1…
[exec] 12/31/2008 4:13:38 PM:Info : Analyzing module WebApplication1…
[exec] 12/31/2008 4:13:39 PM:Info : 2 Cross-Site Scripting issues found.
[exec] 12/31/2008 4:13:39 PM:Info : Analysis completed.
[delete] Deleting file C:\Dev\CatNetExample\catnetout.log.
[xmlpeek] Peeking at ‘C:\Dev\CatNetExample\reports\CatNetReport.xml’ with XPath expression ‘//Rule[TotalResults>0]/Results/Result/ProblemDescription’.
[xmlpeek] Found ‘2′ nodes with the XPath expression ‘//Rule[TotalResults>0]/Results/Result/ProblemDescription’.

BUILD FAILED

C:\Dev\CatNetExample\default.build(70,4):
CAT.NET found at least one problem: A cross-site scripting vulnerability was found through a user controlled variable that enters the application at Default.aspx.cs:27 through the variable stack0 which eventually leads to a cross-site scripting issue at Default.aspx.cs:29.

Total time: 3.1 seconds.

The tool finds an XSS defect and fails the build. There are a few more errors found, but it stops on the first occurance of an error. CAT.NET can be customized to find all kinds of coding problems and it is recommended to customize the configuration for your specific needs. Once it is configured don’t forget to include it in the build process!

Conclusion

CAT.NET is a great tool and it should only get better over time. I sleep better at night knowing that I am catching common security errors, my junior developers (if I had any) learn from their mistakes using automation, and I can cut down the time I spend doing manual code reviews. The next article will look at the use of FxCop to help your project adhere to coding standards. I use the default Microsoft standard set because I’m insane, but you can also customize this tool to your liking.