CodeSonar Release 4.0, patchlevel 0: Release Notes

• What's New
• Customer Tickets Fixed

What's New

Taint Analysis

CodeSonar now includes a taint analysis that tracks the influence of untrusted data on program execution.

• Source code displayed in the following CodeSonar GUI locations is colored to show tainted values in the code.
• A number of new C/C++ warning classes make use of the taint analysis.
• There are three new built-in taint metrics, all with procedure granularity: Is Taint Sink, Is Taint Source, Propagates Taint. These can be displayed in the CodeSonar visualization tool using the built-in mechanism for metric display.
• The CodeSonar Extension API provides programmatic access to taint analysis functionality for use in creating custom checks or extending the coverage of existing checks.

Management Reports

CodeSonar has new management report functionality. Some key features:

• Predefined templates for hub-wide, single-project, and single-analysis reports.
• PDF, HTML, and XML report formats.
• Generate a report directly from the corresponding Home, Project, or Analysis page.
• Create and modify templates using the management report template editor.
• Import/export functionality for templates.

Management Report Tasks:

• Create a new management report template
• Edit an existing management report template
• Generate a management report

Visualization Tool Changes

We introduce the notion of visualization mode: every visualization tool tab is in either basic mode or warning mode, depending on the context in which it was opened. Warning mode tabs have an additional panel - the Warning Path panel - providing path checking functionality and a link to the warning report.

The format of the information presented in the Info panel has changed. For details of the new format, see Visualization Tool Reference: Info Panel.

Warning Classes

New C/C++ Warning Classes

New Java Warning Classes

There are new Java warning classes corresponding to Contemplate ThreadSafe rules.

Mnemonic Changes

Other Warning Class Changes

A new, more-sensitive variant of the Buffer Overrun class has been added. The variant is disabled by default (regular Buffer Overrun checking is still enabled by default). To enable it, add the following WARNING_FILTER rule to the project configuration file.

WARNING_FILTER += allow class="2$Buffer Overrun"

Expanded Search Functionality

This version of CodeSonar adds support for searching in two new domains: Analyses and Projects.

These new domains are supported by the same infrastructure as the existing domains (Warnings, Files, Procedures, Metrics, Code).

• The Advanced Search and Saved Searches pages now have additional Analyses and Projects tabs.
• The domain/scope menu includes options for these domains where appropriate.
• There is an analysis search language and a project search language.
• There are analysis search results and project search results page types.
• Visibility filters "Visible Analyses" and "Visible Projects" apply to appropriate page types, with visibility defaults for new and anonymous users controlled through the Admin Settings page.

GUI changes

There are a number of GUI changes in this version of CodeSonar.

New GUI Page Types

Modified GUI Page Types

Decision Procedure Refinement

Decision procedure refinement aims to filter out some warnings that cannot occur in practice. CodeSonar uses decision procedure refinement in two contexts:

• The CodeSonar analysis will perform decision procedure refinement on the core path for each warning. Configuration file parameters provide user control over the treatment of warnings that are determined to be unfeasible, the treatment of warnings for which the decision procedure times out, and the timeout itself.
• Decision procedure refinement is also applied during extended path checking.

There are two styles of refinement available: "exact" and "approximate". It can be beneficial to enable both forms of refinement, because the sets of warnings that each can dismiss are incomparable. In this case, the two refinement phases are carried out in sequence.

• In exact refinement, the analysis runs the decision procedure on each warning path, handling procedures and loops by splitting the path into separate segments and checking them independently. Exact refinement is controlled by configuration file parameters DP_REFINEMENT_EXACT, DP_REFINEMENT_EXACT_TIMEOUT, DP_REFINEMENT_EXACT_DISMISS, and DP_REFINEMENT_EXACT_DISMISS_TIMEOUT.
• In approximate refinement, the analysis runs the decision procedure on each warning path, handling procedures and loops by converting their summaries into SMT formulas. Approximate refinement is controlled by configuration file parameters DP_REFINEMENT_APPROXIMATE, DP_REFINEMENT_APPROXIMATE_TIMEOUT, DP_REFINEMENT_APPROXIMATE_DISMISS, and DP_REFINEMENT_APPROXIMATE_DISMISS_TIMEOUT.

The effects of exact refinement differ from those of approximate refinement.

• Exact refinement is less likely to produce false negatives than approximate refinement.
• If timeouts are disabled, exact refinement is more likely to produce false positives than approximate refinement. If they are not disabled, there is no predictable relationship between false positive rates for the two.
• Approximate refinement typically entails larger decision problems, and with more free variables, than exact refinement. The approximate refinement problems can therefore be significantly more expensive to solve than the exact ones.

Contemplate Integration

With additional licensing, the Java analysis can now be extended to include Contemplate ThreadSafe.

• There are new Java warning classes corresponding to Contemplate ThreadSafe rules. Relevant CWE IDs are included in the categories list for each class.
• New build options -enable-threadsafe and --threadsafe apply to the Contemplate ThreadSafe analysis.

New Java Build/Analysis Options

There are several new options available for the Java build/analysis.

This release introduces new configuration parameters JAVA_FLAGS_APPEND and JAVA_FLAGS_PREPEND - in most cases we recommend that you specify Java build options through these parameters, rather than directly in your build command.

Metrics

There are three new built-in metrics, and some changes to metric-related API functions and configuration parameters.

New Built-in Metrics

There are three new built-in metrics, all of which describe taint phenomena.

• Is Taint Sink
• Is Taint Source
• Propagates Taint

Metric Changes

• The the metric property previously labeled "metric name" is now labeled "metric tag".
  • Corresponding plug-in API function names have changed.
    • C: csonar_metric_name() is renamed to csonar_metric_tag()
    • Scheme: codesonar:metric-name is renamed to codesonar:metric-tag
• Configuration parameter METRIC_WARNING_RANK is deprecated in favor of METRIC_WARNING_BASE_RANK. See Rank Computation below for more information.

Compiler Models

All compiler models now define macro __CODESONAR__=1 for CodeSonar projects. Use __CODESONAR__ in preference to __CSURF__.

The following models have been added/extended.

Rank Computation

CodeSonar computes and reports Rank in order to suggest a review order for warnings: warnings with a smaller Rank value before warnings with a larger Rank value.

Rank is a composite of several factors:

• The likelihood that the warning represents a true positive.
• The severity of the problem being warned about; in particular, whether it is security-related.
• The complexity of the warning, with more-complex warnings generally receiving larger Rank values than less-complex warnings - more-complex warnings take longer to review and diagnose, so it is typically preferable to address less-complex warnings first.

Rank is determined as follows:

1. CodeSonar determines the base rank for the warning.
   • For warning classes defined using configuration file BAD_FUNCTION_* rules, the base rank is specified by BAD_FUNCTION_BASE_RANK.
   • For warning classes defined using configuration file METRIC_WARNING_* rules, the base rank is specified by METRIC_WARNING_BASE_RANK.
   • For warning classes defined using the CodeSonar Plug-in API, the base rank is specified as an argument to csonar_create_warningclass() (C) / codesonar:create-warningclass (Scheme).
   • For Java warnings reported by PMD, the base rank is scaled from the PMD priority value.
   • For Java warnings reported by FindBugs^™, the base rank is scaled from the Findbugs rank value.
   • For other built-in warning classes, the base rank is computed internally by CodeSonar.
2. Starting with this base rank, CodeSonar applies all the adjustment rules in $CSONAR/codesonar/py/hub/codesonar/rank_rules.py whose @pattern matches the properties of the warning.
3. The resulting value is the Rank.

You can use the CodeSonar SQL query mechanism to investigate Rank values on the hub. For example, to see the Rank distributions for different warning classes in the analysis with Analysis ID 5:

select cs_warningclass.label_xml, 
       min(cs_warninginstance.rank),
       avg(cs_warninginstance.rank), 
       stddev(cs_warninginstance.rank),
       count(cs_warninginstance.rank) 
  from cs_warninginstance,
       cs_warninginstdata, 
       cs_warningclass 
  where cs_warninginstance.data_id=cs_warninginstdata.id and
        cs_warningclass.id=cs_warninginstdata.warningclass_id and
        analysis_id=5
  group by cs_warningclass.label_xml 
  order by min(cs_warninginstance.rank)

Previously-existing rank-related configuration file parameters have been deprecated in favor of new parameters with the same functionality but more explicit names that clarify their roles in rank computation.

• BAD_FUNCTION_RANK is deprecated in favor of BAD_FUNCTION_BASE_RANK.
• METRIC_WARNING_RANK is deprecated in favor of METRIC_WARNING_BASE_RANK.

Configuration Parameters

New Configuration Parameters

Changes to Existing Configuration Parameters

API changes

API changes are described below.

• New API languages
• Extension API Changes
• Plug-In API Changes
• General Purpose API Changes

[BETA] New API languages

This version of CodeSonar has beta-level support for a further set of API interfaces in C++, C#, Java, and Python. For information, see the notes on API Languages.

Extension API

There are several new functions and prototypes.

Plug-In API

Changes to the C version of the Plug-In API.

Changes to the Scheme version of the Plug-In API.

General Purpose API Changes

AST changes

Unnormalized C/C++ ASTs

New classes

Field changes.

• The following classes have one or more new fields.
  • cc:field
  • cc:routine
  • cc:variable
• cc:for-each : most fields now belong to new class cc:enhanced-for (and thus are inherited by cc:for-each).

Hierarchy changes.

• New class cc:enhanced-for is now the superclass of cc:for-each.

Normalized C/C++ ASTs

New classes

Field changes. The following classes have one or more new fields.

• c:abi
• c:array
• c:builtin-type
• c:cast
• c:class
• c:codesurfer-type
• c:complex
• c:fixed-point
• c:float
• c:imaginary
• c:integer
• c:pointer
• c:ptr-to-member
• c:routine
• c:routine-type
• c:struct
• c:template-param
• c:union
• c:vector-type
• c:void

EDG Upgrade

CodeSonar now uses EDG version 4.8. This provides better C++11 support (fewer parse errors).

There are also some useful new front end options:

new front end options:

• --c++0x, --no_c++0x
• --c++11, --no_c++11
• --cs_ext_declaration_after_statement
• --microsoft_build_number

Customer Tickets Fixed