Problem Reporting System Overview

The Problem Reporting System provides functionality to submit, search, and manage problem reports related to the NSI Certification and Verification Program.  It uses a web front-end, which provides access to the repository of Problem Reports relating to the program. The web pages presented to a user of the system are tailored to their privileges on the system, which are controlled using their web authorization by a username and password.  When a Problem Report is submitted, a clear distinction is made between the public and confidential information provided by the submitter. When searching, viewing and reviewing problem reports, only the Certification and Verification Authority (CVA) reviewers will see the confidential information.

The problem reporting system is accessed using a series of URLs that can be bookmarked in a browser.  However, the most natural starting points are the actions of searching the repository for existing problem reports and submitting new problem reports.

Access to the system requires privileges on the hosting web server. Privilege levels are granted based on group or access list memberships on the server. For example, users in one of the reviewer lists may review problem reports using the web interface, but users who are not in a reviewer list may only search for public information on problem reports.

The roles in the problem reporting system are listed below. In the list, SA is the Specification Authority (the group of people who develop and provide interpretations of the specifications), and CVA is the Certification and Verification Authority (the group of people who manage the certification and verification program):

Life-Cycle of a Problem Report

The Problem Reports tracked in the system go though the life-cycle described below. This description covers those parts of the life-cycle that have software support, though this does not include all the possible states in the Problem Reporting and Interpretations process.

Member Action - Raise a Problem Report

All members can submit problem reports. Submitters provide their name, contact, company, and business practice or technology information, which are all kept confidential. Submitters also provide information on the problem, including a description and their own classification of the type of problem (see below), and references to any test suites or specifications involved. This information is public and will be visible to all users of the Problem Reporting system.

When a member first submits a problem report, it enters a review state based on the submitter's classification of the problem. If the classification is a System Deficiency (SD) a CVA review is started; otherwise, if the classification is an Interpretation (INT) request then a SA review is started. The appropriate reviewers will receive a notification email indicating that the Problem Report has been submitted and the review will start. Regardless of the classification of the problem report, the CVA and SA reviewers will always be notified when new reports are submitted.

Reviewer Action - Discuss a Problem Report in a Review

All users in the review group for a given type of review may join in the discussions on the Problem Report using the web interface and the URL provided to them in the review notification email. Comments will be accumulated in the review, but will only be visible to users of the particular review group, not to reviewers in other review groups (e.g. the CVA cannot see the SA review comments).

Responder Action - End a Problem Report Review

After reviewers have reached a conclusion about a Problem Report, one of the responders will provide the official response to the review and this closes the review. This response is the only information concerning a review that is publicly visible. The purpose of a SA review is to provide an opinion to the CVA so the CVA may provide a formal response to the submitter. So, if the review is a SA, a CVA Review is then automatically started (and SA Reviewers notified). If it is a SA Review, then the CVA is notified.

CVA Responder Action - Notify submitter of preliminary review outcome

After the preliminary CVA review of a Problem Report, the CVA notifies the submitter using the web interface. The review may not have been conclusive, and in which case a CVA Detailed Review will automatically be started. If the review was conclusive, the CVA will update the state of the Problem Report indicating the resolution, which will cause a notification email to be sent to the submitter.

CVA Action - Conduct a further Review

For Problem Reports that are still inconclusive after the preliminary review, the CVA will perform a further review, using the same process as is used for the preliminary reviews. A further review cannot return an inconclusive outcome. Either the final resolution is provided, or if the outcome is still not finalized, then an interim resolution must be provided. Any number of further reviews may be carried out.

Submitter Action - approve or appeal resolution

The submitter must respond by email to the CVA to indicate whether they agree to the resolution of their Problem Report, or not, in which case, the submitter may appeal.

CVA Responder Action - Publish Problem Report or mark as Appealed

The CVA will update the Problem Report to reflect the submitter's response. If the response is acceptance of the resolution, the CVA will publish the Problem Report; otherwise it will be marked as appealed.

CVA Responder Action - Notify submitter of appeal outcome

If a submitter has appealed the resolution of a Problem Report, the appeals process defined in the NSI Certification/Verification Policy will be used to resolve the issue. This process is not tracked in the Problem Reporting system. However, once the appeal outcome is known, the Certification and Verification Authority will notify the submitter and publish the Problem Report with the final outcome.

Please note, if the system is configured for Working Group Only Resolution, reviews or actions by Certification and Verification Authority will be eliminated. Once a final resolution is given by the Working Group, the system will automatically reflect the result without Certification and Verification Authority intervention.