Worldwide, risk-informed and performance-based analyses are being introduced into fire protection engineering practice, and the commercial nuclear power industry is no exception. In the last 15 years, the U.S. Nuclear Regulatory Commission (NRC) directed a change in its policy to use risk-informed methods, where practical, to make regulatory decisions. As a result of this change, in the area of fire protection, the National Fire Protection Association (NFPA) completed development of the 2001 edition of NFPA 805, "Performance-Based Standard for Fire Protection for Light-Water Reactor Electric Generating Plants."¹ The NRC amended its fire protection requirements in July 2004 to allow existing reactor licensees to voluntarily adopt the fire protection requirements contained in NFPA 805 as an alternative to the existing prescriptive fire protection requirements.² This allows plant operators and the NRC to use fire modeling and fire risk information, along with prescriptive requirements, to ensure that nuclear power plants can be safely shut down in the event of a fire.

This article provides a brief description of the work performed to assess the relative accuracy of fire models for nuclear power plant applications. Ever since the 1975 Browns Ferry fire, there has been a great deal of interest in predicting the effects of fires in nuclear plants. The NRC and plant operators use fire models in probabilistic risk assessment calculations to identify fire scenarios with safety risks. They also use these tools to determine compliance with, or exemptions from, existing fire protection regulatory requirements. To provide the regulator and the plant operators with confidence in the calculation results, NFPA 805 requires fire models to be verified and validated. To this end, the NRC's Office of Nuclear Regulatory Research, along with the Electric Power Research Institute (EPRI) and the National Institute of Standards and Technology (NIST), has conducted an extensive verification and validation (V&V) study of fire models that support the use of NFPA 805 as a risk-informed/performance-based alternative within the NRC's regulatory system.

article continues below UL/ULC listed DecoShield systems provide you with lightweight, attractive, affordable modular cover/support systems for sprinkler piping, hydronics, plumbing, cable, and conduit. DecoShield products are manufactured using an engineered resin of a Class I/A material meeting most model code requirements, and are clean, paintable, easy to install, tamper-resistant, durable, and economical. The DecoShield cover will not dent, crack, rust, or corrode. As an alternative to constructed soffits and suspended ceilings, the DecoShield systems can be installed without disturbing asbestos ceilings, and can be easily installed in occupied premises. We also offer assistance with design, installation, and anchoring problems. For more information, visit www.decoshield.com

The V&V Process
Given the complexity and range of features in current fire models, it is impractical to evaluate the accuracy of every model output. Thus, the NRC and EPRI identified critical fire protection concerns for nuclear power plants, such as the integrity of electrical cables and fire barriers, the effectiveness of smoke removal systems and the movement of smoke and hot gases from compartment to compartment. In all, 13 predicted quantities were chosen, including the depth and average temperature of the hot upper layer, ceiling jet and plume temperatures, the radiant and total heat flux onto walls and "targets," and the major gas species and smoke concentrations.

The study does not cover the entire spectrum of possible fire scenarios, either in nuclear plants or in other types of structures. To clarify its range of applicability, the study examined a variety of nondimensional and normalized parameters that bound the spectrum of scenarios it does cover and recommends that users of the report be aware that scenarios falling outside of these bounds have not been rigorously validated. The final report³ includes a discussion of the limits of applicability of the results of the study.

Also, the validation study used the heat release rate of the experimental fires as an input, rather than a predicted output. The study provides an assessment of the accuracy of current fire models in predicting the transport of a fire's heat and combustion products throughout a compartment. While some of the models evaluated do have the physical mechanisms to predict fire growth and spread (for example, Fire Dynamics Simulator⁴), this study did not include an assessment of those functions. From the standpoint of nuclear power plant safety, it is important to assess how accurately the models predict the transport of energy from a specified fire, because that is how these models are currently used in the nuclear industry. A major finding of this study is that the current generation of fire models predict transport fairly well.

This V&V study began in earnest in 2003, resulting in the seven-volume report, "Verification and Validation of Selected Fire Models for Nuclear Power Plant Applications,"³ (NUREG1824). The report is available to the public on the NRC's Web site (www.nrc.gov/reading-rm/doc-collections/nuregs/staff/). Five of the seven volumes contain individual evaluations of five fire models: (1) the NRC Fire Dynamics Tools (FDTS),⁵ (2) the EPRI Fire-Induced Vulnerability Evaluation (FIVE),⁶ (3) the NIST zone model Consolidated Fire And Smoke Transport (CFAST),⁷ (4) the Electricité de France zone model MAGIC⁸ and (5) the NIST computational fluid dynamics (CFD) model Fire Dynamics Simulator.⁴

Experimental Uncertainty as a Metric for Evaluating Fire Models
NUREG-1824 is based upon ASTM E1355, "Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models."⁹ The guide describes four steps in the evaluation process for a given model: (1) definition of the model and scenarios, (2) assessment of the appropriateness of the model's theoretical basis and assumptions, (3) assessment of the mathematical and numerical robustness and (4) quantification of the uncertainty and accuracy of the model results in predicting the course of events in similar fire scenarios.

It is this last step, model validation, on which the study focuses. This entails comparing model predictions with full-scale fire experiments and quantifying the results. ASTM E1355 provides some guidance for identifying and selecting experiments and measurements, but it does not define explicitly how the results should be quantified. A useful method to quantitatively evaluate the hundreds of point to-point comparisons of predictions and measurements arose through consideration of uncertainty in the experimental measurements. In selecting experiments for the model evaluations, an emphasis was placed on well-documented uncertainties in both the measurement of the 13 parameters of interest, and also the measurement of model inputs, like material properties and the heat release rate of the fire. The combination of the experimental uncertainty associated with both the model input parameters and the measured model outputs served as a benchmark for evaluation of the models. Photographs of a few of the selected experiments are shown in Figure 1 and Figure 2.

An example of the process to determine experimental uncertainty is as follows: Suppose that the uncertainty in the measurement of the heat release rate of a fire was determined to be about 15 percent. According to the McCaffrey, Quintiere, Harkleroad (MQH) correlation,¹⁰ the upper-layer gas temperature rise in a compartment fire is proportional to the two-thirds power of the heat release rate. This means that the 15 percent uncertainty in the measured heat release rate that is input into the fire models leads to a 10 percent uncertainty in the prediction of the upper layer temperature. Combining this with the uncertainty associated with the thermocouple temperature measurement leads to a combined uncertainty in the reported temperature of about 13 percent. In short, the fire model cannot be shown to be more accurate than about 13 percent. If all of the temperature predictions for the five models and 26 experiments are plotted on a single graph, along with the combined experimental uncertainty as seen in Figure 3, a much better picture of model performance results. In some sense, the experimental uncertainty provides the modeler with a very tangible goal – to predict the outcome of a fire to within experimental accuracy.

How Accurate Are the Models?
To simplify the use of experimental uncertainty as a metric for model accuracy, a simple color system was devised to indicate to what extent the model predictions agreed with the experimental measurements. "Green" was used to indicate that a particular model predicted a particular parameter with accuracy comparable to the experimental uncertainty. "Yellow" indicated that the model predictions were clearly outside of the uncertainty bounds, indicating that the difference between model and experiment could not be explained solely in terms of measurement uncertainty. In cases where the model consistently overpredicted the severity of the fire, a ranking of "yellow+" was used to emphasize the point.

For example, the predicted average hot gas layer temperature rise (determined using a simple two-layer reduction method³) from all the models was compared to the experimental measurements (Figure 3). The hand calculation methods showed the greatest deviation and scatter when compared to the measurements, and were rated "yellow+". Both the zone and CFD models showed less scatter and very similar accuracy for the experiments under consideration, and all were ranked "green" for this parameter.

Next, the predicted heat fluxes onto various horizontally and vertically oriented targets were considered (Figure 4). The CFD model, overall, was more accurate for this parameter, even though the zone and CFD models are of comparable accuracy in predicting the gas temperature. Why is the CFD model more accurate in predicting heat flux? The heat flux at a target is dependent on the thermal environment of the surroundings, the details of which the CFD model is inherently better able to predict. Hand calculations and zone models predict average temperatures over the entire compartment, and thus are less accurate in predicting a heat flux to a single point. Nevertheless, all of the models were assessed as "yellow" for this category, merely to indicate to the model user that even though CFD might be more accurate, it is still challenging to predict a heat flux, especially very close to the fire, with any model.

Whereas the CFD model was more accurate in predicting heat fluxes and surface temperatures, the simpler models performed equally well, sometimes better, for plume and ceiling jet temperatures and flame heights. The reason is that hand calculations and two-zone models use well-established correlations for these fire phenomena. A CFD model solves the basic transport equations, making it truly predictive of these quantities, but not necessarily more accurate. And the increased cost of a CFD calculation is substantial. The spreadsheet and two-zone models produce results in seconds to minutes, versus a CFD model which takes hours to days. If hand calculations and zone model results are obtained faster and are as or more accurate than CFD results, why should an engineer use a CFD model? Real fire scenarios can be more complex than the experiments used in this study and may not conform to the assumptions inherent in the hand calculations and zone models. Fire plumes may not be free and clear of obstacles, because fires sometimes occur in cabinets or near walls. Ceilings might not be flat and unobstructed, because duct work, structural steel and cable trays are often present. Although hand calculations and zone models can be applied in these instances, the results require more extensive explanation and justification. Since CFD models can make predictions on a more local level with fewer assumptions, the results are likely to be more applicable in these more complex situations.

Mark Henry Salley, Jason Dreisbach and Kendra Hill are with the U.S. Nuclear Regulatory Commission. (This paper was prepared in part by employees of the U.S. NRC. The views presented do not represent an official staff position.) Robert Kassawara is with the Electric Power Research Institute. Bijan Najafi and Francisco Joglar are with SAIC. Anthony Hamins, Kevin McGrattan and Richard Peacock are with the National Institute of Standards and Technology. Bernard Gautier is with Electricité de France.

References

_{¹NFPA 805, "Performance-Based Standard for Fire Protection for Light-Water Reactor Electric Generating Plants," National Fire Protection Association, Quincy, MA, 2001.
²U.S. Code of Federal Regulations, Title 10, "Energy," Section 50.48, "Domestic Licensing of Production and Utilization Facilities—Fire Protection," Washington, DC, 2005.
³NUREG-1824 and EPRI 1011999, "Verification and Validation of Selected Fire Models for Nuclear Power Plant Applications," Vols. 1-7, U.S. Nuclear Regulatory Commission, Washington, DC and Electric Power Research Institute, Palo Alto, CA, 2007.
⁴McGrattan, K. & Forney, G. "Fire Dynamics Simulator (Version 4) User's Guide," NIST Special Publication 1019. National Institute of Standards and Technology, Gaithersburg, MD, USA, 2006.
⁵NUREG 1805, "Fire Dynamics Tools (FDTs): Quantitative Fire Hazard analysis Methods for the U.S. Nuclear Regulatory Commissions Fire Protection Inspection Program," U.S. Nuclear Regulatory Commission, Washington, DC, 2004.
⁶EPRI TR-1002981, "Fire Modeling Guide for Nuclear Power Plant Applications," Electric Power Research Institute, Palo Alto, CA, 2002.
⁷Jones, W., Peacock, R., Forney, G., and Reneke, P. "CFAST: An Engineering Tool for Estimating Fire and Smoke Transport, Version 5 – Technical Reference Guide," SP 1030, National Institute of Standards and Technology, Gaithersburg, MD, 2004.
⁸Gay, L. & Epiard, C. "MAGIC Software Version 4.1.1: Mathematical Model," EDF HI82/04/024/P, Electricité de France, 2005.
⁹ASTM E1355, "Standard Guide for Evaluating Predictive Capability of Deterministic Fire Models," American Society for Testing and Materials, West Conshohoken, PA, 2005.
¹⁰McCaffrey, B., Quintiere, J. & Harkleroad, M. "Estimating Room Fire Temperatures and the Liklihood of Flashover Using Fire Test Data Correlations, "Fire Technology, 17, 2, pp. 98-119, 1981.}