Luật Sư X Mixi

Tổng hợp các bài viết thuộc chủ đề Luật Sư X Mixi xem nhiều nhất, được cập nhật mới nhất ngày 17/01/2021 trên website Athena4me.com. Hy vọng nội dung bài viết sẽ đáp ứng được nhu cầu của bạn, chúng tôi sẽ thường xuyên cập nhật mới nội dung Luật Sư X Mixi để bạn nhận được thông tin nhanh chóng và chính xác nhất. Cho đến thời điểm hiện tại, chủ đề này đã đạt được 495 lượt xem.

Có 5 tin bài trong chủ đề【Luật Sư X Mixi】

【#1】Sensory Evaluation Of Food: Principles And Practices, Second Edition (Food Science Text Series)

Food Science Text Series

The Food Science Text Series provides faculty with the leading teaching tools. The Editorial Board has outlined the most appropriate and complete content for each food science course in a typical food science program and has identified textbooks of the highest quality, written by the leading food science educators.

Series Editor Dennis R. Heldman Editorial Board David A. Golden, Ph.D., Professor of Food Microbiology, Department of Food Science and Technology, University of Tennessee Richard W. Hartel, Professor of Food Engineering, Department of Food Science, University of Wisconsin Hildegarde Heymann, Professor of Food Sensory Science, Department of Food Science and Technology, University of California-Davis Joseph H. Hotchkiss, Professor, Institute of Food Science and Institute for Comparative and Environmental Toxicology, and Chair, Food Science Department, Cornell University Michael G. Johnson, Ph.D., Professor of Food Safety and Microbiology, Department of Food Science, University of Arkansas Joseph Montecalvo, Jr., Professor, Department of Food Science and Nutrition, California Polytechnic and State University-San Luis Obispo S. Suzanne Nielsen, Professor and Chair, Department of Food Science, Purdue University Juan L. Silva, Professor, Department of Food Science, Nutrition and Health Promotion, Mississippi State University

For further volumes: http://www.springer.com/series/5999

Harry T. Lawless · Hildegarde Heymann

Sensory Evaluation of Food Principles and Practices Second Edition

123

Harry T. Lawless Department of Food Science Cornell University Stocking Hall, Room 106 14853 Ithaca NY, USA

ISSN 1572-0330 ISBN 978-1-4419-6487-8 e-ISBN 978-1-4419-6488-5 DOI 10.1007/978-1-4419-6488-5 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2010932599 © Springer Science+Business Media, LLC 2010 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expssion of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

vi

Preface

Preface

vii

Peryam, David Stevens, Herb Meiselman, Elaine Skinner, Howard Schutz, Howard Moskowitz, Rose Marie Pangborn, Beverley Kroll, W. Frank Shipe, Lawrence E. Marks, Joseph C. Stevens, Arye Dethmers, Barbara Klein, Ann Noble, Harold Hedrick, William C Stringer, Roger Boulton, Kay McMath, Joel van Wyk, and Roger Mitchell. Ithaca, New York Davis, California

Harry T. Lawless Hildegarde Heymann

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Introduction and Overview . . . . . . . . . . . . . . . . . . . 1.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . 1.1.2 Measurement . . . . . . . . . . . . . . . . . . . . . 1.2 Historical Landmarks and the Three Classes of Test Methods . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Difference Testing . . . . . . . . . . . . . . . . . . . 1.2.2 Descriptive Analyses . . . . . . . . . . . . . . . . . 1.2.3 Affective Testing . . . . . . . . . . . . . . . . . . . 1.2.4 The Central Dogma-Analytic Versus Hedonic Tests . . . . . . . . . . . . . . . . . . . . . 1.3 Applications: Why Collect Sensory Data? . . . . . . . . . . . 1.3.1 Differences from Marketing Research Methods . . . 1.3.2 Differences from Traditional Product Grading Systems . . . . . . . . . . . . . . . . . . . 1.4 Summary and Conclusions . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Physiological and Psychological Foundations of Sensory Function . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 2.2 Classical Sensory Testing and Psychophysical Methods 2.2.1 Early Psychophysics . . . . . . . . . . . . . 2.2.2 The Classical Psychophysical Methods . . . . 2.2.3 Scaling and Magnitude Estimation . . . . . . 2.2.4 Critiques of Stevens . . . . . . . . . . . . . . 2.2.5 Empirical Versus Theory-Driven Functions . 2.2.6 Parallels of Psychophysics and Sensory Evaluation . . . . . . . . . . . . . . . . . . . 2.3 Anatomy and Physiology and Functions of Taste . . . . 2.3.1 Anatomy and Physiology . . . . . . . . . . . 2.3.2 Taste Perception: Qualities . . . . . . . . . . 2.3.3 Taste Perception: Adaptation and Mixture Interactions . . . . . . . . . . . . . . . . . . 2.3.4 Inpidual Differences and Taste Genetics . . 2.4 Anatomy and Physiology and Functions of Smell . . . . 2.4.1 Anatomy and Cellular Function . . . . . . . .

1 1 1 3 4 5 6 7 8 10 13 15 16 17

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

19 19 20 20 21 23 25 25

. . . .

. . . .

. . . .

. . . .

26 27 27 30

. . . .

. . . .

. . . .

. . . .

30 33 34 34 ix

x

Contents

2.4.2 2.4.3 2.4.4 2.4.5

Retronasal Smell . . . . . . . . . . . . . . Olfactory Sensitivity and Specific Anosmia Odor Qualities: Practical Systems . . . . . Functional Properties: Adaptation, Mixture Suppssion, and Release . . . . . . . . . . 2.5 Chemesthesis . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Qualities of Chemesthetic Experience . . . 2.5.2 Physiological Mechanisms of Chemesthesis 2.5.3 Chemical “Heat” . . . . . . . . . . . . . . 2.5.4 Other Irritative Sensations and Chemical Cooling . . . . . . . . . . . . . . . . . . . 2.5.5 Astringency . . . . . . . . . . . . . . . . . 2.5.6 Metallic Taste . . . . . . . . . . . . . . . . 2.6 Multi-modal Sensory Interactions . . . . . . . . . . . 2.6.1 Taste and Odor Interactions . . . . . . . . . 2.6.2 Irritation and Flavor . . . . . . . . . . . . . 2.6.3 Color-Flavor Interactions . . . . . . . . . . 2.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Principles of Good Practice . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . 3.2 The Sensory Testing Environment . . . . . . 3.2.1 Evaluation Area . . . . . . . . . . 3.2.2 Climate Control . . . . . . . . . . 3.3 Test Protocol Considerations . . . . . . . . 3.3.1 Sample Serving Procedures . . . . 3.3.2 Sample Size . . . . . . . . . . . . 3.3.3 Sample Serving Temperatures . . 3.3.4 Serving Containers . . . . . . . . 3.3.5 Carriers . . . . . . . . . . . . . . 3.3.6 Palate Cleansing . . . . . . . . . . 3.3.7 Swallowing and Expectoration . . 3.3.8 Instructions to Panelists . . . . . . 3.3.9 Randomization and Blind Labeling 3.4 Experimental Design . . . . . . . . . . . . . 3.4.1 Designing a Study . . . . . . . . . 3.4.2 Design and Treatment Structures . 3.5 Panelist Considerations . . . . . . . . . . . 3.5.1 Incentives . . . . . . . . . . . . . 3.5.2 Use of Human Subjects . . . . . . 3.5.3 Panelist Recruitment . . . . . . . 3.5.4 Panelist Selection and Screening . 3.5.5 Training of Panelists . . . . . . . 3.5.6 Panelist Performance Assessment . 3.6 Tabulation and Analysis . . . . . . . . . . . 3.6.1 Data Entry Systems . . . . . . . . 3.7 Conclusion . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

36 37 38

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

39 41 41 42 43

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

44 45 46 47 47 49 49 50 50

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57 57 58 59 62 63 63 63 64 64 65 65 66 66 66 66 66 69 72 72 73 74 74 75 75 75 75 76 76

Contents

xi

4 Discrimination Testing . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Discrimination Testing . . . . . . . . . . . . . . . . . . . . . . 4.2 Types of Discrimination Tests . . . . . . . . . . . . . . . . . . 4.2.1 Paired Comparison Tests . . . . . . . . . . . . . . . 4.2.2 Triangle Tests . . . . . . . . . . . . . . . . . . . . . 4.2.3 Duo-Trio Tests . . . . . . . . . . . . . . . . . . . . 4.2.4 n-Alternative Forced Choice (n-AFC) Methods . . . 4.2.5 A-Not-A tests . . . . . . . . . . . . . . . . . . . . . 4.2.6 Sorting Methods . . . . . . . . . . . . . . . . . . . . 4.2.7 The ABX Discrimination Task . . . . . . . . . . . . 4.2.8 Dual-Standard Test . . . . . . . . . . . . . . . . . . 4.3 Reputed Strengths and Weaknesses of Discrimination Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Data Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Binomial Distributions and Tables . . . . . . . . . . 4.4.2 The Adjusted Chi-Square (χ2 ) Test . . . . . . . . . . 4.4.3 The Normal Distribution and the Z-Test on Proportion . . . . . . . . . . . . . . . . . . . . . 4.5 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 The Power of the Statistical Test . . . . . . . . . . . 4.5.2 Replications . . . . . . . . . . . . . . . . . . . . . . 4.5.3 Warm-Up Effects . . . . . . . . . . . . . . . . . . . 4.5.4 Common Mistakes Made in the Interptation of Discrimination Tests . . . . . . . . . . . . . . . . Appendix: A Simple Approach to Handling the A, Not-A, and Same/Different Tests . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79 79 80 80 83 84 85 85 87 88 88

5 Similarity, Equivalence Testing, and Discrimination Theory . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Common Sense Approaches to Equivalence . . . . . . . . . . 5.3 Estimation of Sample Size and Test Power . . . . . . . . . . . 5.4 How Big of a Difference Is Important? Discriminator Theory . . . . . . . . . . . . . . . . . . . . . . 5.5 Tests for Significant Similarity . . . . . . . . . . . . . . . . . 5.6 The Two One-Sided Test Approach (TOST) and Interval Testing . . . . . . . . . . . . . . . . . . . . . . . 5.7 Claim Substantiation . . . . . . . . . . . . . . . . . . . . . . . 5.8 Models for Discrimination: Signal Detection Theory . . . . . . 5.8.1 The Problem . . . . . . . . . . . . . . . . . . . . . . 5.8.2 Experimental Setup . . . . . . . . . . . . . . . . . . 5.8.3 Assumptions and Theory . . . . . . . . . . . . . . . 5.8.4 An Example . . . . . . . . . . . . . . . . . . . . . . 5.8.5 A Connection to Paired Comparisons Results Through the ROC Curve . . . . . . . . . . . . . . . 5.9 Thurstonian Scaling . . . . . . . . . . . . . . . . . . . . . . . 5.9.1 The Theory and Formulae . . . . . . . . . . . . . . . 5.9.2 Extending Thurstone’s Model to Other Choice Tests . . . . . . . . . . . . . . . . . . . . . .

101 101 103 104

88 89 89 90 90 92 92 94 97 97 98 99

105 108 110 111 111 112 112 113 114 116 116 116 118

xii

Contents

5.10

Extensions of the Thurstonian Methods, R-Index . . . 5.10.1 Short Cut Signal Detection Methods . . . . 5.10.2 An Example . . . . . . . . . . . . . . . . . 5.11 Conclusions . . . . . . . . . . . . . . . . . . . . . . Appendix: Non-Central t-Test for Equivalence of Scaled Data References . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

119 119 120 120 122 122

6 Measurement of Sensory Thresholds . . . . . . . . . . . . . . . . . 6.1 Introduction: The Threshold Concept . . . . . . . . . . . . . . 6.2 Types of Thresholds: Definitions . . . . . . . . . . . . . . . . 6.3 Practical Methods: Ascending Forced Choice . . . . . . . . . . 6.4 Suggested Method for Taste/Odor/Flavor Detection Thresholds . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Ascending Forced-Choice Method of Limits . . . . . 6.4.2 Purpose of the Test . . . . . . . . . . . . . . . . . . 6.4.3 Preliminary Steps . . . . . . . . . . . . . . . . . . . 6.4.4 Procedure . . . . . . . . . . . . . . . . . . . . . . . 6.4.5 Data Analysis . . . . . . . . . . . . . . . . . . . . . 6.4.6 Alternative Graphical Solution . . . . . . . . . . . . 6.4.7 Procedural Choices . . . . . . . . . . . . . . . . . . 6.5 Case Study/Worked Example . . . . . . . . . . . . . . . . . . 6.6 Other Forced Choice Methods . . . . . . . . . . . . . . . . . . 6.7 Probit Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Sensory Adaptation, Sequential Effects, and Variability . . . . 6.9 Alternative Methods: Rated Difference, Adaptive Procedures, Scaling . . . . . . . . . . . . . . . . . . . . . . . 6.9.1 Rated Difference from Control . . . . . . . . . . . . 6.9.2 Adaptive Procedures . . . . . . . . . . . . . . . . . 6.9.3 Scaling as an Alternative Measure of Sensitivity . . . 6.10 Dilution to Threshold Measures . . . . . . . . . . . . . . . . . 6.10.1 Odor Units and Gas-Chromatography Olfactometry (GCO) . . . . . . . . . . . . . . . . . 6.10.2 Scoville Units . . . . . . . . . . . . . . . . . . . . . 6.11 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix: MTBE Threshold Data for Worked Example . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

125 125 127 128

140 142 142 143 145

7 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Some Theory . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Common Methods of Scaling . . . . . . . . . . . . . . . . 7.3.1 Category Scales . . . . . . . . . . . . . . . . . . 7.3.2 Line Scaling . . . . . . . . . . . . . . . . . . . . 7.3.3 Magnitude Estimation . . . . . . . . . . . . . . . 7.4 Recommended Practice and Practical Guidelines . . . . . . 7.4.1 Rule 1: Provide Sufficient Alternatives . . . . . . 7.4.2 Rule 2: The Attribute Must Be Understood . . . . 7.4.3 Rule 3: The Anchor Words Should Make Sense . 7.4.4 To Calibrate or Not to Calibrate . . . . . . . . . . 7.4.5 A Warning: Grading and Scoring are Not Scaling

149 149 151 152 152 155 156 158 159 159 159 159 160

. . . . . . . . . . . . .

. . . . . . . . . . . . .

129 129 129 130 131 131 131 133 133 134 136 136 137 137 138 140 140

Contents

xiii

7.5

Variations-Other Scaling Techniques . . . . . . . . . . . . . 7.5.1 Cross-Modal Matches and Variations on Magnitude Estimation . . . . . . . . . . . . . . . . . 7.5.2 Category-Ratio (Labeled Magnitude) Scales . . . . . 7.5.3 Adjustable Rating Techniques: Relative Scaling . . . 7.5.4 Ranking . . . . . . . . . . . . . . . . . . . . . . . . 7.5.5 Indirect Scales . . . . . . . . . . . . . . . . . . . . . 7.6 Comparing Methods: What is a Good Scale? . . . . . . . . . . 7.7 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7.1 “Do People Make Relative Judgments” Should They See Their Previous Ratings? . . . . . . 7.7.2 Should Category Rating Scales Be Assigned Integer Numbers in Data Tabulation? Are They Interval Scales? . . . . . . . . . . . . . . . . . 7.7.3 Is Magnitude Estimation a Ratio Scale or Simply a Scale with Ratio Instructions? . . . . . . . 7.7.4 What is a “Valid” Scale? . . . . . . . . . . . . . . . 7.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 1: Derivation of Thurstonian-Scale Values for the 9-Point Scale . . . . . . . . . . . . . . . . . . . . . . . Appendix 2: Construction of Labeled Magnitude Scales . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Time-Intensity Methods . . . . . . . . . . . . . . . . . . . . . . 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 A Brief History . . . . . . . . . . . . . . . . . . . . . . . 8.3 Variations on the Method . . . . . . . . . . . . . . . . . . 8.3.1 Discrete or Discontinuous Sampling . . . . . . . 8.3.2 “Continuous” Tracking . . . . . . . . . . . . . . 8.3.3 Temporal Dominance Techniques . . . . . . . . . 8.4 Recommended Procedures . . . . . . . . . . . . . . . . . . 8.4.1 Steps in Conducting a Time-intensity Study . . . 8.4.2 Procedures . . . . . . . . . . . . . . . . . . . . . 8.4.3 Recommended Analysis . . . . . . . . . . . . . . 8.5 Data Analysis Options . . . . . . . . . . . . . . . . . . . . 8.5.1 General Approaches . . . . . . . . . . . . . . . . 8.5.2 Methods to Construct or Describe Average Curves 8.5.3 Case Study: Simple Geometric Description . . . 8.5.4 Analysis by Principal Components . . . . . . . . 8.6 Examples and Applications . . . . . . . . . . . . . . . . . 8.6.1 Taste and Flavor Sensation Tracking . . . . . . . 8.6.2 Trigeminal and Chemical/Tactile Sensations . . . 8.6.3 Taste and Odor Adaptation . . . . . . . . . . . . 8.6.4 Texture and Phase Change . . . . . . . . . . . . 8.6.5 Flavor Release . . . . . . . . . . . . . . . . . . . 8.6.6 Temporal Aspects of Hedonics . . . . . . . . . . 8.7 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

160 160 162 164 165 166 167 168 168

169 169 169 170 171 172 174 179 179 180 182 182 183 184 185 185 186 186 187 187 188 189 192 193 193 194 194 195 195 196 197 198 198

xiv

Contents

9 Context Effects and Biases in Sensory Judgment . . . . . . . 9.1 Introduction: The Relative Nature of Human Judgment . 9.2 Simple Contrast Effects . . . . . . . . . . . . . . . . . 9.2.1 A Little Theory: Adaptation Level . . . . . . 9.2.2 Intensity Shifts . . . . . . . . . . . . . . . . 9.2.3 Quality Shifts . . . . . . . . . . . . . . . . . 9.2.4 Hedonic Shifts . . . . . . . . . . . . . . . . . 9.2.5 Explanations for Contrast . . . . . . . . . . . 9.3 Range and Frequency Effects . . . . . . . . . . . . . . 9.3.1 A Little More Theory: Parducci’s Range and Frequency Principles . . . . . . . . . . . 9.3.2 Range Effects . . . . . . . . . . . . . . . . . 9.3.3 Frequency Effects . . . . . . . . . . . . . . . 9.4 Biases . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.1 Idiosyncratic Scale Usage and Number Bias . 9.4.2 Poulton’s Classifications . . . . . . . . . . . 9.4.3 Response Range Effects . . . . . . . . . . . . 9.4.4 The Centering Bias . . . . . . . . . . . . . . 9.5 Response Correlation and Response Restriction . . . . 9.5.1 Response Correlation . . . . . . . . . . . . . 9.5.2 “Dumping” Effects: Inflation Due to Response Restriction in Profiling . . . . . 9.5.3 Over-Partitioning . . . . . . . . . . . . . . . 9.6 Classical Psychological Errors and Other Biases . . . . 9.6.1 Errors in Structured Sequences: Anticipation and Habituation . . . . . . . . . . . . . . . . 9.6.2 The Stimulus Error . . . . . . . . . . . . . . 9.6.3 Positional or Order Bias . . . . . . . . . . . . 9.7 Antidotes . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.1 Avoid or Minimize . . . . . . . . . . . . . . 9.7.2 Randomization and Counterbalancing . . . . 9.7.3 Stabilization and Calibration . . . . . . . . . 9.7.4 Interptation . . . . . . . . . . . . . . . . . 9.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Descriptive Analysis . . . . . . . . . . . . . . . . . . . . . 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . 10.2 Uses of Descriptive Analyses . . . . . . . . . . . . 10.3 Language and Descriptive Analysis . . . . . . . . . 10.4 Descriptive Analysis Techniques . . . . . . . . . . R 10.4.1 Flavor Profile . . . . . . . . . . . . . . R 10.4.2 Quantitative Descriptive Analysis . . . R . . . . . . . . . . . . . 10.4.3 Texture Profile R 10.4.4 Sensory Spectrum . . . . . . . . . . . . 10.5 Generic Descriptive Analysis . . . . . . . . . . . . 10.5.1 How to Do Descriptive Analysis in Three Easy Steps . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

203 203 206 206 207 207 208 209 210

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

210 210 211 212 212 213 214 215 216 216

. . . . . . . . . . . .

217 218 218

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

218 219 219 219 219 220 221 222 222 223

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

227 227 228 228 231 231 234 237 238 240

. . . . . .

240

. . . . . . . . . .

. . . . . . . . . .

Contents

xv

10.5.2

Studies Comparing Different Conventional Descriptive Analysis Techniques . . . . . . 10.6 Variations on the Theme . . . . . . . . . . . . . . . . 10.6.1 Using Attribute Citation Frequencies Instead of Attribute Intensities . . . . . . . . . . . 10.6.2 Deviation from Reference Method . . . . . 10.6.3 Intensity Variation Descriptive Method . . . 10.6.4 Combination of Descriptive Analysis and Time-Related Intensity Methods . . . . 10.6.5 Free Choice Profiling . . . . . . . . . . . . 10.6.6 Flash Profiling . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

246 247

. . . . . . . . . . . . . . .

247 248 249

. . . .

. . . .

. . . .

. . . .

. . . .

249 249 252 253

Texture Evaluation . . . . . . . . . . . . . . . . . . . . . . . 11.1 Texture Defined . . . . . . . . . . . . . . . . . . . . . 11.2 Visual, Auditory, and Tactile Texture . . . . . . . . . . 11.2.1 Visual Texture . . . . . . . . . . . . . . . . . 11.2.2 Auditory Texture . . . . . . . . . . . . . . . 11.2.3 Tactile Texture . . . . . . . . . . . . . . . . . 11.2.4 Tactile Hand Feel . . . . . . . . . . . . . . . 11.3 Sensory Texture Measurements . . . . . . . . . . . . . 11.3.1 Texture Profile Method . . . . . . . . . . . . 11.3.2 Other Sensory Texture Evaluation Techniques 11.3.3 Instrumental Texture Measurements and Sensory Correlations . . . . . . . . . . . 11.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

259 259 262 262 262 264 268 270 270 272

. . . . . . . . . . . .

274 276 276

12

Color and Appearance . . . . . . . . . . . . . . . . . . 12.1 Color and Appearance . . . . . . . . . . . . . . . 12.2 What Is Color? . . . . . . . . . . . . . . . . . . . 12.3 Vision . . . . . . . . . . . . . . . . . . . . . . . 12.3.1 Normal Human Color Vision Variations 12.3.2 Human Color Blindness . . . . . . . . . 12.4 Measurement of Appearance and Color Attributes 12.4.1 Appearance . . . . . . . . . . . . . . . 12.4.2 Visual Color Measurement . . . . . . . 12.5 Instrumental Color Measurement . . . . . . . . . 12.5.1 Munsell Color Solid . . . . . . . . . . . 12.5.2 Mathematical Color Systems . . . . . . 12.6 Conclusions . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

283 283 284 285 286 286 286 286 289 293 293 294 299 299

13

Preference Testing . . . . . . . . . . . . . . . . . . 13.1 Introduction-Consumer Sensory Evaluation 13.2 Preference Tests: Overview . . . . . . . . . 13.2.1 The Basic Comparison . . . . . . 13.2.2 Variations . . . . . . . . . . . . . 13.2.3 Some Cautions . . . . . . . . . . 13.3 Simple Paired Preference Testing . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

303 303 305 305 305 306 306

11

. . . . . . .

. . . . . . .

. . . . . . .

xvi

Contents

13.3.1 Recommended Procedure . . . . . . . . . . . . . . . 13.3.2 Statistical Basis . . . . . . . . . . . . . . . . . . . . 13.3.3 Worked Example . . . . . . . . . . . . . . . . . . . 13.3.4 Useful Statistical Approximations . . . . . . . . . . 13.3.5 The Special Case of Equivalence Testing . . . . . . . 13.4 Non-forced Preference . . . . . . . . . . . . . . . . . . . . . . 13.5 Replicated Preference Tests . . . . . . . . . . . . . . . . . . . 13.6 Replicated Non-forced Preference . . . . . . . . . . . . . . . . 13.7 Other Related Methods . . . . . . . . . . . . . . . . . . . . . 13.7.1 Ranking . . . . . . . . . . . . . . . . . . . . . . . . 13.7.2 Analysis of Ranked Data . . . . . . . . . . . . . . . 13.7.3 Best-Worst Scaling . . . . . . . . . . . . . . . . . . 13.7.4 Rated Degree of Preference and Other Options . . . . 13.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 1: Worked Example of the Ferris k-Visit Repeated Preference Test Including the No-Preference Option . . . . . . Appendix 2: The “Placebo” Preference Test . . . . . . . . . . . . . . . Appendix 3: Worked Example of Multinomial Approach to Analyzing Data with the No-Preference Option . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Acceptance Testing . . . . . . . . . . . . . . . . . . . . . . 14.1 Introduction: Scaled Liking Versus Choice . . . . . . 14.2 Hedonic Scaling: Quantification of Acceptability . . . 14.3 Recommended Procedure . . . . . . . . . . . . . . . 14.3.1 Steps . . . . . . . . . . . . . . . . . . . . . 14.3.2 Analysis . . . . . . . . . . . . . . . . . . . 14.3.3 Replication . . . . . . . . . . . . . . . . . 14.4 Other Acceptance Scales . . . . . . . . . . . . . . . . 14.4.1 Line Scales . . . . . . . . . . . . . . . . . 14.4.2 Magnitude Estimation . . . . . . . . . . . . 14.4.3 Labeled Magnitude Scales . . . . . . . . . 14.4.4 Pictorial Scales and Testing with Children . 14.4.5 Adjustable Scales . . . . . . . . . . . . . . 14.5 Just-About-Right Scales . . . . . . . . . . . . . . . . 14.5.1 Description . . . . . . . . . . . . . . . . . 14.5.2 Limitations . . . . . . . . . . . . . . . . . 14.5.3 Variations on Relative-to-Ideal Scaling . . . 14.5.4 Analysis of JAR Data . . . . . . . . . . . . 14.5.5 Penalty Analysis or “Mean Drop” . . . . . 14.5.6 Other Problems and Issues with JAR Scales 14.6 Behavioral and Context-Related Approaches . . . . . 14.6.1 Food Action Rating Scale (FACT) . . . . . 14.6.2 Appropriateness Scales . . . . . . . . . . . 14.6.3 Acceptor Set Size . . . . . . . . . . . . . . 14.6.4 Barter Scales . . . . . . . . . . . . . . . . 14.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

306 307 308 309 310 311 313 313 315 315 316 317 318 320 320 321 322 323 325 325 326 327 327 328 328 328 328 330 331 332 333 334 334 335 336 336 339 340 340 341 341 342 343 343 344

Contents

xvii

15

16

Consumer Field Tests and Questionnaire Design . . . . . . . . . . . 15.1 Sensory Testing Versus Concept Testing . . . . . . . . . . . . 15.2 Testing Scenarios: Central Location, Home Use . . . . . . . . 15.2.1 Purpose of the Tests . . . . . . . . . . . . . . . . . . 15.2.2 Consumer Models . . . . . . . . . . . . . . . . . . . 15.2.3 Central Location Tests . . . . . . . . . . . . . . . . 15.2.4 Home Use Tests (HUT) . . . . . . . . . . . . . . . . 15.3 Practical Matters in Conducting Consumer Field Tests . . . . . 15.3.1 Tasks and Test Design . . . . . . . . . . . . . . . . . 15.3.2 Sample Size and Stratification . . . . . . . . . . . . 15.3.3 Test Designs . . . . . . . . . . . . . . . . . . . . . . 15.4 Interacting with Field Services . . . . . . . . . . . . . . . . . 15.4.1 Choosing Agencies, Communication, and Test Specifications . . . . . . . . . . . . . . . . 15.4.2 Incidence, Cost, and Recruitment . . . . . . . . . . . 15.4.3 Some Tips: Do’s and Don’ts . . . . . . . . . . . . . 15.4.4 Steps in Testing with Research Suppliers . . . . . . . 15.5 Questionnaire Design . . . . . . . . . . . . . . . . . . . . . . 15.5.1 Types of Interviews . . . . . . . . . . . . . . . . . . 15.5.2 Questionnaire Flow: Order of Questions . . . . . . . 15.5.3 Interviewing . . . . . . . . . . . . . . . . . . . . . . 15.6 Rules of Thumb for Constructing Questions . . . . . . . . . . 15.6.1 General Principles . . . . . . . . . . . . . . . . . . . 15.6.2 Brevity . . . . . . . . . . . . . . . . . . . . . . . . . 15.6.3 Use Plain Language . . . . . . . . . . . . . . . . . . 15.6.4 Accessibility of the Information . . . . . . . . . . . 15.6.5 Avoid Vague Questions . . . . . . . . . . . . . . . . 15.6.6 Check for Overlap and Completeness . . . . . . . . . 15.6.7 Do Not Lead the Respondent . . . . . . . . . . . . . 15.6.8 Avoid Ambiguity and Double Questions . . . . . . . 15.6.9 Be Careful in Wording: Present Both Alternatives . . 15.6.10 Beware of Halos and Horns . . . . . . . . . . . . . . 15.6.11 Pre-test . . . . . . . . . . . . . . . . . . . . . . . . 15.7 Other Useful Questions: Satisfaction, Agreement, and Open-Ended Questions . . . . . . . . . . . . . . . . . . . 15.7.1 Satisfaction . . . . . . . . . . . . . . . . . . . . . . 15.7.2 Likert (Agree-Disagree) Scales . . . . . . . . . . . . 15.7.3 Open-Ended Questions . . . . . . . . . . . . . . . . 15.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 1: Sample Test Specification Sheet . . . . . . . . . . . . . . Appendix 2: Sample Screening Questionnaire . . . . . . . . . . . . . . Appendix 3: Sample Product Questionnaire . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qualitative Consumer Research Methods . . . . . . . 16.1 Introduction . . . . . . . . . . . . . . . . . . . 16.1.1 Resources, Definitions, and Objectives 16.1.2 Styles of Qualitative Research . . . . 16.1.3 Other Qualitative Techniques . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

349 349 351 351 352 353 354 355 355 355 356 358 358 359 360 360 362 362 362 363 364 364 364 364 365 365 365 365 366 366 366 366 367 367 367 367 368 370 371 374 378 379 380 380 380 382

xviii

Contents

16.2

Characteristics of Focus Groups . . . . . . . . . . . . . . . . . 383 16.2.1 Advantages . . . . . . . . . . . . . . . . . . . . . . 383 16.2.2 Key Requirements . . . . . . . . . . . . . . . . . . . 384 16.2.3 Reliability and Validity . . . . . . . . . . . . . . . . 384 16.3 Using Focus Groups in Sensory Evaluation . . . . . . . . . . . 385 16.4 Examples, Case Studies . . . . . . . . . . . . . . . . . . . . . 386 16.4.1 Case Study 1: Qualitative Research Before Conjoint Measurement in New Product Development 387 16.4.2 Case Study 2: Nutritional and Health Beliefs About Salt 387 16.5 Conducting Focus Group Studies . . . . . . . . . . . . . . . . 388 16.5.1 A Quick Overview . . . . . . . . . . . . . . . . . . 388 16.5.2 A Key Requirement: Developing Good Questions . . 389 16.5.3 The Discussion Guide and Phases of the Group Interview . . . . . . . . . . . . . . . . 390 16.5.4 Participant Requirements, Timing, Recording . . . . 391 16.6 Issues in Moderating . . . . . . . . . . . . . . . . . . . . . . . 392 16.6.1 Moderating Skills . . . . . . . . . . . . . . . . . . . 392 16.6.2 Basic Principles: Nondirection, Full Participation, and Coverage of Issues . . . . . . . . . 393 16.6.3 Assistant Moderators and Co-moderators . . . . . . . 394 16.6.4 Debriefing: Avoiding Selective Listening and Premature Conclusions . . . . . . . . . . . . . . 395 16.7 Analysis and Reporting . . . . . . . . . . . . . . . . . . . . . 395 16.7.1 General Principles . . . . . . . . . . . . . . . . . . . 395 16.7.2 Suggested Method (“Sorting/Clustering Approach”), also Called Classical Transcript Analysis . . . . . . . . . . . . . . . . . . . . . . . . 396 16.7.3 Report Format . . . . . . . . . . . . . . . . . . . . . 397 16.8 Alternative Procedures and Variations of the Group Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 16.8.1 Groups of Children, Telephone Interviews, Internet-Based Groups . . . . . . . . . . . . . . . . 398 16.8.2 Alternatives to Traditional Questioning . . . . . . . . 399 16.9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 Appendix: Sample Report Group Report . . . . . . . . . . . . . . . . . 402 Boil-in-bag Pasta Project Followup Groups . . . . . . . . . . . . . . . 402 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 17 Quality Control and Shelf-Life (Stability) Testing . 17.1 Introduction: Objectives and Challenges . . . 17.2 A Quick Look at Traditional Quality Control . 17.3 Methods for Sensory QC . . . . . . . . . . . 17.3.1 Cuttings: A Bad Example . . . . . . 17.3.2 In-Out (Pass/Fail) System . . . . . 17.3.3 Difference from Control Ratings . . 17.3.4 Quality Ratings with Diagnostics . . 17.3.5 Descriptive Analysis . . . . . . . . 17.3.6 A Hybrid Approach: Quality Ratings with Diagnostics . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

407 408 409 409 409 410 411 412 413

. . . . . . . . .

414

Contents

xix

17.3.7 The Multiple Standards Difference Test . . . . . . . Recommended Procedure: Difference Scoring with Key Attribute Scales . . . . . . . . . . . . . . . . . . . . . . . 17.5 The Importance of Good Practice . . . . . . . . . . . . . . . . 17.6 Historical Footnote: Expert Judges and Quality Scoring . . . . 17.6.1 Standardized Commodities . . . . . . . . . . . . . . 17.6.2 Example 1: Dairy Product Judging . . . . . . . . . . 17.6.3 Example 2: Wine Scoring . . . . . . . . . . . . . . . 17.7 Program Requirements and Program Development . . . . . . . 17.7.1 Desired Features of a Sensory QC System . . . . . . 17.7.2 Program Development and Management Issues . . . 17.7.3 The Problem of Low Incidence . . . . . . . . . . . . 17.8 Shelf-Life Testing . . . . . . . . . . . . . . . . . . . . . . . . 17.8.1 Basic Considerations . . . . . . . . . . . . . . . . . 17.8.2 Cutoff Point . . . . . . . . . . . . . . . . . . . . . . 17.8.3 Test Designs . . . . . . . . . . . . . . . . . . . . . . 17.8.4 Survival Analysis and Hazard Functions . . . . . . . 17.8.5 Accelerated Storage . . . . . . . . . . . . . . . . . . 17.9 Summary and Conclusions . . . . . . . . . . . . . . . . . . . Appendix 1: Sample Screening Tests for Sensory Quality Judges . . . . Appendix 2: Survival/Failure Estimates from a Series of Batches with Known Failure Times . . . . . . . . . . . . . . Appendix 3: Arrhenius Equation and Q10 Modeling . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

414

17.4

18

19

415 417 419 419 419 420 422 422 423 424 424 424 426 426 427 428 428 429 429 430 431

Data Relationships and Multivariate Applications . . . . . . . . . . 18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 18.2 Overview of Multivariate Statistical Techniques . . . . . . . . 18.2.1 Principal Component Analysis . . . . . . . . . . . . 18.2.2 Multivariate Analysis of Variance . . . . . . . . . . . 18.2.3 Discriminant Analysis (Also Known as Canonical Variate Analysis) . . . . . . . . . . . . 18.2.4 Generalized Procrustes Analysis . . . . . . . . . . . 18.3 Relating Consumer and Descriptive Data Through Preference Mapping . . . . . . . . . . . . . . . . . . . . . . . 18.3.1 Internal Preference Mapping . . . . . . . . . . . . . 18.3.2 External Preference Mapping . . . . . . . . . . . . . 18.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

440 442 442 445 446

Strategic Research . . . . . . . . . . . . . . . 19.1 Introduction . . . . . . . . . . . . . . . 19.1.1 Avenues for Strategic Research 19.1.2 Consumer Contact . . . . . . . 19.2 Competitive Surveillance . . . . . . . . 19.2.1 The Category Review . . . . . 19.2.2 Perceptual Mapping . . . . . . 19.2.3 Multivariate Methods: PCA . . 19.2.4 Multi-dimensional Scaling . .

451 451 451 453 453 453 455 456 458

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

433 433 434 434 437 438 439

xx

Contents

19.2.5

Cost-Efficient Methods for Data Collection: Sorting . . . . . . . . . . . . . . . . . . . . . . 19.2.6 Vector Projection . . . . . . . . . . . . . . . . 19.2.7 Cost-Efficient Methods for Data Collection: Projective Mapping, aka Napping . . . . . . . . 19.3 Attribute Identification and Classification . . . . . . . . . 19.3.1 Drivers of Liking . . . . . . . . . . . . . . . . 19.3.2 The Kano Model . . . . . . . . . . . . . . . . 19.4 Preference Mapping Revisited . . . . . . . . . . . . . . . 19.4.1 Types of Preference Maps . . . . . . . . . . . . 19.4.2 Preference Models: Vectors Versus Ideal Points 19.5 Consumer Segmentation . . . . . . . . . . . . . . . . . . 19.6 Claim Substantiation Revisited . . . . . . . . . . . . . . 19.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 19.7.1 Blind Testing, New Coke, and the Vienna Philharmonic . . . . . . . . . . . . . . . . . . 19.7.2 The Sensory Contribution . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

459 460

. . . . . . . . . .

. . . . . . . . . .

461 462 462 463 464 464 464 465 467 468

. . . . . . . . .

468 469 469

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

473 473 474 475 476 478 478 478 479 480 482 482

. . . . .

. . . . .

. . . . .

484 485 485 486 487

. . . . . . . .

. . . . . . . .

. . . . . . . .

489 489 490 493 493 494 494 495

. . . . .

496

Appendix A Basic Statistical Concepts for Sensory Evaluation . . A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . A.2 Basic Statistical Concepts . . . . . . . . . . . . . . . . . A.2.1 Data Description . . . . . . . . . . . . . . . . A.2.2 Population Statistics . . . . . . . . . . . . . . . A.3 Hypothesis Testing and Statistical Inference . . . . . . . A.3.1 The Confidence Interval . . . . . . . . . . . . . A.3.2 Hypothesis Testing . . . . . . . . . . . . . . . A.3.3 A Worked Example . . . . . . . . . . . . . . . A.3.4 A Few More Important Concepts . . . . . . . . A.3.5 Decision Errors . . . . . . . . . . . . . . . . . A.4 Variations of the t-Test . . . . . . . . . . . . . . . . . . . A.4.1 The Sensitivity of the Dependent t-Test for Sensory Data . . . . . . . . . . . . . . . . . . A.5 Summary: Statistical Hypothesis Testing . . . . . . . . . A.6 Postscript: What p-Values Signify and What They Do Not A.7 Statistical Glossary . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix B Nonparametric and Binomial-Based Statistical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.1 Introduction to Nonparametric Tests . . . . . . . . . B.2 Binomial-Based Tests on Proportions . . . . . . . . . B.3 Chi-Square . . . . . . . . . . . . . . . . . . . . . . . B.3.1 A Measure of Relatedness of Two Variables B.3.2 Calculations . . . . . . . . . . . . . . . . . B.3.3 Related Samples: The McNemar Test . . . . B.3.4 The Stuart-Maxwell Test . . . . . . . . . . B.3.5 Beta-Binomial, Chance-Corrected Beta-Binomial, and Dirichlet Multinomial Analyses . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . . . .

Contents

xxi

B.4

Useful Rank Order Tests . . . . . . . . . . . . . . . . B.4.1 The Sign Test . . . . . . . . . . . . . . . . B.4.2 The Mann-Whitney U-Test . . . . . . . . . B.4.3 Ranked Data with More Than Two Samples, Friedman and Kramer Tests . . . . . . . . . B.4.4 Rank Order Correlation . . . . . . . . . . . B.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . B.6 Postscript . . . . . . . . . . . . . . . . . . . . . . . . B.6.1 Proof showing equivalence of binomial approximation Z-test and χ 2 test for difference of proportions . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

499 499 500

. . . .

. . . .

501 502 503 503

. . . . . . . . . .

503 504

. . . .

Appendix C Analysis of Variance . . . . . . . . . . . . . . . . . . . C.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . C.1.1 Overview . . . . . . . . . . . . . . . . . . . . C.1.2 Basic Analysis of Variance . . . . . . . . . . . C.1.3 Rationale . . . . . . . . . . . . . . . . . . . . C.1.4 Calculations . . . . . . . . . . . . . . . . . . . C.1.5 A Worked Example . . . . . . . . . . . . . . . C.2 Analysis of Variance from Complete Block Designs . . . C.2.1 Concepts and Partitioning Panelist Variance from Error . . . . . . . . . . . . . . . . . . . . C.2.2 The Value of Using Panelists As Their Own Controls . . . . . . . . . . . . . C.3 Planned Comparisons Between Means Following ANOVA C.4 Multiple Factor Analysis of Variance . . . . . . . . . . . C.4.1 An Example . . . . . . . . . . . . . . . . . . . C.4.2 Concept: A Linear Model . . . . . . . . . . . . C.4.3 A Note About Interactions . . . . . . . . . . . C.5 Panelist by Product by Replicate Designs . . . . . . . . . C.6 Issues and Concerns . . . . . . . . . . . . . . . . . . . . C.6.1 Sensory Panelists: Fixed or Random Effects? . C.6.2 A Note on Blocking . . . . . . . . . . . . . . . C.6.3 Split-Plot or Between-Groups (Nested) Designs C.6.4 Statistical Assumptions and the Repeated Measures ANOVA . . . . . . . . . . . . . . . C.6.5 Other Options . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix D Correlation, Regression, and Measures of Association D.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . D.2 Correlation . . . . . . . . . . . . . . . . . . . . . . . . . D.2.1 Pearson’s Correlation Coefficient Example . . . D.2.2 Coefficient of Determination . . . . . . . . . . D.3 Linear Regression . . . . . . . . . . . . . . . . . . . . . D.3.1 Analysis of Variance . . . . . . . . . . . . . . D.3.2 Analysis of Variance for Linear Regression . . D.3.3 Prediction of the Regression Line . . . . . . . . D.3.4 Linear Regression Example . . . . . . . . . . .

. . . .

. . . . . . . .

. . . .

. . . . . . . .

. . . . . . . .

507 507 507 508 508 509 509 510

. . .

510

. . . . . . . . . . .

. . . . . . . . . . .

512 513 514 514 515 516 516 519 519 520 520

. . . . . . . . .

521 522 522

. . . . . . . . . .

525 525 527 528 529 529 530 530 530 531

. . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

xxii

Contents

D.4 D.5

Multiple Linear Regression . . . . . . . . . . . . . . Other Measures of Association . . . . . . . . . . . . D.5.1 Spearman Rank Correlation . . . . . . . . . D.5.2 Spearman Correlation Coefficient Example D.5.3 Cramér’s V Measure . . . . . . . . . . . . . D.5.4 Cramér Coefficient Example . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

531 531 531 532 532 533 533

Appendix E Statistical Power and Test Sensitivity . . . . . . . . E.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . E.2 Factors Affecting the Power of Statistical Tests . . . . . E.2.1 Sample Size and Alpha Level . . . . . . . . . E.2.2 Effect Size . . . . . . . . . . . . . . . . . . . E.2.3 How Alpha, Beta, Effect Size, and N Interact E.3 Worked Examples . . . . . . . . . . . . . . . . . . . . E.3.1 The t-Test . . . . . . . . . . . . . . . . . . . E.3.2 An Equivalence Issue with Scaled Data . . . E.3.3 Sample Size for a Difference Test . . . . . . . E.4 Power in Simple Difference and Preference Tests . . . . E.5 Summary and Conclusions . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

535 535 537 537 538 539 541 541 542 544 545 548 549

Appendix F Statistical Tables . . . . . . . . . . . . . . . . . . . . . . . . . Table F.A Cumulative probabilities of the standard normal distribution. Entry area 1-α under the standard normal curve from −∞ to z(1-α) . . . . . . . . . . . . . Table F.B Table of critical values for the t-distribution . . . . . . . . Table F.C Table of critical values of the chi-square (χ 2 ) distribution . . . . . . . . . . . . . . . . . . . . . . . . . Table F.D1 Critical values of the F-distribution at α = 0.05 . . . . . . Table F.D2 Critical values of the F-distribution at α = 0.01 . . . . . . Table F.E Critical values of U for a one-tailed alpha at 0.025 or a two-tailed alpha at 0.05 . . . . . . . . . . . . . . . . Table F.F1 Table of critical values of ρ (Spearman Rank correlation coefficient) . . . . . . . . . . . . . . . . . . . Table F.F2 Table of critical values of r (Pearson’s correlation coefficient) . . . . . . . . . . . . . . . . . . . . . . . . . Table F.G Critical values for Duncan’s multiple range test (p, df, α = 0.05) . . . . . . . . . . . . . . . . . . . . . . . Table F.H1 Critical values of the triangle test for similarity (maximum number correct as a function of the number of observations (N), beta, and proportion discriminating) . . . . . . . . . . . . . . . . . . . . . . . Table F.H2 Critical values of the duo-trio and paired comparison tests for similarity (maximum number correct as a function of the number of observations (N), beta, and proportion discriminating) . . . . . . . . . . Table F.I Table of probabilities for values as small as observed values of x associated with the binomial test (p=0.50) . . . . . . . . . . . . . . . . . . . . . . . .

551

552 553 554 555 556 556 557 558 559

560

561

562

Contents

xxiii

Table F.J

Critical values for the differences between rank sums (α = 0.05) . . . . . . . . . . . . . . . . . . . . . . . Critical values of the beta binomial distribution . . . . . . Minimum numbers of correct judgments to establish significance at probability levels of 5 and 1% for paired difference and duo-trio tests (one tailed, p = 1/2) and the triangle test (one tailed, p = 1/3) . . . . . . . . . . . . . . . . . . Minimum numbers of correct judgments to establish significance at probability levels of 5 and 1% for paired pference test (two tailed, p = 1/2) . . . . . . . . . . . . . . . . . . . . . . . . . . . Minimum number of responses (n) and correct responses (x) to obtain a level of Type I and Type II risks in the triangle test. Pd is the chance-adjusted percent correct or proportion of discriminators . . . . . . . . . . . . . . . . . . . . . . Minimum number of responses (n) and correct responses (x) to obtain a level of Type I and Type II risks in the duo-trio test. Pc is the chance-adjusted percent correct or proportion of discriminators . . . . . . . . . . . . . . . . . . . . . . d and B (variance factor) values for the duo-trio and 2-AFC (paired comparison) difference tests . . . . . . d and B (variance factor) values for the triangle and 3-AFC difference tests . . . . . . . . . . . . . . . . . Random permutations of nine . . . . . . . . . . . . . . . Random numbers . . . . . . . . . . . . . . . . . . . . . .

569 571 572

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

573

Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

587

Table F.K Table F.L

Table F.M

Table F.N1

Table F.N2

Table F.O1 Table F.O2 Table F.P Table F.Q

563 564

565

566

567

567 568

Chapter 1

Introduction

Abstract In this chapter we carefully parse the definition for sensory evaluation, briefly discuss validity of the data collected before outlining the early history of the field. We then describe the three main methods used in sensory evaluation (discrimination tests, descriptive analysis, and hedonic testing) before discussing the differences between analytical and consumer testing. We then briefly discuss why one may want to collect sensory data. In the final sections we highlight the differences and similarities between sensory evaluation and marketing research and between sensory evaluation and commodity grading as used in, for example, the dairy industry.

Sensory evaluation is a child of industry. It was spawned in the late 40’s by the rapid growth of the consumer product companies, mainly food companies. . . . Future development in sensory evaluation will depend upon several factors, one of the most important being the people and their pparation and training. – Elaine Skinner (1989)

Contents Introduction and Overview . . . . . . . . . 1.1.1 Definition . . . . . . . . . . . . . . 1.1.2 Measurement . . . . . . . . . . . . 1.2 Historical Landmarks and the Three Classes of Test Methods . . . . . . . . . . . . . . 1.2.1 Difference Testing . . . . . . . . . . 1.2.2 Descriptive Analyses . . . . . . . . . 1.2.3 Affective Testing . . . . . . . . . . 1.2.4 The Central Dogma-Analytic Versus Hedonic Tests . . . . . . . . . . . . 1.3 Applications: Why Collect Sensory Data? . . 1.3.1 Differences from Marketing Research Methods . . . . . . . . . . . . . . 1.3.2 Differences from Traditional Product Grading Systems . . . . . . . . . . 1.4 Summary and Conclusions . . . . . . . . . References . . . . . . . . . . . . . . . . . . . 1.1

1.1 Introduction and Overview . . .

1 1 3

. . . .

4 5 6 7

. .

8 10

.

13

. . .

15 16 17

1.1.1 Definition The field of sensory evaluation grew rapidly in the second half of the twentieth century, along with the expansion of the processed food and consumer products industries. Sensory evaluation comprises a set of techniques for accurate measurement of human responses to foods and minimizes the potentially biasing effects of brand identity and other information influences on consumer perception. As such, it attempts to isolate the sensory properties of foods themselves and provides important and useful information to product developers, food scientists, and managers about the sensory characteristics of their products. The field was comphensively reviewed by Amerine, Pangborn, and Roessler in 1965, and more recent texts have been published by Moskowitz et al. (2006), Stone and Sidel (2004), and Meilgaard et al. (2006). These three later sources are practical works aimed at sensory specialists

H.T. Lawless, H. Heymann, Sensory Evaluation of Food, Food Science Text Series, DOI 10.1007/978-1-4419-6488-5_1, © Springer Science+Business Media, LLC 2010

1

2

in industry and reflect the philosophies of the consulting groups of the authors. Our goal in this book is to provide a comphensive overview of the field with a balanced view based on research findings and one that is suited to students and practitioners alike. Sensory evaluation has been defined as a scientific method used to evoke, measure, analyze, and interpt those responses to products as perceived through the senses of sight, smell, touch, taste, and hearing (Stone and Sidel, 2004). This definition has been accepted and endorsed by sensory evaluation committees within various professional organizations such as the Institute of Food Technologists and the American Society for Testing and Materials. The principles and practices of sensory evaluation involve each of the four activities mentioned in this definition. Consider the words “to evoke.” Sensory evaluation gives guidelines for the pparation and serving of samples under controlled conditions so that biasing factors are minimized. For example, people in a sensory test are often placed in inpidual test booths so that the judgments they give are their own and do not reflect the opinions of those around them. Samples are labeled with random numbers so that people do not form judgments based upon labels, but rather on their sensory experiences. Another example is in how products may be given in different orders to each participant to help measure and counterbalance for the sequential effects of seeing one product after another. Standard procedures may be established for sample temperature, volume, and spacing in time, as needed to control unwanted variation and improve test pcision. Next, consider the words, “to measure.” Sensory evaluation is a quantitative science in which numerical data are collected to establish lawful and specific relationships between product characteristics and human perception. Sensory methods draw heavily from the techniques of behavioral research in observing and quantifying human responses. For example, we can assess the proportion of times people are able to discriminate small product changes or the proportion of a group that expsses a pference for one product over another. Another example is having people generate numerical responses reflecting their perception of how strong a product may taste or smell. Techniques of behavioral research and experimental psychology offer guidelines as to how such measurement techniques should be employed and what their potential pitfalls and liabilities may be.

1 Introduction

The third process in sensory evaluation is analysis. Proper analysis of the data is a critical part of sensory testing. Data generated from human observers are often highly variable. There are many sources of variation in human responses that cannot be completely controlled in a sensory test. Examples include the mood and motivation of the participants, their innate physiological sensitivity to sensory stimulation, and their past history and familiarity with similar products. While some screening may occur for these factors, they may be only partially controlled, and panels of humans are by their nature heterogeneous instruments for the generation of data. In order to assess whether the relationships observed between product characteristics and sensory responses are likely to be real, and not merely the result of uncontrolled variation in responses, the methods of statistics are used to analyze evaluation data. Hand-in-hand with using appropriate statistical analyses is the concern of using good experimental design, so that the variables of interest are investigated in a way that allows sensible conclusions to be drawn. The fourth process in sensory evaluation is the interptation of results. A sensory evaluation exercise is necessarily an experiment. In experiments, data and statistical information are only useful when interpted in the context of hypotheses, background knowledge, and implications for decisions and actions to be taken. Conclusions must be drawn that are reasoned judgments based upon data, analyses, and results. Conclusions involve consideration of the method, the limitations of the experiment, and the background and contextual framework of the study. The sensory evaluation specialists become more than mere conduits for experimental results, but must contribute interptations and suggest reasonable courses of action in light of the numbers. They should be full partners with their clients, the end-users of the test results, in guiding further research. The sensory evaluation professional is in the best situation to realize the appropriate interptation of test results and the implications for the perception of products by the wider group of consumers to whom the results may be generalized. The sensory specialist best understands the limitations of the test procedure and what its risks and liabilities may be. A sensory scientist who is ppared for a career in research must be trained in all four of the phases mentioned in the definition. They must understand products, people as measuring instruments, statistical

1.1 Introduction and Overview

1.1.2 Measurement Sensory evaluation is a science of measurement. Like other analytical test procedures, sensory evaluation is concerned with pcision, accuracy, sensitivity, and avoiding false positive results (Meiselman, 1993). Precision is similar to the concept in the behavioral sciences of reliability. In any test procedure, we would like to be able to get the same result when a test is repeated. There is usually some error variance around an obtained value, so that upon repeat testing, the value will not always be exactly the same. This is especially true of sensory tests in which human perceptions are necessarily part of the generation of data. However, in many sensory test procedures, it is desirable to minimize this error variance as much as possible and to have tests that are low in error associated with repeated measurements. This is achieved by several means. As noted above, we isolate the sensory response to the factors of interest, minimizing extraneous influences, controlling sample pparation and psentation. Additionally, as necessary, sensory scientists screen and train panel participants. A second concern is the accuracy of a test. In the physical sciences, this is viewed as the ability of a test instrument to produce a value that is close to the “true” value, as defined by independent measurement from another instrument or set of instruments that have been appropriately calibrated. A related idea in the behavioral sciences, this principle is called the validity of a test. This concerns the ability of a test procedure to measure what it was designed and intended to measure. Validity is established in a number of ways. One useful criterion is pdictive validity, when a test result is of value in pdicting what would occur in another situation or another measurement. In sensory testing, for example, the test results should reflect the perceptions and opinions of consumers that might buy the product. In other words, the results of the sensory test should generalize to the larger population. The test results might correlate with instrumental measures, process or ingredient variables, storage factors, shelf life times,

3

or other conditions known to affect sensory properties. In considering validity, we have to look at the end use of the information provided by a test. A sensory test method might be valid for some purposes, but not others (Meiselman, 1993). A simple difference test can tell if a product has changed, but not whether people will like the new version. A good sensory test will minimize errors in measurement and errors in conclusions and decisions. There are different types of errors that may occur in any test procedure. Whether the test result reflects the true state of the world is an important question, especially when error and uncontrolled variability are inherent in the measurement process. Of primary concern in sensory tests is the sensitivity of the test to differences among products. Another way to phrase this is that a test should not often miss important differences that are psent. “Missing a difference” implies an insensitive test procedure. To keep sensitivity high, we must minimize error variance wherever possible by careful experimental controls and by selection and training of panelists where appropriate. The test must involve sufficient numbers of measurements to insure a tight and reliable statistical estimate of the values we obtain, such as means or proportions. In statistical language, detecting true differences is avoiding Type II error and the minimization of β-risk. Discussion of the power and sensitivity of tests from a statistical perspective occurs in Chapter 5 and in the Appendix. The other error that may occur in a test result is that of finding a positive result when none is actually psent in the larger population of people and products outside the sensory test. Once again, a positive result usually means detection of a statistically significant difference between test products. It is important to use a test method that avoids false positive results or Type I error in statistical language. Basic statistical training and common statistical tests applied to scientific findings are oriented toward avoiding this kind of error. The effects of random chance deviations must be taken into account in deciding if a test result reflects a real difference or whether our result is likely to be due to chance variation. The common procedures of inferential statistics provide assurance that we have limited our possibility of finding a difference where one does not really exist. Statistical procedures reduce this risk to some comfortable level, usually with a ceiling of 5% of all tests we conduct.

4

1 Introduction

(Lawless and Klein, 1989). An important function of sensory scientists in an academic setting is to provide consulting and resources to insure that quality tests are conducted by other researchers and students who seek to understand the sensory impact of the variables they are studying. In government services such as food inspection, sensory evaluation plays a key role (York, 1995). Sensory principles and appropriate training can be key in insuring that test methods reflect the current knowledge of sensory function and test design. See Lawless (1993) for an overview of the education and training of sensory scientists-much of this piece still rings true more than 15 years later.

1.2 Historical Landmarks and the Three Classes of Test Methods The human senses have been used for centuries to evaluate the quality of foods. We all form judgments about foods whenever we eat or drink (“Everyone carries his own inch-rule of taste, and amuses himself by applying it, triumphantly, wherever he travels.”-Henry Adams, 1918). This does not mean that all judgments are useful or that anyone is qualified to participate in a sensory test. In the past, production of good quality foods often depended upon the sensory acuity of a single expert who was in charge of production or made decisions about process changes in order to make sure the product would have desirable characteristics. This was the historical tradition of brewmasters, wine tasters, dairy judges, and other food inspectors who acted as the arbiters of quality. Modern sensory evaluation replaced these single authorities with panels of people participating in specific test methods that took the form of planned experiments. This change occurred for several reasons. First, it was recognized that the judgments of a panel would in general be more reliable than the judgments of single inpidual and it entailed less risk since the single expert could become ill, travel, retire, die, or be otherwise unavailable to make decisions. Replacement of such an inpidual was a nontrivial problem. Second, the expert might or might not reflect what consumers or segments of the consuming public might want in a product. Thus for issues of product quality and overall appeal, it was safer (although often more time consuming and expensive)

1.2 Historical Landmarks and the Three Classes of Test Methods

to go directly to the target population. Although the tradition of informal, qualitative inspections such as benchtop “cuttings” persists in some industries, they have been gradually replaced by more formal, quantitative, and controlled observations (Stone and Sidel, 2004). The current sensory evaluation methods comprise a set of measurement techniques with established track records of use in industry and academic research. Much of what we consider standard procedures comes from pitfalls and problems encountered in the practical experience of sensory specialists over the last 70 years of food and consumer product research, and this experience is considerable. The primary concern of any sensory evaluation specialist is to insure that the test method is appropriate to answer the questions being asked about the product in the test. For this reason, tests are usually classified according to their primary purpose and most valid use. Three types of sensory testing are commonly used, each with a different goal and each using participants selected using different criteria. A summary of the three main types of testing is given in Table 1.1.

1.2.1 Difference Testing The simplest sensory tests merely attempt to answer whether any perceptible difference exists between two types of products. These are the discrimination tests or simple difference testing procedures. Analysis is usually based on the statistics of frequencies and proportions (counting right and wrong answers). From the test results, we infer differences based on the proportions of persons who are able to choose a test product correctly from among a set of similar or control products. A classic example of this test was the triangle procedure, used in the Carlsberg breweries and in the Seagrams distilleries in the 1940s (Helm and Trolle,

5

1946; Peryam and Swartz, 1950). In this test, two products were from the same batch while a third product was different. Judges would be asked to pick the odd sample from among the three. Ability to discriminate differences would be inferred from consistent correct choices above the level expected by chance. In breweries, this test served primarily as a means to screen judges for beer evaluation, to insure that they possessed sufficient discrimination abilities. Another multiple-choice difference test was developed at about the same time in distilleries for purposes of quality control (Peryam and Swartz, 1950). In the duo-trio procedure, a reference sample was given and then two test samples. One of the test samples matched the reference while the other was from a different product, batch or process. The participant would try to match the correct sample to the reference, with a chance probability of one-half. As in the triangle test, a proportion of correct choices above that expected by chance is considered evidence for a perceivable difference between products. A third popular difference test was the paired comparison, in which participants would be asked to choose which of two products was stronger or more intense in a given attribute. Partly due to the fact that the panelist’s attention is directed to a specific attribute, this test is very sensitive to differences. These three common difference tests are shown in Fig. 1.1. Simple difference tests have proven very useful in application and are in widespad use today. Typically a discrimination test will be conducted with 25-40 participants who have been screened for their sensory acuity to common product differences and who are familiar with the test procedures. This generally provides an adequate sample size for documenting clear sensory differences. Often a replicate test is performed while the respondents are psent in the sensory test facility. In part, the popularity of these tests is due to the simplicity of data analysis. Statistical tables derived from the binomial distribution give the minimum number of correct responses needed to conclude statistical

Table 1.1 Classification of test methods in sensory evaluation Class Question of interest

Type of test

Panelist characteristics

Discrimination

Are products perceptibly different in any way

“Analytic”

Descriptive

How do products differ in specific sensory characteristics How well are products liked or which products are pferred

“Analytic”

Screened for sensory acuity, oriented to test method, sometimes trained Screened for sensory acuity and motivation, trained or highly trained Screened for products, untrained

Affective

“Hedonic”

6

1 Introduction

Fig. 1.1 Common methods for discrimination testing include the triangle, duo-trio, and paired comparison procedures.

significance as a function of the number of participants. Thus a sensory technician merely needs to count answers and refer to a table to give a simple statistical conclusion, and results can be easily and quickly reported.

the consensus of a panel was likely to be more reliable and accurate than the judgment of a single inpidual. Second, it provided a means to characterize the inpidual attributes of flavor and provide a comphensive analytical description of differences among a group of products under development. Several variations and refinements in descriptive analysis techniques were forthcoming. A group at the General Foods Technical Center in the early 1960s developed and refined a method to quantify food texture, much as the flavor profile had enabled the quantification of flavor properties (Brandt et al., 1963, Szczesniak et al., 1975). This technique, the Texture Profile method, used a fixed set of force-related and shape-related attributes to characterize the rheological and tactile properties of foods and how these changed over time with mastication. These characteristics have parallels in the physical evaluation of food breakdown or flow. For example, perceived hardness is related to the physical force required to penetrate a sample. Perceived thickness of a fluid or semisolid is related in part to physical viscosity. Texture profile panelists were also trained to recognize specific intensity points along each scale, using standard products or formulated pseudo-foods for calibration. Other approaches were developed for descriptive analysis problems. At Stanford Research Institute in the early 1970s, a group proposed a method for descriptive analysis that would remedy some of the R apparent shortcomings of the Flavor Profile method

1.2 Historical Landmarks and the Three Classes of Test Methods

Table 1.2 Descriptive evaluation of cookies-texture attributes Phase Attributes Word anchors Surface

First bite

First chew Chew down

Residual

Roughness Particles Dryness Fracturability Hardness Particle size Denseness Uniformity of chew Moisture absorption Cohesiveness of mass Toothpacking Grittiness Oiliness Particles Chalky

Smooth-rough None-many Oily-dry Crumbly-brittle Soft-hard Small-large Airy-dense Even-uneven None-much Loose-cohesive None-much None-much Dry-oily None-many Not chalky-very chalky

7

a uniform and controlled manner, typical of an analytical sensory test procedure. For example, the first bite may be defined as cutting with the incisors. The panel for such an analysis would consist of perhaps 10- 12 well-trained inpiduals, who were oriented to the meanings of the terms and given practice with examples. Intensity references to exemplify scale points are also given in some techniques. Note the amount of detailed information that can be provided in this example and bear in mind that this is only looking at the product’s texture-flavor might form an equally detailed sensory analysis, perhaps with a separate trained panel. The relatively small number of panelists (a dozen or so) is justified due to their level of calibration. Since they have been trained to use attribute scales in a similar manner, error variance is lowered and statistical power and test sensitivity are maintained in spite of fewer observations (fewer data points per product). Similar examples of texture, flavor, fragrance, and tactile analyses can be found in Meilgaard et al. (2006).

8

1 Introduction

the food product (the stimulus). However, since a sensory perception involves the necessary interaction of a person with a stimulus, it should be apparent that similar test methods are necessary to characterize this person-product interaction.

1.2.4 The Central Dogma-Analytic Versus Hedonic Tests

Fig. 1.2 The 9-point hedonic scale used to assess liking and disliking. This scale, originally developed at the U.S. Army Food and Container Institute (Quartermaster Corps), has achieved widespad use in consumer testing of foods.

The central principle for all sensory evaluation is that the test method should be matched to the objectives of the test. Figure 1.3 shows how the selection of the test procedure flows from questions about the objective of the investigation. To fulfill this goal, it is necessary to have clear communication between the sensory test manager and the client or end-user of the information. A dialogue is often needed. Is the important question whether or not there is any difference at all among the products? If so, a discrimination test is indicated. Is the question one of whether consumers like the new product better than the pvious version? A consumer acceptance test is needed. Do we need to know what attributes have changed in the sensory characteristics of the new product? Then a descriptive analysis procedure is called for. Sometimes there are multiple objectives and a sequence of different tests is required (Lawless and Claassen, 1993). This can psent problems if all the answers are required at once or under severe time pssure during competitive product development. One of the most important jobs of the sensory specialist in the food industry is to insure a clear understanding and specification of the type of information needed by the end-users. Test design may require a number of conversations, interviews with different people, or even written test requests that specify why the information is to be collected and how the results will be used in making specific decisions and subsequent actions to be taken. The sensory specialist is the best position to understand the uses and limitations of each procedure and what would be considered appropriate versus inappropriate conclusions from the data. There are two important corollaries to this principle. The sensory test design involves not only the selection of an appropriate method but also the selection of appropriate participants and statistical analyses. The three classes of sensory tests can be pided into two types, analytical sensory tests including discrimination

1.2 Historical Landmarks and the Three Classes of Test Methods

9

Fig. 1.3 A flowchart showing methods determination. Based on the major objectives and research questions, different sensory test methods are selected. Similar decision processes are made in panelist selection, setting up response scales, in choosing experimental designs, statistical analysis, and other tasks in designing a sensory test (reprinted with permission from Lawless, 1993).

and descriptive methods and affective or hedonic tests such as those involved in assessing consumer liking or pferences (Lawless and Claassen, 1993). For the analytical tests, panelists are selected based on having average to good sensory acuity for the critical characteristics (tastes, smells, textures, etc.) of products to be evaluated. They are familiarized with the test procedures and may undergo greater or lesser amounts of training, depending upon the method. In the case of descriptive analysis, they adopt an analytical frame of mind, focusing on specific aspects of the product as directed by the scales on their questionnaires. They are asked to put personal pferences and hedonic reactions aside, as their job is only to specify what attributes are psent in the product and at what levels of sensory intensity, extent, amount, or duration. In contrast to this analytical frame of mind, consumers in an affective test act in a much more integrative fashion. They perceive a product as a whole pattern. Although their attention is sometimes captured by a specific aspect of a product (especially if it is a bad, unexpected, or unpleasant one), their reactions to the product are often immediate and based on the integrated pattern of sensory stimulation from the product and expssed as liking or disliking. This occurs without a great deal of thought or dissection of the product’s specific profile. In other words, consumers are effective at rendering impssions based on the integrated pattern of perceptions. In such consumer

tests, participants must be chosen carefully to insure that the results will generalize to the population of interest. Participants should be frequent users of the product, since they are most likely to form the target market and will be familiar with similar products. They possess reasonable expectations and a frame of reference within which they can form an opinion relative to other similar products they have tried. The analytic/hedonic distinction gives rise to some highly important rules of thumb and some warnings about matching test methods and respondents. It is unwise to ask trained panelists about their pferences or whether they like or dislike a product. They have been asked to assume a different, more analytical frame of mind and to place personal pference aside. Furthermore, they have not necessarily been selected to be frequent users of the product, so they are not part of the target population to which one would like to generalize hedonic test results. A common analogy here is to an analytical instrument. You would not ask a gas chromatograph or a pH meter whether it liked the product, so why ask your analytical descriptive panel (O’Mahony, 1979). Conversely, problems arise when consumers are asked to furnish very specific information about product attributes. Consumers not only act in a non-analytic frame of mind but also often have very fuzzy concepts about specific attributes, confusing sour and bitter tastes, for example. Inpiduals often differ markedly

10

in their interptations of sensory attribute words on a questionnaire. While a trained texture profile panel has no trouble in agreeing how cohesive a product is after chewing, we cannot expect consumers to provide pcise information on such a specific and technical attribute. In summary, we avoid using trained panelists for affective information and we avoid asking consumers about specific analytical attributes. Related to the analytic-hedonic distinction is the question of whether experimental control and pcision are to be maximized or whether validity and generalizability to the real world are more important. Often there is a tradeoff between the two and it is difficult to maximize both simultaneously. Analytic tests in the lab with specially screened and trained judges are more reliable and lower in random error than consumer tests. However, we give up a certain amount of generalizability to real-world results by using artificial conditions and a special group of participants. Conversely, in the testing of products by consumers in their own homes we have not only a lot of real-life validity but also a lot of noise in the data. Brinberg and McGrath (1985) have termed this struggle between pcision and validity one of “conflicting desiderata.” O’Mahony (1988) has made a distinction between sensory evaluation Type I and Type II. In Type I sensory evaluation, reliability and sensitivity are key factors, and the participant is viewed much like an analytical instrument used to detect and measure changes in a food product. In Type II sensory evaluation, participants are chosen to be repsentative of the consuming population, and they may evaluate food under more naturalistic conditions. Their emphasis here is on pdiction of consumer response. Every sensory test falls somewhere along a continuum where reliability versus real-life extrapolation are in a potential tradeoff relationship. This factor must also be discussed with end-users of the data to see where their emphasis lies and what level of tradeoff they find comfortable. Statistical analyses must also be chosen with an eye to the nature of the data. Discrimination tests involve choices and counting numbers of correct responses. The statistics derived from the binomial distribution or those designed for proportions such as chi-square are appropriate. Conversely, for most scaled data, we can apply the familiar parametric statistics appropriate to normally distributed and continuous data, such as means, standard deviations, t-tests, analysis of variance. The choice of an appropriate statistical test is not

1 Introduction

always straightforward, so sensory specialists are wise to have thorough training in statistics and to involve statistical and design specialists in a complex project in its earliest stages of planning. Occasionally, these central principles are violated. They should not be put aside as a matter of mere expediency or cost savings and never without a logical analysis. One common example is the use of a discrimination test before consumer acceptance. Although our ultimate interest may lie in whether consumers will like or dislike a new product variation, we can conduct a simple difference test to see whether any change is perceivable at all. The logic in this sequence is the following: if a screened and experienced discrimination panel cannot tell the difference under carefully controlled conditions in the sensory laboratory, then a more heterogeneous group of consumers is unlikely to see a difference in their less controlled and more variable world. If no difference is perceived, there can logically be no systematic pference. So a more time consuming and costly consumer test can sometimes be avoided by conducting a simpler but more sensitive discrimination test first. The added reliability of the controlled discrimination test provides a “safety net” for conclusions about consumer perception. Of course, this logic is not without its pitfalls-some consumers may interact extensively with the product during a home use test period and may form stable and important opinions that are not captured in a short duration laboratory test, and there is also always the possibility of a false negative result (the error of missing a difference). MacRae and Geelhoed (1992) describe an interesting case of a missed difference in a triangle test where a significant pference was then observed between water samples in a paired comparison. The sensory professional must be aware that these anomalies in experimental results will sometimes arise, and must also be aware of some of the reasons why they occur.

1.3 Applications: Why Collect Sensory Data? Human perceptions of foods and consumer products are the results of complex sensory and interptation processes. At this stage in scientific history, perceptions of such multidimensional stimuli as conducted

1.3 Applications: Why Collect Sensory Data?

by the parallel processing of the human nervous system are difficult or impossible to pdict from instrumental measures. In many cases instruments lack the sensitivity of human sensory systems-smell is a good example. Instruments rarely mimic the mechanical manipulation of foods when tasted nor do they mimic the types of peri-receptor filtering that occur in biological fluids like saliva or mucus that can cause chemical partitioning of flavor materials. Most importantly, instrumental assessments give values that miss an important perceptual process: the interptation of sensory experience by the human brain prior to responding. The brain lies interposed between sensory input and the generation of responses that form our data. It is a massively parallel-distributed processor and computational engine, capable of rapid feats of pattern recognition. It comes to the sensory evaluation task complete with a personal history and experiential frame of reference. Sensory experience is interpted, given meaning within the frame of reference, evaluated relative to expectations and can involve integration of multiple simultaneous or sequential inputs. Finally judgments are rendered as our data. Thus there is a “chain of perception” rather than simply stimulus and response (Meilgaard et al., 2006). Only human sensory data provide the best models for how consumers are likely to perceive and react to food products in real life. We collect, analyze, and interpt sensory data to form pdictions about how products have changed during a product development program. In the food and consumer products industries, these changes arise from three important factors: ingredients, processes, and packaging. A fourth consideration is often the way a product ages, in other words its shelf life, but we may consider shelf stability to be one special case of processing, albeit usually a very passive one (but also consider products exposed to temperature fluctuation, light-catalyzed oxidation, microbial contamination, and other “abuses”). Ingredient changes arise for a number of reasons. They may be introduced to improve product quality, to reduce costs of production, or simply because a certain supply of raw materials has become unavailable. Processing changes likewise arise from the attempt to improve quality in terms of sensory, nutritional, microbiological stability factors, to reduce costs or to improve manufacturing productivity. Packaging changes arise from considerations of product stability or other quality factors, e.g., a certain

11

amount of oxygen permeability may insure that a fresh beef product remains red in color for improved visual appeal to consumers. Packages function as carriers of product information and brand image, so both sensory characteristics and expectations can change as a function of how this information can be carried and displayed by the packaging material and its print overlay. Packaging and print ink may cause changes in flavor or aroma due to flavor transfer out of the product and sometimes transfer of off-flavors into the product. The package also serves as an important barrier to oxidative changes, to the potentially deleterious effects of light-catalyzed reactions, and to microbial infestations and other nuisances. The sensory test is conducted to study how these product manipulations will create perceived changes to human observers. In this sense, sensory evaluation is in the best traditions of psychophysics, the oldest branch of scientific psychology, that attempts to specify the relationships between different energy levels impinging upon the sensory organs (the physical part of psychophysics) and the human response (the psychological part). Often, one cannot pdict exactly what the sensory change will be as a function of ingredients, processes, or packaging, or it is very difficult to do so since foods and consumer products are usually quite complex systems. Flavors and aromas depend upon complex mixtures of many volatile chemicals. Informal tasting in the lab may not bring a reliable or sufficient answer to sensory questions. The benchtop in the development laboratory is a poor place to judge potential sensory impact with distractions, competing odors, nonstandard lighting, and so on. Finally, the nose, eyes, and tongue of the product developer may not be repsentative of most other people who will buy the product. So there is some uncertainty about how consumers will view a product especially under more natural conditions. Uncertainty is the key here. If the outcome of a sensory test is perfectly known and pdictable, there is no need to conduct the formal evaluation. Unfortunately, useless tests are often requested of a sensory testing group in the industrial setting. The burden of useless routine tests arises from overly entrenched product development sequences, corporate traditions, or merely the desire to protect oneself from blame in the case of unexpected failures. However, the sensory test is only as useful as the amount of reduction in uncertainty that occurs. If there is no uncertainty, there

12

is no need for the sensory test. For example, doing a sensory test to see if there is a perceptible color difference between a commercial red wine and a commercial white wine is a waste of resources, since there is no uncertainty! In the industrial setting, sensory evaluation provides a conduit for information that is useful in management business decisions about directions for product development and product changes. These decisions are based on lower uncertainty and lower risk once the sensory information is provided. Sensory evaluation also functions for other purposes. It may be quite useful or even necessary to include sensory analyses in quality control (QC) or quality assurance. Modification of traditional sensory practices may be required to accommodate the small panels and rapid assessments often required in online QC in the manufacturing environment. Due to the time needed to assemble a panel, ppare samples for testing, analyze and report sensory data, it can be quite challenging to apply sensory techniques to quality control as an on-line assessment. Quality assurance involving sensory assessments of finished products are more readily amenable to sensory testing and may be integrated with routine programs for shelf life assessment or quality monitoring. Often it is desirable to establish correlations between sensory response and instrumental measures. If this is done well, the instrumental measure can sometimes be substituted for the sensory test. This is especially applicable under conditions in which rapid turnaround is needed. Substitution of instrumental measurements for sensory data may also be useful if the evaluations are likely to be fatiguing to the senses, repetitive, involve risk in repeated evaluations (e.g., insecticide fragrances), and are not high in business risk if unexpected sensory problems arise that were missed. In addition to these product-focused areas of testing, sensory research is valuable in a broader context. A sensory test may help to understand the attributes of a product that consumers view as critical to product acceptance and thus success. While we keep a wary eye on the fuzzy way that consumers use language, consumer sensory tests can provide diagnostic information about a product’s points of superiority or shortcomings. Consumer sensory evaluations may suggest hypotheses for further inquiry such as exploration of new product opportunities. There are recurrent themes and enduring problems in sensory science. In 1989, the ASTM Committee

1 Introduction

1.3 Applications: Why Collect Sensory Data?

(4)

(5)

(6)

(7)

control in multi-plant manufacturing situations, as well as international product development and the problem of multiple sensory testing sites and panels. Environmental and biochemical factors. Kamen recognized that pferences may change as a function of the situation (food often tastes better outdoors and when you are hungry). Meiselman (1993) questioned whether sufficient sensory research is being performed in realistic eating situations that may be more pdictive of consumer reactions, and recently sensory scientists have started to explore this area of research (for example, Giboreau and Fleury, 2009; Hein et al., 2009; Mielby and Frøst, 2009). Resolving discrepancies between laboratory and field studies. In the search for reliable, detailed, and pcise analytical methods in the sensory laboratory, some accuracy in pdicting field test results may be lost. Management must be aware of the potential of false positive or negative results if a full testing sequence is not carried out, i.e., if shortcuts are made in the testing sequence prior to marketing a new product. Sensory evaluation specialists in industry do not always have time to study the level of correlation between laboratory and field tests, but a prudent sensory program would include periodic checks on this issue. Inpidual differences. Since Kamen’s era, a growing literature has illuminated the fact that human panelists are not identical, interchangeable measuring instruments. Each comes with different physiological equipment, different frames of reference, different abilities to focus and maintain attention, and different motivational resources. As an example of differences in physiology, we have the growing literature on specific anosmias-smell “blindnesses” to specific chemical compounds among persons with otherwise normal senses of smell (Boyle et al., 2006; Plotto et al., 2006; Wysocki and Labows, 1984). It should not be surprising that some olfactory characteristics are difficult for even trained panelists to evaluate and to come to agreement (Bett and Johnson, 1996). Relating sensory differences to product variables. This is certainly the meat of sensory science in industrial practice. However, many product developers do not sufficiently involve their sensory specialists in the underlying research questions.

13

They also may fall into the trap of never ending sequences of paired tests, with little or no planned designs and no modeling of how underlying physical variables (ingredients, processes) create a dynamic range of sensory changes. The relation of graded physical changes to sensory response is the essence of psychophysical thinking. (8) Sensory interactions. Foods and consumer products are multidimensional. The more sensory scientists understand interactions among characteristics such as enhancement and masking effects, the better they can interpt the results of sensory tests and provide informed judgments and reasoned conclusions in addition to reporting just numbers and statistical significance. (9) Sensory education. End-users of sensory data and people who request sensory tests often expect one tool to answer all questions. Kamen cited the simple dichotomy between analytical and hedonic testing (e.g., discrimination versus pference) and how explaining this difference was a constant task. Due to the lack of widespad training in sensory science, the task of sensory education is still with us today, and a sensory professional must be able to explain the rationale behind test methods and communicate the importance and logic of sensory technology to non-sensory scientists and managers.

14 Table 1.3 Contrast of sensory evaluation consumer tests with market research tests

1 Introduction Sensory testing with consumers Participants screened to be users of the product category Blind-labeled samples-random codes with minimal conceptual information Determines if sensory properties and overall appeal met targets Expectations based on similar products used in the category Not intended to assess response/appeal of product concept Market research testing (concept and/or product test) Participants in product-testing phase selected for positive response to concept Conceptual claims, information, and frame of reference are explicit Expectations derived from concept/claims and similar product usage Unable to measure sensory appeal in isolation from concept and expectations

designed to make the product conceptually appealing (e.g., bringing attention to convenience factors in pparation). In a sensory test all these potentially biasing factors are stripped away in order to isolate the opinion based on sensory properties only. In the tradition of scientific inquiry, we need to isolate the variables of interest (ingredients, processing, packaging changes) and assess sensory properties as a function of these variables, and not as a function of conceptual influences. This is done to minimize the influence of a larger cognitive load of expectations generated from complex conceptual information. There are many potential response biases and task demands that are entailed in “selling” an idea as well as in selling a product. Participants often like to please the experimenter and give results consistent with what they think the person wants. There is a large literature on the effect of factors such as brand label on consumer response. Product information interacts in complex ways with consumer attitudes and expectancies (Aaron et al., 1994; Barrios and Costell, 2004; Cardello and Sawyer, 1992; Costell et al., 2009; Deliza and MacFie, 1996; Giménez et al., 2008; Kimura et al., 2008; Mielby and Frøst, 2009; Park and Lee, 2003; Shepherd et al., 1991/1992). Expectations can cause assimilation of sensory reactions toward what is expected under some conditions and under other conditions will show contrast effects, enhancing differences when expectations are not met (Siegrist and Cousin, 2009; Lee et al., 2006; Yeomans et al., 2008; Zellner et al., 2004). Packaging and brand information will also affect sensory judgments (Dantas et al., 2004; Deliza et al., 1999; Enneking et al., 2007). So the apparent resemblance of a blind sensory test and a fully concept-loaded market research test are quite illusory. Corporate management needs to be reminded of this important distinction. There continues to be

1.3 Applications: Why Collect Sensory Data?

1.3.2 Differences from Traditional Product Grading Systems A second arena of apparent similarity to sensory evaluation is with the traditional product quality grading systems that use sensory criteria. The grading of agricultural commodities is a historically important influence on the movement to assure consumers of quality standards in the foods they purchase. Such techniques were widely applicable to simple products such as fluid milk and butter (Bodyfelt et al., 1988, 2008), where an ideal product could be largely agreed upon and the defects that could arise in poor handling and processing gave rise to well-known sensory effects. Further impetus came from the fact that competitions could be held to examine whether novice judges-in-training could match the opinions of experts. This is much in the tradition of livestock grading-a young person could judge a cow and receive awards at a state fair for learning to use the same criteria and critical eye as the expert judges. There are noteworthy differences in the ways in which sensory testing and quality judging are performed. Some of these are outlined in Table 1.4. The commodity grading and the inspection tradition have severe limitations in the current era of highly processed foods and market segmentation. There are fewer and fewer “standard products” relative to the wide variation in flavors, nutrient levels (e.g., low fat), convenience pparations, and other choices that

15

line the supermarket shelves. Also, one person’s product defect may be another’s marketing bonanza, as in the glue that did not work so well that gave us the ubiquitous post-it notes. Quality judging methods are poorly suited to research support programs. The techniques have been widely criticized on a number of scientific grounds (Claassen and Lawless, 1992; Drake, 2007; O’Mahony, 1979; Pangborn and Dunkley, 1964; Sidel et al., 1981), although they still have their proponents in industry and agriculture (Bodyfelt et al., 1988, 2008). The defect identification in quality grading emphasizes root causes (e.g., oxidized flavor) whereas the descriptive approach uses more elemental singular terms to describe perceptions rather than to infer causes. In the case of oxidized flavors, the descriptive analysis panel might use a number of terms (oily, painty, and fishy) since oxidation causes a number of qualitatively different sensory effects. Another notable difference from mainstream sensory evaluation is that the quality judgments combine an overall quality scale (psumably reflecting consumer dislikes) with diagnostic information about defects, a kind of descriptive analysis looking only at the negative aspects of products. In mainstream sensory evaluation, the descriptive function and the consumer evaluation would be clearly separate in two distinct tests with different respondents. Whether the opinion of a single expert can effectively repsent consumer opinion is highly questionable at this time in history.

Table 1.4 Contrast of sensory evaluation tests with quality inspection Sensory testing Separates hedonic (like-dislike) and descriptive information into separate tests Uses repsentative consumers for assessment of product appeal (liking/disliking) Uses trained panelists to specify attributes, but not liking/disliking Oriented to research support Flexible for new, engineered, and innovative products Emphasizes statistical inference for decision making, suitable experimental designs, and sample sizes Quality inspection Used for pass-fail online decisions in manufacturing Provides quality score and diagnostic information concerning defects in one test Uses sensory expertise of highly trained inpiduals May use only one or very few trained experts Product knowledge, potential problems, and causes are stressed Traditional scales are multi-dimensional and poorly suited to statistical analyses Decision-making basis may be qualitative Oriented to standard commodities

16

1 Introduction

1.4 Summary and Conclusions Sensory evaluation comprises a set of test methods with guidelines and established techniques for product psentation, well-defined response tasks, statistical methods, and guidelines for interptation of results. Three primary kinds of sensory tests focus on the

existence of overall differences among products (discrimination tests), specification of attributes (descriptive analysis), and measuring consumer likes and dislikes (affective or hedonic testing). Correct application of sensory technique involves correct matching of method to the objective of the tests, and this requires good communication between sensory specialists and

Methods Selection

yes Consumer Acceptability Question? no Choose from: Preference/choice Ranking Rated Acceptability

no go to Panel Setup

Sensory Analytical Question? yes

yes

Simple Same/different Question?

re-open discussion of objectives

no

no

Choose from: Overall difference tests n-alternative forced choice Rated difference from control

go to Panel Setup Nature of Difference Question? yes

Choose from: descriptive analysis techniques or modifications

go to Panel Setup

Fig. 1.4 A sensory evaluation department may interact with many other departments in a food or consumer products company. Their primary interaction is in support of product research and development, much as marketing research supports the

References

end-users of the test results. Logical choices of test participants and appropriate statistical analyses form part of the methodological mix. Analytic tests such as the discrimination and descriptive procedures require good experimental control and maximization of test pcision. Affective tests on the other hand require use of repsentative consumers of the products and test conditions that enable generalization to how products are experienced by consumers in the real world. Sensory tests provide useful information about the human perception of product changes due to ingredients, processing, packaging, or shelf life. Sensory evaluation departments not only interact most heavily with new product development groups but may also provide information to quality control, marketing research, packaging, and, indirectly, to other groups throughout a company (Fig. 1.4). Sensory information reduces risk in decisions about product development and strategies for meeting consumer needs. A wellfunctioning sensory program will be useful to a company in meeting consumer expectations and insuring a greater chance of marketplace success. The utility of the information provided is directly related to the quality of the sensory measurement. . . . , sensory food science stands at the intersection of many disciplines and research traditions, and the stakeholders are many (Tuorila and Monteleone, 2009). Quantities derive from measurement, ps from quantities, comparisons from ps, and victory from comparisons (Sun Tzu – The Art of War (Ch. 4, v.18)).

References Aaron, J. I., Mela, D. J. and Evans, R. E. 1994. The influence of attitudes, beliefs and label information on perceptions of reduced-fat spad. Appetite, 22(1), 25-38. Adams, H. 1918. The Education of Henry Adams. The Modern Library, New York. Amerine, M. A., Pangborn, R. M. and Roessler, E. B. 1965. Principles of Sensory Evaluation of Food. Academic, New York. ASTM E1958. 2008. Standard guide for sensory claim substantiation. ASTM International, West Conshohocken, PA. ASTM. 1989. Sensory evaluation. In celebration of our beginnings. Committee E-18 on Sensory Evaluation of Materials and Products. ASTM, Philadelphia. Barrios, E. X. and Costell, E. 2004. Review: use of methods of research into consumers’ opinions and attitudes in food research. Food Science and Technology International, 10, 359-371.

18 Giboreau, A. and Fleury, H. 2009. A new research platform to contribute to the pleasure of eating and healthy food behaviors through academic and applied food and hospitality research. Food Quality and Preference, 20, 533-536 Giménez, A., Ares, G. and Gámbaro, A. 2008. Consumer attitude toward shelf-life labeling: does it influence acceptance? Journal of Sensory Studies, 23, 871-883. Hein, K. A., Hamid, N., Jaeger, S. R. and Delahunty, C. M. 2009. Application of a written scenario to evoke a consumption context in a laboratory setting: effects on hedonic ratings. Food Quality and Preference. doi:10.1016/j.foodqual.2009.10.003 Helm, E. and Trolle, B. 1946. Selection of a taste panel. Wallerstein Laboratory Communications, 9, 181-194. Jones, L. V., Peryam, D. R. and Thurstone, L. L. 1955. Development of a scale for measuring soldier’s food pferences. Food Research, 20, 512-520. Kamen, J. 1989. Observations, reminiscences and chatter. In: Sensory Evaluation. In celebration of our Beginnings. Committee E-18 on Sensory Evaluation of Materials and Products. ASTM, Philadelphia, pp. 118-122. Kimura, A., Wada, Y., Tsuzuki, D., Goto, S., Cai, D. and Dan, I. 2008. Consumer valuation of packaged foods. Interactive effects of amount and accessibility of information. Appetite, 51, 628-634. Lawless, H. T. 1993. The education and training of sensory scientists. Food Quality and Preference, 4, 51-63. Lawless, H. T. and Claassen, M. R. 1993. The central dogma in sensory evaluation. Food Technology, 47(6), 139-146. Lawless, H. T. and Klein, B. P. 1989. Academic vs. industrial perspectives on sensory evaluation. Journal of Sensory Studies, 3, 205-216. Lee, L., Frederick, S. and Ariely, D. 2006. Try it, you’ll like it. Psychological Science, 17, 1054-1058. MacRae, R. W. and Geelhoed, E. N. 1992. Preference can be more powerful than detection of oddity as a test of discriminability. Perception and Psychophysics, 51, 179-181. Meilgaard, M., Civille, G. V. and Carr, B. T. 2006. Sensory Evaluation Techniques. Fourth Second edition. CRC, Boca Raton. Meiselman, H. L. 1993. Critical evaluation of sensory techniques. Food Quality and Preference, 4, 33-40. Mielby, L. H. and Frøst, M. B. 2009. Expectations and surprise in a molecular gastronomic meal. Food Quality and Preference. doi:10.1016/j.foodqual.2009.09.005 Moskowitz, H. R., Beckley, J. H. and Resurreccion, A. V. A. 2006. Sensory and Consumer Research in Food Product Design and Development. Wiley-Blackwell, New York. Moskowitz, H. R. 1983. Product Testing and Sensory Evaluation of Foods. Food and Nutrition, Westport, CT. Oliver, T. 1986. The Real Coke, The Real Story. Random House, New York. O’Mahony, M. 1988. Sensory difference and pference testing: The use of signal detection measures. Chpater 8 In: H. R. Moskowitz (ed.), Applied Sensory Analysis of Foods. CRC, Boca Raton, FL, pp. 145-175.

1 Introduction O’Mahony, M. 1979. Psychophysical aspects of sensory analysis of dairy products: a critique. Journal of Dairy Science, 62, 1954-1962. Pangborn, R. M. and Dunkley, W. L. 1964. Laboratory procedures for evaluating the sensory properties of milk. Dairy Science Abstracts, 26, 55-121. Park, H. S. and Lee, S. Y. 2003. Genetically engineered food labels, information or warning to consumers? Journal of Food Products Marketing, 9, 49-61. Peryam, D. R. and Swartz, V. W. 1950. Measurement of sensory differences. Food Technology, 4, 390-395. Plotto, A., Barnes, K. W. and Goodner, K. L. 2006. Specific anosmia observed for β-ionone, but not for α-ionone: Significance for flavor research. Journal of Food Science, 71, S401-S406. Shepherd, R., Sparks, P., Belleir, S. and Raats, M. M. 1991/1992. The effects of information on sensory ratings and pferences: The importance of attitudes. Food Quality and Preference, 3, 1-9. Sidel, J. L., Stone, H. and Bloomquist, J. 1981. Use and misuse of sensory evaluation in research and quality control. Journal of Dairy Science, 61, 2296-2302. Siegrist, M. and Cousin, M-E. 2009. Expectations influence sensory experience in a wine tasting. Appetite, 52, 762-765. Skinner, E. Z. 1989. (Commentary). Sensory evaluation. In celebration of our beginnings. Committee E-18 on Sensory Evaluation of Materials and Products. ASTM, Philadelphia, pp. 58-65. Stone, H. and Sidel, J. L. 2004. Sensory Evaluation Practices, Third Edition. Academic, San Deigo. Stone, H., Sidel, J., Oliver, S., Woolsey, A. and Singleton, R. C. 1974. Sensory evaluation by quantitative descriptive analysis. Food Technology 28(1), 24, 26, 28, 29, 32, 34. Sun Tzu (Sun Wu) 1963 (trans.), orig. circa 350 B.C.E. The Art of War. S.B. Griffith, trans. Oxford University. Szczesniak, A. S., Loew, B. J. and Skinner, E. Z. 1975. Consumer texture profile technique. Journal of Food Science, 40, 1253-1257. Tuorila, H. and Monteleone, E. 2009. Sensory food science in the changing society: opportunities, needs and challenges. Trends in Food Science and Technology, 20, 54-62. Wysocki, C. J. and Labows, J. 1984. Inpidual differences in odor perception. Perfumer and Flavorist, 9, 21-24. Yeomans, M. R., Chambers, L., Blumenthal, H. and Blake, A. 2008. The role of expectation in sensory and hedonic evaluation: The case of salmon smoked ice-cream. Food Quality and Preference, 19, 565-573. York, R. K. 1995. Quality assessment in a regulatory environment. Food Quality and Preference, 6, 137-141. Zellner, D. A., Strickhouser, D. and Tornow, C. E. 2004. Disconfirmed hedonic expectations produce perceptual contrast, not assimilation. The American Journal of Psychology, 117, 363-387.

Chapter 2

Physiological and Psychological Foundations of Sensory Function

Abstract This chapter reviews background material underpinning sensory science and sensory evaluation methodologies. Basic and historical psychophysical methods are reviewed as well as the anatomy, physiology, and function of the chemical senses. The chapter concludes with a discussion of multi-modal sensory interactions. There is no conception in man’s mind which hath not at first, totally or in parts, been begotten upon by the organs of sense. -Thomas Hobbes, Leviathan (1651)

Contents 2.1 2.2

2.3

2.4

2.5

Introduction . . . . . . . . . . . . . . . . . Classical Sensory Testing and Psychophysical Methods . . . . . . . . . . . . . . . . . . . 2.2.1 Early Psychophysics . . . . . . . . . . 2.2.2 The Classical Psychophysical Methods . . . . . . . . . . . . . . . 2.2.3 Scaling and Magnitude Estimation . . . 2.2.4 Critiques of Stevens . . . . . . . . . . 2.2.5 Empirical Versus Theory-Driven Functions . . . . . . . . . . . . . . . 2.2.6 Parallels of Psychophysics and Sensory Evaluation . . . . . . . . . . . . . . Anatomy and Physiology and Functions of Taste . . . . . . . . . . . . . . . . . . . 2.3.1 Anatomy and Physiology . . . . . . . . 2.3.2 Taste Perception: Qualities . . . . . . . 2.3.3 Taste Perception: Adaptation and Mixture Interactions . . . . . . . . . . . . . . 2.3.4 Inpidual Differences and Taste Genetics . . . . . . . . . . . . . . . Anatomy and Physiology and Functions of Smell . . . . . . . . . . . . . . . . . . . 2.4.1 Anatomy and Cellular Function . . . . . 2.4.2 Retronasal Smell . . . . . . . . . . . 2.4.3 Olfactory Sensitivity and Specific Anosmia . . . . . . . . . . . . . . . 2.4.4 Odor Qualities: Practical Systems . . . . 2.4.5 Functional Properties: Adaptation, Mixture Suppssion, and Release . . . . . . . . Chemesthesis . . . . . . . . . . . . . . . . 2.5.1 Qualities of Chemesthetic Experience . . . . . . . . . . . . . .

2.5.2

19 20 20 21 23 25 25

Physiological Mechanisms of Chemesthesis . . . . . . . . . . . 2.5.3 Chemical “Heat” . . . . . . . . . . 2.5.4 Other Irritative Sensations and Chemical Cooling . . . . . . . . . . . . . . 2.5.5 Astringency . . . . . . . . . . . . . 2.5.6 Metallic Taste . . . . . . . . . . . . 2.6 Multi-modal Sensory Interactions . . . . . 2.6.1 Taste and Odor Interactions . . . . . . 2.6.2 Irritation and Flavor . . . . . . . . . 2.6.3 Color-Flavor Interactions . . . . . . 2.7 Conclusions . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . .

. .

42 43

. . . . . . . . .

44 45 46 47 47 49 49 50 50

26

2.1 Introduction 27 27 30 30 33 34 34 36 37 38 39 41 41

In order to design effective sensory tests and provide insightful interptation of the results, a sensory professional must understand the functional properties of the sensory systems that are responsible for the data. By a functional property, we mean a phenomenon like mixture interactions such as masking or suppssion. Another example is sensory adaptation, a commonly observed decrease in responsiveness to conditions of more or less constant stimulation. In addition, it is useful to understand the anatomy and physiology of the senses involved as well as their functional limitations. A good example of a functional limitation is the threshold or minimal amount of a stimulus needed for perception. Knowing about the anatomy of the senses

H.T. Lawless, H. Heymann, Sensory Evaluation of Food, Food Science Text Series, DOI 10.1007/978-1-4419-6488-5_2, © Springer Science+Business Media, LLC 2010

19

20

2

Physiological and Psychological Foundations of Sensory Function

can help us understand how consumers and panelists interact with the products to stimulate their senses and by what routes. Different routes of smelling, for example, are the orthonasal or sniffing route, when odor molecules enter the nose from the front (nostrils), versus retronasal smell, when odor molecules pass into the nose from the mouth or from breathing out, and thus have a reversed airflow pathway from that of external sniffing. Another basic area that the sensory professional should have as background knowledge involves the sensory testing methods and human measurement procedures that are the historical antecedents to the tests we do today. This is part of the science of psychophysics, the quantification and measurement of sensory experiences. Psychophysics is a very old discipline that formed the basis for the early studies in experimental psychology. Parallels exist between psychophysics and sensory evaluation. For example, the difference test using paired comparisons is a version of the method used for measuring difference thresholds called the method of constant stimuli. In descriptive analysis with trained panels, we work very hard to insure that panelists use singular uni-dimensional scales. These numerical systems usually refer to a single sensory continuum like sweetness or odor strength and are thus based on changes in perceived intensity. They do not consider multiple attributes and fold them into a single score like the old-quality grading methods. Thus there is a clear psychophysical basis for the attribute scales used in descriptive analysis. This chapter is designed to provide the reader some background in the sensory methods of psychophysics. A second objective is to give an overview of the structure and function of the chemical senses of taste, smell, and the chemesthetic sense. Chemesthesis refers to chemically induced sensations that seem to be at least partly tactile in nature, such as pepper heat, astringency, and chemical cooling. These three senses together comprise what we loosely call flavor and are the critical senses for appciating foods, along with the tactile, force, and motion-related experiences that are part of food texture and mouthfeel. Texture is dealt with in Chapter 11 and color and appearance evaluations in Chapter 12. The auditory sense is not a large part of food perception, although many sounds can be perceived when we eat or manipulate foods. These provide another sense modality to accompany and reinforce our texture perceptions, as in the case of crisp

or crunchy foods, or the audible hissing sound we get from carbonated beverages (Vickers, 1991). One growing area of interest in the senses concerns our human biopersity, differences among people in sensory function. These differences can be due to genetic, dietary/nutritional, physiological (e.g., aging), or environmental factors. The research into the genetics of the chemical senses, for example, has experienced a period of enormous expansion since the first edition of this book. The topic is too large and too rapidly changing to receive a comphensive treatment here. We will limit our discussion of inpidual differences and genetic factors to those areas that are well understood, such as bitter sensitivity, smell blindness, and color vision anomalies. The sensory practitioner should be mindful that people exist in somewhat different sensory worlds. These differences contribute to the persity of consumer pferences. They also limit the degree to which a trained panel can be “calibrated” into uniform ways of responding. Inpidual differences can impact sensory evaluations in many ways.

2.2 Classical Sensory Testing and Psychophysical Methods 2.2.1 Early Psychophysics The oldest branch of experimental psychology is that of psychophysics, the study of relationships between physical stimuli and sensory experience. The first true psychophysical theorist was the nineteenth century German physiologist, E. H. Weber. Building on earlier observations by Bernoulli and others, Weber noted that the amount that a physical stimulus needed to be increased to be just perceivably different was a constant ratio. Thus 14.5 and 15 ounces could be told apart, but with great difficulty, and the same could be said of 29 and 30 ounces or 14.5 and 15 drams (Boring, 1942). This led to the formulation of Weber’s law, generally written nowadays as I/I = k

(2.1)

where I is the increase in the physical stimulus that was required to be just discriminably different from some starting level, I. The fraction, I/I, is sometimes called the “Weber fraction” and is an index of

2.2 Classical Sensory Testing and Psychophysical Methods

how well the sensory system detects changes. This relationship proved generally useful and provided the first quantitative operating characteristic of a sensory system. Methods for determining the difference threshold or just-noticeable-difference (j.n.d.) values became the stock in trade of early psychological researchers. These methods were codified by G. T. Fechner in a book called Elemente der Psychophysik (Elements of Psychophysics) in 1860. Fechner was a philosopher as well as a scientist and developed an interest in Eastern religions, in the nature of the soul, and in the Cartesian mind-body dichotomy. Fechner’s broader philosophical interests have been largely overlooked, but his little book on sensory methods was to become a classic text for the psychology laboratory. Fechner also had a valuable insight. He realized that the j.n.d. might be used as a unit of measurement and that by adding up j.n.d.s one could construct a psychophysical relationship between physical stimulus intensity and sensory intensity. This relationship approximated a log function, since the integral of 1/x dx is proportional to the natural log of x. So a logarithmic relationship appeared useful as a general psychophysical “law:” S = k log I

(2.2)

where S is sensation intensity and I is once again the physical stimulus intensity. This relationship known as Fechner’s law was to prove a useful rule of thumb for nearly 75 years, until it was questioned by acoustical researchers who supplanted it with a power law (see Section 2.2.3).

21

with a particular type of measured response of sensory systems. The method of limits was well suited to determine absolute or detection thresholds. The method of constant stimuli could be used to determine difference thresholds and the method of adjustment to establish sensory equivalence. In the method of limits the physical stimulus is changed by successive discrete steps until a change in response is noted. For example, when the stimulus is increasing in intensity, the response will change from “no sensation” to “I detect something.” When the stimulus is decreasing in intensity, at some step the response will change back to “no sensation.” Over many trials, the average point of change can be taken as the person’s absolute threshold (see Fig. 2.1). This is the minimum intensity required for detection of the stimulus. Modern variations on this method often use only an ascending series and force the participants to choose a target sample among alternative “blank” samples at each step. Each concentration must be discriminated from a background level such as plain water in the case of taste thresholds. Forced-choice methods for determining thresholds are discussed in detail in Chapter 6. In the method of constant stimuli, the test stimulus is always compared against a constant reference level (a standard), usually the middle point on a series of physical intensity levels. The subject’s job is to respond to each test item as “greater than” or “less

2.2.2 The Classical Psychophysical Methods Fechner’s enduring contribution was to assemble and publish the details of sensory test methods and how several important operating characteristics of sensory systems could be measured. Three important methods were the method of limits, the method of constant stimuli (called the method of right and wrong cases in those days), and the method of adjustment or average error (Boring, 1942). The methods are still used today in some research situations and variations on these methods form part of the toolbox of applied sensory evaluation. Each of the three methods was associated

CONCENTRATION

16.0 8.0 4.0 2.0 1.0 0.5 .25 A1

D1

A2

D2

A3

D3

TRIAL

Fig. 2.1 An example of the method of limits. The circled reversal points would be averaged to obtain the person’s threshold. A: ascending series. D: descending series. In taste and smell, only ascending series are commonly used to pvent fatigue, adaptation or carry-over of persistent sensations.

22

2

Physiological and Psychological Foundations of Sensory Function

Frequency (percent) judged “sweeter” than the standard

80

60

40 UDL

20 standard

2

4

6

8

10

12

14

16

18

SUCROSE CONCENTRATION (%)

Fig. 2.2 A psychometric function derived from the Method of Constant Stimuli, a repeated series of paired comparisons against a constant (standard) stimulus, in this case 10% sucrose. Frequency of judgments in which the comparison stimulus is

judged sweeter than the standard are plotted against concentration. The difference threshold is determined by the concentration difference between the standard and the interpolated 75% (or 25%) point. UDL: Upper difference limen (threshold).

than” the standard. Many replications of each intensity level are psented. The percentage of times the response is “greater than” can be plotted as in Fig. 2.2. This S-shaped curve is called a psychometric function (Boring, 1942). The difference threshold was taken as the difference between the 50 and 75% points interpolated on the function. The method of constant stimuli bears a strong resemblance to current techniques of paired comparison, with two exceptions. One point of difference is that the method was geared toward interval estimation, rather than testing for statistically significant differences. That is, the technique estimated points on the psychometric function (25, 50, and 75%) and researchers were not concerned with statistical significance of difference tests. Also, a range of comparison stimuli were tested against the standard and not just a single paired comparison of products. The third major method in classical psychophysics was the method of adjustment or average error. The subject was given control over a variable stimulus like a light or a tone and asked to match a standard in brightness or loudness. The method could be used to determine difference thresholds based on the variability of the subject over many attempts at matching, for example, using the standard deviation as a measure

of difference threshold. A modern application is in measuring sensory tradeoff relationships. In this type of experiment the duration of a very brief tone could be balanced against a varying sound pssure level to yield a constant perception of loudness. Similarly, the duration of a flash of light could be traded off against its photometric intensity to create a constant perceived brightness. For very brief tones or brief flashes, there is summation of the intensity over time in the nervous system, so that increasing duration can be balanced against decreasing physical intensity to create a constant perception. These methods have proven useful in understanding the physiological response of different senses to the temporal properties of stimuli, for example, how the auditory and visual systems integrate energy over time. Adjustment methods have not proven so useful for assessing sensory equivalence in applied food testing, although adjustment is one way of trying to optimize an ingredient level (Hernandez and Lawless, 1999; Mattes and Lawless, 1985). Pangborn and co-workers employed an adjustment method to study inpidual pferences (Pangborn, 1988; Pangborn and Braddock, 1989). Adding flavors or ingredients “to taste” at the benchtop is a common way of initially formulating

2.2 Classical Sensory Testing and Psychophysical Methods

products. It is also fairly common to make formula changes to produce approximate sensory matches to some target, either a standard formula or perhaps some competitor’s successful product. However, the method as applied in the psychophysics laboratory is an unwieldy technique for the senses of taste and smell where elaborate equipment is needed to provide adjustable stimulus control. So methods of equivalency adjustment are somewhat rare with food testing.

23

they are a log scale of sound pssure relative to a reference (db = 20 log (P/P0 ) where P is the sound pssure and P0 is the reference sound pssure, usually a value for absolute threshold). However, discrepancies were observed between decibels and loudness proportions. Instead, Stevens found with the direct magnitude estimation procedure that loudness was a power function of stimulus intensity, with an exponent of about 0.6. Scaling of other sensory continua also gave power functions, each with its characteristic exponent (Stevens, 1957, 1962). Thus the following relationship held:

2.2.3 Scaling and Magnitude Estimation S = kI n or log S = n log I + log k A very useful technique for sensory measurement has been the direct application of rating scales to measure the intensity of sensations. Historically known as the “method of single stimuli,” the procedure is highly cost efficient since one stimulus psentation yields one data point. This is in contrast to a procedure like the method of constant stimuli, where the psentation of many pairs is necessary to give a frequency count of the number of times each level is judged stronger than a standard. Rating scales have many uses. One of the most common is to specify a psychophysical function, a quantitative relationship between the perceived intensity of a sensation and the physical intensity of the stimulus. This is another way of describing a dose-response curve or in other words, capturing the input-output function of a sensory system over its dynamic range. The technique of magnitude estimation grew out of earlier procedures in which subjects would be asked to fractionate an adjustable stimulus. For example, a subject would be asked to adjust a light or tone until it seemed half as bright as a comparison stimulus. The technique was modified so that the experimenter controlled the stimulus and the subject responded using (unrestricted) numbers to indicate the proportions or ratios of the perceived intensities. Thus if the test stimulus was twice as bright as the standard, it would be assigned a number twice as large as the rating for the standard and if one-third as bright, a number one-third as large. An important observation in S. S. Stevens’ laboratory at Harvard was that the loudness of sounds was not exactly proportional to the decibel scale. If Fechner’s log relationship was correct, rated loudness should grow in a linear fashion with decibels, since

(2.3)

where n was the characteristic exponent and k was a proportionality constant determined by the units of measurement. In other words, the function formed a straight line in a log-log plot with the exponent equal to the slope of the linear function. This was in contrast to the Fechnerian log function which was a straight line in a semilog plot (response versus log physical intensity). One of the more important characteristics of a power function is that it can accommodate relationships that are expanding or positively accelerated while the log function does not. The power function with an exponent less than one fits a law of diminishing returns, i.e., larger and larger physical increases are required to maintain a constant proportional increase in the sensation level. Other continua such as response to electric shocks and some tastes were found to have a power function exponent greater than one (Meiselman, 1971; Moskowitz, 1971; Stevens, 1957). A comparison of power functions with different exponents is shown in Fig. 2.3. Many sensory systems show an exponent less that one. This shows a compssive energy relationship that may have adaptive value for an organism responding to a wide range of energy in the environment. The range from the loudest sound one can tolerate to the faintest audible tone is over 100 dB. This repsents over 10 log units of sound energy, a ratio of 10 billion to one. The dynamic range for the visual response of the eyes to different levels of light energy is equally broad. Thus exponents less than one have ecological significance for sensory systems that are tuned to a broad range of physical energy levels.

24

2

Physiological and Psychological Foundations of Sensory Function

Psychological Magnitude, S

Electric Shock

Apparent Length n=1

Brightness n1 n=1 n

1 2

(4.3)

The verbal form of the alternative hypothesis is that the population would be correct (saying that AB and BA pairs are different and that AA and BB pairs are the same) more than half the time. The results of the paired difference test will only indicate whether the panelists could significantly discriminate between the samples. Unlike the paired directional test, no specification or direction of difference is indicated. In other words, the sensory scientist will only know that the samples are perceptibly different but not in which attribute(s) the samples differed. An alternative analysis is psented in the Appendix to this chapter, where each panelist sees an identical pair (AA or BB) and one test pair (AB or BA) in randomized sequence.

4.2.2 Triangle Tests In the triangle test, three samples are psented simultaneously to the panelists, two samples are from the same formulation and one is from the different formulation. Each panelist has to indicate either which sample is the odd sample or which two samples are most similar. The usual form of the score sheet asks the panelist to indicate the odd sample. However, some sensory specialists will ask the panelist to indicate the pair of similar samples. It probably does not matter which question is asked. However, the sensory specialist should not change the format when re-using panelists since they will get confused. See Fig. 4.3 for a sample score sheet. Similarly to the paired difference test the panelist must be trained to understand the task as described by the score sheet. The null hypothesis for the triangle test states that the long-run probability (Pt ) of making a correct selection when there is no perceptible difference between the samples is one in three (H0 :Pt = 1/3). The alternative hypothesis states that the probability that the underlying population will make the correct decision

Fig. 4.3 Example of a triangle score sheet.

84

4

1 3

(4.4)

This is a one-sided alternative hypothesis and the test is one tailed. In this case there are six possible serving orders (AAB, ABA, BAA, BBA, BAB, ABB) which should be counterbalanced across all panelists. As with the difference paired comparison, the triangle test allows the sensory specialist to determine if two samples are perceptibly different but the direction of the difference is not indicated by the triangle test. Again, the sensory scientist will only know that the samples are perceptibly different but not in which attribute(s) the samples differed.

Discrimination Testing

Again, the panelists should be trained to perform the task as described by the score sheet correctly. Duo-trio tests allow the sensory specialist to determine if two samples are perceptibly different but the direction of the difference is not indicated by the duo-trio test. In other words, the sensory scientist will only know that the samples are perceptibly different but not in which attribute(s) the samples differed. There are two formats to the duo-trio test, namely the constant reference duo-trio test and the balanced reference duo-trio test. From the point of view of the panelists the two formats of the duo-trio test are identical (see Figs. 4.4a and b), but to the sensory specialist the two formats differ in the sample(s) used as the reference. 4.2.3.1 Constant Reference Duo-Trio Test

1 2

(4.5)

In this case, all panelists receive the same sample formulation as the reference. The constant reference duo-trio test has two possible serving orders (RA BA, RA AB) which should be counterbalanced across all panelists. The constant reference duo-trio test seems to be more sensitive especially if the panelists have had prior experience with the product (Mitchell, 1956). For example, if product X is the current formulation (familiar to the panelists) and product Z is a new reformulation then a constant reference duo-trio test with product X as reference would be the method of choice.

Before starting please rinse your mouth with water and expectorate. There are three samples in each of the two duo−trio sets for you to evaluate. In each set, one of the coded pairs is the same as the reference. For each set taste the reference first. Then taste each of the coded samples in the sequence psented, from left to right. Take the entire sample in your mouth. NO RETASTING. Circle the number of the sample which is most similar to the reference. Do not swallow any of the sample or the water. Expectorate into the container provided. Rinse your mouth with water between sets 1 and 2.

Fig. 4.4a Example of a constant reference duo-trio score sheet.

Set 1

Reference

2

Reference

4.2 Types of Discrimination Tests

85

Fig. 4.4b Example of a balanced reference duo-trio score sheet.

Reference

2

Reference

reference and the other half of the panelists receive the other sample formulation as the reference. In this case, there are four possible serving orders (RA BA, RA AB, RB AB, RB BA) which should be counterbalanced across all panelists. This method is used when both products are prototypes (unfamiliar to the panelists) or when there is not a sufficient quantity of the more familiar product to perform a constant reference duo-trio test.

differ in the specified dimension and which sample is higher in perceived intensity of the specified attribute. The danger is that other sensory changes will occur in a food when one attribute is modified and these may obscure the attribute in question. Another version of the n-AFC asks panelists to pick out the weakest or strongest in overall intensity, rather than in a specific attribute. This is a very difficult task for panelists when they are confronted with a complex food system.

4.2.5 A-Not-A tests There are two types of A-not-A tests referenced in the literature. The first and the more commonly used version has a training phase with the two products followed by monadic evaluation phase (Bi and Ennis, 2001a, b), we will call this the standard A-not-A test. The second version is essentially a sequential paired difference test or simple difference test (Stone and Sidel, 2004), which we will call the alternate A-notA test. The alternate A-not-A test is not frequently used. In the next section we will discuss the alternate A-not-A test first since the statistical analysis for this version is similar to that of the paired comparison discrimination test. The statistical analyses for the various standard A-not-A tests are based on a different theory and somewhat more complex and will be discussed later.

4.2.5.1 Alternate A-Not-A test This is a sequential same/difference paired difference test where the panelist receives and evaluates the first

86

4

Fig. 4.5 Example of a three-alternative forced choice score sheet.

relevant to the objective of the study. However, the differences in color or shape or size have to be very subtle and only obvious when the samples are psented simultaneously. If the differences are not subtle the panelists are likely to remember these and they will make their decision based on these extraneous differences.

4.2.5.2 Standard A-Not-A Test Panelists inspect multiple examples of products that are labeled “A” and usually also products that are labeled “not-A.” Thus there is a learning period. Then once the training period has been completed the panelists receive samples one at a time and are asked whether each one is either A or not-A. As discussed by Bi and Ennis (2001a) the standard A-not-A test potentially has four different designs. For the monadic A-not-A test the panelist, after the training phase, is psented with a single sample (either A or not-A). In the paired A-not-A version the panelist, after completion of the training phase, is psented with a pair of samples, sequentially (one A and one not-A, counter balanced across panelists). In the replicated monadic A-not-A version the panelist, after completion of training, receives a series of samples of either A or not-A but not both. This version is rarely used in practice. Lastly, in the replicated mixed A-not-A version the panelist, after completion of training, receives a series of A and not-A samples. Each of these different formats requires different statistical models and using an inappropriate model could lead to a misleading conclusion. As described by Bi and Ennis (2001a) “The statistical models for the A-Not A method are different from that of other discrimination methods such as the m-AFC, the triangle, and the duo-trio methods.” “Pearson’s and McNemar’s chi-square statistics with one degree of freedom can be used for the

4.2 Types of Discrimination Tests

standard A-Not A method while binomial tests based on the proportion of correct responses can be used for the m-AFC, the triangle, and the duo-trio methods. The basic difference between the two types of difference tests is that the former involves a comparison of two proportions (i.e., the proportion of “A” responses for the A sample versus that for the Not A sample) or testing independence of two variables (sample and response) while the latter is a comparison of one proportion with a fixed value (i.e., the proportion of correct responses versus the guessing probability)”. Articles by Bi and Ennis (2001a, b) clearly describe data analysis methods for these tests.. Additionally, the article by Brockhoff and Christensen (2009) describes a R-package called SensR (http://www.cran.rproject.org/package=sensR/) that may be used for the data analyses of some Standard A-not-A tests. The data analyses associated with the standard A-not-A tests are beyond the scope of this textbook, but see the Appendix of this chapter which shows the application of the McNemar chi-square for a simple A-not-A test where each panelist received one standard product (a “true” example of A) and one test product. Each is psented separately and a judgment is collected for both products.

87

4.2.6.2 The Harris-Kalmus Test

4.2.6 Sorting Methods In sorting tests the panelists are given a series of samples and they are asked to sort them into two groups. The sorting tests can be extremely fatiguing and are not frequently used for taste and aroma sensory evaluation but they are used when sensory specialists want to determine if two samples are perceptibly different in tactile or visual dimensions. The sorting tests are statistically very efficient since the long-run probability of the null hypotheses of the sorting tests can be very small. For example, the null hypothesis of the two-out-of-five test is 1 in 10 (P2/5 = 0.1) and for the Harris-Kalmus test the null hypothesis is 1 in 70 (P4/8 = 0.0143). These tests are discussed below. 4.2.6.1 The Two-Out-of-Five Test The panelists receive five samples and are asked to sort the samples into two groups, one group should contain the two samples that are different from the other

88

4

Discrimination Testing

4.2.8 Dual-Standard Test The dual standard was first used by Peryam and Swartz (1950) with odor samples. It is essentially a duo-trio test with two reference standards-the control and the variant. The two standards allow the panelists to create a more stable criterion as to the potential difference between the samples. The potential serving orders for this test are R(A) R(B) , AB, R(A) R(B) BA, R(B) R(A) AB, R(B) R(A) BA. The probability of guessing the correct answer by chance is 0.5 and the data analyses for this test are identical to that of the duo-trio test. Peryam and Swartz felt quite strongly that the technique would work best with odor samples due to the relatively quick recovery and that the longer recovery associated with taste samples would pclude the use of the test. The test was used by Pangborn and Dunkley (1966) to detect additions of lactose, algin gum, milk salts, and proteins to milk. O’Mahony et al. (1986) working with lemonade found that the dual-standard test elicited superior performance over the duo-trio test. But O’Mahony (personal communication, 2009) feels that this result is in error, since the panelists were not instructed to evaluate the standards prior to each pair evaluation and therefore the panelists were probably reverting to a 2-AFC methodology. This would be in agreement with Huang and Lawless (1998) who studied sucrose additions to orange juice and they did not find superiority in performance between the dual standard and the duo-trio or the ABX tests.

4.3 Reputed Strengths and Weaknesses of Discrimination Tests If the batch-to-batch variation within a sample formulation is as large as the variation between formulations

4.4 Data Analyses

then the sensory specialist should not use triangle or duo-trio tests (Gacula and Singh, 1984). In this case the paired comparison difference test could be used but the first question that the sensory specialist should ask is whether the batch-to-batch variation should not be studied and improved prior to any study of new or different formulations. The major weakness of all discrimination tests is that they do not indicate the magnitude of the sensory difference(s) between the sample formulations. As the simple discrimination tests are aimed at a yes/no decision about the existence of a sensory difference, they are not designed to give information on the magnitude of a sensory difference, only whether one is likely to be perceived or not. The sensory specialist should not be tempted to conclude that a difference is large or small based on the significance level or the probability (pvalue) from the statistical analysis. The significance and p-value depend in part upon the number of panelists in the test as well as the inherent difficulty of the particular type of discrimination test method. So these are no acceptable indices of the size of the perceivable difference. However, it is sensible that a comparison in which 95% of the judges answered correctly has a larger sensory difference between control and test samples than a comparison in which performance was only at 50% correct. This kind of reasoning works only if a sufficient number of judges were tested, the methods were the same, and all test conditions were constant. Methods for interval level scaling of sensory differences based on proportions of correct discriminations in forced choice tests are discussed further in Chapter 5 as Thurstonian scaling methods. These methods are indirect measures of small differences. They are also methodologically and mathematically complex and require certain assumptions to be met in order to be used effectively. Therefore we feel that the sensory specialist is wiser to base conclusions about the degree of difference between samples on scaled (direct) comparisons, rather than indirect estimates from choice performance in discrimination tests. However, there are alternative opinions in the sensory community and we suggest that interested parties read Lee and O’Mahony (2007). With the exception of the 2-AFC and 3-AFC tests the other discrimination tests also do not indicate the nature of the sensory difference between the samples. The major strength of the discrimination tests is that the task that the panelists perform is quite simple and

89

intuitively grasped by the panelists. However, it is frequently the very simplicity of these tests that lead to the generation of garbage data. Sensory specialists must be very aware of the power, replication, and counterbalancing issues associated with discrimination tests. These issues are discussed later in this chapter.

4.4 Data Analyses The data from discrimination tests may be analyzed by any of the following statistical methods. The three data analyses are based on the binomial, chi-square, or normal distributions, respectively. All these analyses assume that the panelists were forced to make a choice. Thus they had to choose one sample or another and could not say that they did not know the answer. In other words, each panelist either made a correct or incorrect decision, but they all made a decision.

4.4.1 Binomial Distributions and Tables The binomial distribution allows the sensory specialist to determine whether the result of the study was due to chance alone or whether the panelists actually perceived a difference between the samples. The following formula allows the sensory scientists to calculate the probability of success (of making a correct decision; p) or the probability of failure (of making an incorrect decision; q) using the following formula. P(y) =

n! py pn−y y!(n − y)!

(4.6)

where n = total number of judgments y = total number of correct judgments p = probability of making the correct judgment by chance In this formula, n! describes the mathematical factorial function which is calculated as n×(n-1)× (n-2). . .×2×1. Before the widespad availability of calculators and computers, calculation of the binomial formula was quite complicated, and even now

90

4

it remains somewhat tedious. Roessler et al. (1978) published a series of tables that use the binomial formula to calculate the number of correct judgments and their probability of occurrence. These tables make it very easy to determine if a statistical difference were detected between two samples in discrimination tests. However, the sensory scientist may not have these tables easily available thus he/she should also know how to analyze discrimination data using statistical tables that are more readily available. We abridged the tables from Roessler et al. (1978) into Table 4.3. Using this table is very simple. For example, in a duo-trio test using 45 panelists, 21 panelists correctly matched the sample to the reference. In Table 4.3, in the section for duo-trio tests, we find that the table value for 45 panelists at 5% probability is 29. This value is larger than 21 and therefore the panelists could not detect a difference between the samples. In a different study, using a triangle test, 21 of 45 panelists correctly identified the odd sample. In Table 4.3, in the section for triangle tests, we find that the table value for 45 panelists at 5% probability is 21. This value is equal to 21 and therefore the panelists could detect a significant difference between the samples at the alpha probability of 5%.

Discrimination Testing

O2 = observed number incorrect choices E1 = expected number of correct choices E1 is equal to total number of observations (n) times probability (p) of a correct choice, by chance alone in a single judgment where p = 0.100 for the two-out-of-five test p = 0.500 for duo-trio, paired difference, paired directional, alternate A-not-A tests p = 0.333 for triangle tests E2 = expected number of incorrect choices E2 is equal to total number of observations (n) times probability (q) of an incorrect choice, by chance alone in a single judgment where q = 1−p q = 0.900 for the two-out-of-five test q = 0.500 for duo-trio, paired difference, paired directional, alternate A-not-A tests, ABX tests q = 0.667 for triangle tests The use of discrimination tests allows the sensory scientist to determine whether two products are statistically perceived to be different, therefore the degrees of freedom equal one (1). Therefore, a χ 2 table using df = 1 should be consulted, for alpha (α) at 5% the critical χ2 value is 3.84. For other alpha levels consult the chi-square table in the Appendix.

4.4.2 The Adjusted Chi-Square (χ2 ) Test

4.4.3 The Normal Distribution and the Z-Test on Proportion The chi-square distribution allows the sensory scientist to compare a set of observed frequencies with a matching set of expected (hypothesized) frequencies. The chi-square statistic can be calculated from the following formula (Amerine and Roessler, 1983), which includes the number -0.5 as a continuity correction. The continuity correction is needed because the χ 2 distribution is continuous and the observed frequencies from discrimination tests are integers. It is not possible for one-half of a person to get the right answer and so the statistical approximation can be off by as much as 1/2, maximally. χ = 2

The sensory specialist can also use the areas under the normal probability curve to estimate the probability of chance in the results of discrimination tests. The tables associated with the normal curve specify areas under the curve (probabilities) associated with specified values of the normal deviate (z). The following two formulae (Eqs. (4.8) and (4.9)) can be used to calculate the z-value associated with the results of a specific discrimination test (Stone and Sidel, 1978):

z=

+ 1.645(1.5) (X/N)(1−X/N) N

(5.7) Here is a worked example. Suppose we do a triangle test with 60 panelists, and 30 get the correct answer. We can ask the following questions: What is the best estimate of the number of discriminators? The proportion of discriminators? What is the upper 95% confidence interval on the number of discriminators? What is the confidence interval on the proportion of discriminators? Finally, could we conclude that there is significant similarity, based on a maximum allowable proportion of discriminators of 50%? The solution is as follows: Let X = number correct, D = number of discriminators, so X = 30 and N = 60. We have 1.5(30)-0.5(60) = D or 15 discriminators, or 25% of our judges detecting the difference, as our best estimate.

110

5 Similarity, Equivalence Testing, and Discrimination Theory

The standard error is given by

1.5

(30/60)

(6.4)

variation was used by Stevens et al. (1988) in a landmark paper on the inpidual variability in olfactory thresholds. In this case, five correct pairs were required to score the concentration as correctly detected, and this performance was confirmed at the next highest concentration level. The most striking finding of this study was that among the three inpiduals tested 20 times, their inpidual thresholds for butanol, pyridine, and phenylethylmethylethyl carbinol (a rose odorant) varied over 2,000- to 10,000-fold in concentration. Variation within an inpidual was as wide as the variation typically seen across a population of test subjects. This surprising result suggests that day-to-day variation in olfactory sensitivity is large and that thresholds for an inpidual are not very stable (for an example, see Lawless et al., 1995). More recent work using extensive testing of inpiduals at each concentration step suggests that these estimates of variability may be high. Walker et al. (2003) used a simple yes/no procedure (like the A, not-A test, or signal detection test) with 15 trials of targets and 15 trials of blanks at each concentration level. Using a model for statistical significant differences between blank and target trials, they were able to get sharp gradients for the inpidual threshold estimates. In summary, an ascending forced-choice method is a reasonably useful compromise between the need to pcisely define a threshold level and the problems encountered in sensory adaptation and observer fatigue when extensive measurements are made. However, the user of an ascending forced-choice procedure should be aware of the procedural choices that can affect the obtained threshold value. The following choices will affect the measured value: the number of alternatives (both targets and blanks), the stopping rule, or the number of correct steps in a row required to establish a threshold, the number of replicated correct trials required at any one step, and the rule to determine at what level of concentration steps the threshold value is assigned. For example, the inpidual threshold might be assigned at the lowest level correct, the geometric mean between the lowest level correct and highest level incorrect. Other specific factors include the chosen step size of concentration units (factors of two or three are common in taste and smell), the method of averaging or combining replicated ascending runs on the same inpidual and finally the method of averaging or combining group data. Geometric means are commonly used for the last two purposes.

136

6 Measurement of Sensory Thresholds

6.7 Probit Analysis

7.0

97.5 95 (90)

90 (80)

(85)

6.0

Cumulative Percentage

80 70

(57)

60 5.0

50

(40)

40

PROBIT

It is often useful to apply some kind of transformation or graphing method to the group data to linearize the curve used to find the 50% point in a group. Both the psychometric curve that repsents the behavior of an inpidual in multiple trials of a threshold test and the cumulative distribution of a group will resemble an S-shaped function similar to the cumulative normal distribution. A number of methods for graphing such data are shown in the ASTM standard E-1432 (ASTM, 2008b). One simple way to graph the data is simply to plot the cumulative percentages on “probability paper.” This p-printed solution provides a graph in which equal standard deviations are marked off along the ordinate, effectively stretching the percentile intervals at the ends and compssing them in the midrange to conform to the density of the normal distribution. Another way to achieve the straightening of the S-shaped response curve is to transform the data by taking z-scores. Statistical packages for data analysis often provide options for transformation of the data. A related method was once widely used in threshold measurement, called Probit analysis (ASTM, 2008b; Dravnieks and Prokop, 1975; Finney, 1971). In this approach, the inpidual points are transformed relative to the mean value, pided by the standard deviation and then a constant value of +5 is added to translate all the numbers to positive values for convenience. A linear fitted function can now be interpolated at the value of 5 to estimate the threshold as in Fig. 6.6. The conversion (to a z-score +5) tends to make an S-shaped curve more linear. An example of this can be found in the paper by Brown et al. (1978), using data from a multiple paired test. First the percent correct is adjusted for chance. Then the data are transformed from the percent correct (across the group) at each concentration level by conversion to z-scores and a constant of 5 is added. The mean value or Probit equal to 5 can be found by interpolation or curve fitting. An example of this technique for estimating threshold from a group of 20 panelists is shown in Meilgaard et al. (1991) and in ASTM (2008b). Probit plots can be used for any cumulative proportions, as well as ranked data and analysis of inpiduals who are more extensively tested than in the 3-AFC method example shown earlier.

30 (18)

20 10

4.0 (6)

5 2.5

3.0 1

2 4 8 16 32 64 Threshold Concentration (log scale)

Fig. 6.6 An example of Probit analysis. Numbers in parentheses are the cumulative percentages of panelists reaching threshold at each concentration step. Note the uneven scale on the left axis. Probits mark off equal standard deviations and are based on the z-score for any proportion plus the constant, 5. Interpolation at the 50% or Probit 5.0 gives the threshold value.

6.8 Sensory Adaptation, Sequential Effects, and Variability Inpidual variability, both among a group of people and within an inpidual over repeated measurements psents a challenge to the idea that the threshold is anything like a fixed value. For example, stable olfactory thresholds of an inpidual are difficult to measure. The test-retest correlation for inpidual’s olfactory threshold is often low (Punter, 1983). Even within an inpidual, threshold values will generally decrease with practice (Engen, 1960; Mojet et al., 2001; Rabin and Cain, 1986), and superimposed upon this practice effect is a high level of seemingly random variation (Stevens et al., 1988). Inpiduals may become sensitive to odorants to which they were formerly anosmic, apparently through simple exposure (Wysocki et al., 1989). Increased sensitivity as a function of exposure may be a common phenomenon among women of childbearing age (Dalton et al., 2002; Diamond et al., 2005). Sensory adaptation and momentary changes in sensitivity due to sequences may have occurred in

6.9 Alternative Methods: Rated Difference, Adaptive Procedures, Scaling

the experiments of Stevens et al. (1988) and could have contributed to some instability in the measurements. As pdicted by sequential sensitivity analysis (Masuoka et al., 1995; O’Mahony and Odbert, 1985) the specific stimulus sequence will render discrimination more or less difficult. After the stronger of two stimuli, an additional second strong stimulus psented next may be partially adapted and seem weaker than normal. Stevens et al. remarked that sometimes subjects would get all five pairs correct at one level with some certainty that they “got the scent” but lost the signal at the next level before getting it back. This report and the reversals of performance in threshold data are consistent with adaptation effects temporarily lessening sensitivity. The sensory impssion will sometimes “fade in and out” at levels near threshold. In attempts to avoid adaptation effects, other researchers have gone to fewer psentations of the target stimulus. For example, Lawless et al. (1995) used one target among three blank stimuli, a 4-AFC test that has appeared in pvious studies (e.g., Engen, 1960; Punter, 1983). This lowers the chance performance level and lessens the potential adaptation at any one concentration step. To guard against the effects of correct guessing, threshold was taken as the lowest concentration step with a correct choice when all higher concentrations were also correct. Thresholds were measured in duplicate ascending runs in a test session, and a duplicate session of two more ascending runs was run on a second day. Correlations across the four ascending series ranged from 0.75 to 0.92 for cineole and from 0.51 to 0.92 for carvone. For carvone, thresholds were better duplicated within a day (r = 0.91 and 0.88) than across days (r from 0.51 to 0.70). This latter result suggests some drift over time in odor thresholds, in keeping with the variability seen by Stevens et al. (1988). However, results with this ascending method may not be this reliable for all compounds. Using the ascending 4-AFC test and a sophisticated olfactometer, Punter (1983) found median retest correlations for 11 compounds to be only 0.40. The sense of taste may fare somewhat better. In a study of electrogustometric thresholds with ascending paired tests requiring five correct responses, retest correlations for an elderly population were 0.95 (Murphy et al., 1995). In many forced-choice studies, high variability in smell thresholds is also noted across the testing pool.

137

Brown et al. (1978) stated that for any test compound, a number of insensitive inpiduals would likely be seen in the data set, when 25 or more persons were tested to determine an average threshold. Among any given pool of participants, a few people with otherwise normal smell acuity will have high thresholds. This is potentially important for sensory professionals who need to screen panelists for detection of specific flavor or odor notes such as defects or taints. In an extensive survey of thresholds for branched-chain fatty acids, Brennand et al. (1989) remarked that “some judges were unable to identify the correct samples in the pairs even in the highest concentrations provided” and that ” panelists who were sensitive to most fatty acids found some acids difficulty to perceive” (p. 109). Wide variation in sensitivity was also observed to the common flavor compound, diacetyl, a buttery-smelling by-product of lactic bacteria fermentation (Lawless et al., 1994). Also, simple exposure to some chemicals can modify specific anosmia and increase sensitivity (Stevens and O’Connell, 1995).

6.9 Alternative Methods: Rated Difference, Adaptive Procedures, Scaling 6.9.1 Rated Difference from Control Another practical procedure for estimating threshold has involved the use of ratings on degree-of-difference scales, where a sample containing the to-be-recognized stimulus is compared to some control or blank stimulus (Brown et al., 1978; Lundahl et al., 1986). Rated difference may use a line scale or a category scale, ranging from no difference or “exact same” to a large difference, as discussed in Chapter 4. In these procedures ratings for the sensory difference from the control sample will increase as the intensity of the target gets stronger. A point on the plot of ratings versus concentration is assigned as threshold. In some variations on this method, a blind control sample is also rated. This provides the opportunity to estimate a baseline or false alarm rate based on the ratings (often nonzero) of the control against itself. Identical samples will often get nonzero difference estimates due to the moment-to-moment variability in sensations.

138

of the inpidual thresholds, due to the larger number of observations, another oddity of using statistical significance to determine the threshold. Instead of using statistical significance as a criterion, Marin et al. determined the point of maximum curvature on the dose-response curve as the threshold. Such an approach makes sense from consideration of the general form of the dose-response (psychophysical) curve for most tastes and odors. Figure 6.7 shows a semi-log plot for the Beidler taste equation, a widely applied dose-response relationship in studies of the chemical senses (see Chapter 2). This function has two sections of curvature (change in slope, i.e., acceleration) when plotted as a function of log concentration. There is a point at which the response appears to be slowly increasing out from the background noise and then rises steeply to enter the middle of the dynamic range of response. The point of maximum curvature can be estimated graphically or determined from curve fitting and finding the maximum rate of change (i.e., maximum of the second derivative) (Marin et al., 1991).

Psychophysical function (semi-hyperbolic) 100

Response (percent of maximum)

In one application of this technique for taste and smell thresholds, a substance was added in various levels to estimate the threshold in a food or beverage. In each inpidual trial, three samples would be compared to the control sample with no added flavor-two adjacent concentration steps of the target compound and one blind control sample (Lundahl et al., 1986). Samples were rated on a simple 9-point scale, from zero (no difference) to eight (extreme difference). This provided a comparison of the control to itself and a cost-effective way of making three comparisons in one set of samples. Since sample concentrations within the three rated test samples were randomized, the procedure was not a true ascending series and was dubbed the “semi-ascending paired difference method.” How is the threshold defined in these procedures? One approach is to compare the difference ratings for a given level with the difference ratings given to the control sample. Then the threshold can be based on some measure of when these difference ratings perge, such as when they become significantly different by a t-test (see Brown et al., 1978). Another approach is simply to subtract the difference score given to the blind control from the difference score give to each test sample and treat these adjusted scores as a new data set. In the original paper of Lundahl et al. (1986), this latter method was used. In the analysis, they performed a series of ttests. Two values were taken to bracket the range of the threshold. The upper level was the first level yielding a significant t-test versus zero, and the lower level was the nearest lower concentration yielding a significant ttest versus the first. This provided an interval in which the threshold (as defined by this method) lies between the two bracketing concentrations. One problem with this approach is that when the threshold is based on the statistical significance of t-statistics (or any such significance test), the value of threshold will depend upon the number of observations in the test. This creates a nonsensical situation where the threshold value will decrease as a function of the number of panelists used in the test. This is an irrelevant variable, a choice of the experimenter, and has nothing to do with the physiological sensitivity of the panelist or the biological potency of the substance being tested, a problem recognized by Brown et al. (1978) and later by Marin et al. (1991). Marin et al. also pointed out that a group threshold, based on a larger number of observations than an inpidual threshold, would be lower than the mean

6 Measurement of Sensory Thresholds

80

60

R = (Rmax C) / (K + C) K = 1000, Rmax = 100

40

20

zone of inflection = threshold range

0 0

2

4 log Concentration

6

8

Fig. 6.7 Beidler curve.

6.9.2 Adaptive Procedures Popular methods for threshold measurement for visual and auditory stimuli have been procedures in which the next stimulus intensity level to be tested depends

6.9 Alternative Methods: Rated Difference, Adaptive Procedures, Scaling

SAMPLE UDTR RECORD CONCENTRATION STEP

CONCENTRATION STEP

the part of the respondent. Psychophysical researchers have found ways to undo this sequential dependence to counteract observer expectancies. One example is the double random staircase procedure (Cornsweet, 1962) in which trials from two staircase sequences are randomly intermixed. One staircase starts above the threshold and descends, while the other starts below the threshold and ascends. On any given trial, the observer is unaware which of the two sequences the stimulus is chosen from. As in the simple staircase procedure, the level chosen for a trial depends upon detection or discrimination in the pvious trial, but of that particular sequence. Further refinements of the procedure involve the introduction of forced-choice (Jesteadt, 1980) to eliminate response bias factors involved in simple yes/no detection. Another modification to the adaptive methods has been to adjust the ascending and descending rules so that some number of correct or incorrect judgments is required before changing intensity levels, rather than the one trial as in the simple staircase (Jesteadt, 1980). An example is the “up down transformed response” rule or UDTR (Wetherill and Levitt, 1965). Wetherill and Levitt gave an example where two positive judgments were required before moving down, and only one negative judgment at a given level before moving up. An example is shown in Fig. 6.9. Rather than estimating the 50% point on a traditional psychometric function, this more stringent requirement now tends to converge on the 71% mark as an average of the peaks and valleys in the series. A forced choice can be added to an adaptive procedure. Sometimes

SAMPLE STAIRCASE RECORD

64 32 16 8 4

R 2 R R 2

3

4

R

R

1

1

5

139

6

7

R 8

9

10

TRIAL

Fig. 6.8 Staircase example.

TRIAL

Fig. 6.9 Staircase example.

140

6.9.3 Scaling as an Alternative Measure of Sensitivity Threshold measurements are not the only way to screen inpiduals for insensitivity to specific compounds like PTC or to screen for specific anosmia. Do thresholds bear any relation to suprathreshold responding? While it has been widely held that there is no necessary relationship between threshold sensitivity and suprathreshold responding (Frijters, 1978; Pangborn, 1981), this assertion somewhat overstates the case. Counter-examples of good correlations can be seen in tests involving compounds like PTC where there are insensitive groups. For example, there is a -0.8 correlation between simple category taste intensity ratings for PTC and the threshold, when the rated concentration is near the antimode or center between the modes of a bimodal threshold frequency distribution (Lawless, 1980). Thus ratings of a carefully chosen level can be used for a rapid screening method for PTC taster status (e.g., Mela, 1989). Similar results have been noted for smell. Berglund and Högman (1992) reported better reliability of suprathreshold ratings than threshold determinations in screening for olfactory sensitivity. Stevens and O’Connell (1991) used category ratings of perceived intensity as well as qualitative descriptors as a screening tool before threshold testing for specific anosmia. Threshold versus rating correlations were in the range of -0.6 for cineole, -0.3 for carvone, and -0.5 for diacetyl (Lawless et al., 1994, 1995). The correlations were obtained after subtraction of ratings to a blank stimulus, in order to correct for differences in scale usage. Thus there is a moderate negative correlation of sensitivity and rated intensity when one

6 Measurement of Sensory Thresholds

examines the data across a highly variable group as is the case with specific anosmia or tasting PTC bitterness. The correlation is negative since higher thresholds indicate lower sensitivity and thus lower rated intensity.

6.10 Dilution to Threshold Measures 6.10.1 Odor Units and Gas-Chromatography Olfactometry (GCO) In this section, several applied methods will be described that make use of the threshold concept in trying to determine the sensory impact of various flavors and odor materials. The first group of methods concerns the olfactory potency of volatile aroma compounds as they are found in foods or food components. The issue here becomes one of not just threshold determination, but determination of both the threshold and the actual concentration psent in a food sample. The ratio of these concentrations (actual concentration to threshold concentration) can help indicate whether or not a given flavor substance is likely to contribute to the overall sensory impssion in a food. These ratios are commonly called “odor units.” The second much older method is similar in logic and was developed to determine the point at which the irritative or heat sensations from pepper compounds would be first detectable when diluted to a given extent, the Scoville procedure. Both of these techniques then use dilution-to-threshold as a measure of sensory impact. When a complex natural product like a fruit extract is analyzed for its chemical components, hundreds or even thousands of chemicals may be identified, many of which have odor properties. The number of potential flavor compounds identified in any product seems only to be limited by the resolution and sensitivity of the current methods in analytical chemistry. These methods are always improving leading to longer and longer lists of possible contributing flavor materials (Piggott, 1990). Flavor scientists need to find a way to narrow the list or to separate those compounds which are most likely contributing to the overall flavor from those compounds that are psent in

6.10 Dilution to Threshold Measures

such low concentrations that they are probably not important. Obviously, a sensory-based method is needed in conjunction with the analytical chemistry to provide a bioassay for possible sensory impact (Acree, 1993). Thresholds can be useful in addressing this kind of problem. The reasoning goes that only those compounds that are psent in the product in concentrations above their threshold are likely to be contributors to the flavor of the product. There are a number of potential flaws in this thinking discussed below, but for now let us see how this can be put to use. Given a concentration C psent in a natural product, a dimensionless quantity can be derived by piding that concentration by the threshold concentration Ct , and the ratio C/Ct defines the number of odor units (or flavor units) for compounds assessed by smell. According to this logic, only those compounds with odor units greater than one will contribute to the aroma of the product. This reasoning is extended sometimes to include the idea that the greater the number of odor units, the greater the potential contribution. However, it is now widely recognized that the odor unit is a concentration multiple and not a measure of subjective magnitude. Only direct scaling methods can assess the actual magnitude of sensation above threshold and the psychophysical relationship between concentration and odor intensity (Frijters, 1978). Furthermore, this idea ignores the possibility of subthreshold additivity or synergy (Day et al., 1963). A closely related group of chemical compounds might all be psent below their inpidual thresholds, but together could stimulate common receptors so as to produce an above-threshold sensation. Such additivity is not pdicted by the odor unit approach and such a group of compounds could be missed in dilution analysis. Nonetheless, thresholds provide at least one isointense reference point on the dose response curve, so they have some utility as a measure of potency used to compare different odor compounds. In analyzing a food, one could look up literature values for all the identified compounds in the product in one of the published compendia of thresholds (e.g., ASTM, 1978; van Gemert, 2003). If the concentration in the product is determined, then the odor unit value can be calculated by simply piding by threshold. However, it is important to remember that the literature values for thresholds depend upon the method and the medium

141

of testing. Unless the same techniques are used and the same medium was used as the carrier (rarely the case) the values may not be necessarily comparable for different compounds. A second approach is to actually measure the dilutions necessary to reach threshold for each compound, starting with the product itself. This necessitates the use of a separatory procedure, so that each compound may be inpidually perceived. The combination of gas chromatography with odor port sniffing of a dilution series is a popular technique (Acree, 1993). Various catchy names have been applied to such techniques in the flavor literature, including Aroma Extract Dilution Analysis (for examples, see Guth and Grosch, 1994; Milo and Grosch, 1993; Schieberle and Grosch, 1988), CHARM analysis (Acree et al., 1984) or more generically, gas chromatography olfactometry or GCO. The basis of these techniques is to have subjects respond when an odor is perceived when sniffing the exit port during a GC run. In recent years, the effluent has been embedded in a cooled, humidified air stream to improve the comfort of the observer and to increase sensory resolution of the eluting compounds. Over several dilutions, the response will eventually drop out, and the index of smell potency is related to the reciprocal of the dilution factor. The sniffer’s responses occur on a time base that can be cross-referenced to a retention index and then the identity of the compound can be determined by a combination of retention index, mass spectrometry, and aroma character. In practice, these techniques considerably shorten the list of potential aroma compounds contributing to flavor in a natural product (e.g., Cunningham et al., 1986). The method has also been used as an assessment technique for measuring the sensitivity of human panelists, as opposed to being a tool to determine the sensory impact of flavor compounds (Marin et al., 1988). In this approach, the gas chromatograph once again serves as an olfactometer. Known compounds can be psented as a dilution series of mixtures. Variation in inpidual thresholds can readily be assessed for a variety of compounds since they can be combined in a GC run as long as they have different retention times, i.e., do not co-elute on the column of choice. The potential use of GCO for screening panelists, for assessing odor responses in general, and for assessing specific anosmias has also been attempted (Friedrich and Acree, 2000; Kittel and Acree, 2008).

142

6.10.2 Scoville Units Another example of a dilution method is the traditional Scoville procedure for scoring pepper heat in the spice trade. This procedure was named for W. Scoville who worked in the pharmaceutical industry in the early twentieth century. He was interested in the topical application of spice compounds like pepper extracts as counterirritants, and he needed to establish units that could be used to measure their potency. His procedure consisted of finding the number of dilutions necessary for sensations to disappear and then using this number of dilutions as an estimate of potency. In other words, potency was defined as a reciprocal threshold. A variation of this procedure was adopted by the Essential Oil Association, British Standards Institution, International Standards Organization, American Spice Trade Association (ASTA), and adopted as an Indian standard method (for a review, see Govindarajan, 1987). The procedure defined units of pungency as the highest dilution at which a definite “bite” would be perceived and thus contains instructions consistent with a recognition threshold. Scoville units were dilution factors, now commonly given as mL/g. ASTA Method 21 (ASTA, 1968) is widely used and contains some modifications in an attempt to overcome problems with the original Scoville procedure. In brief, the method proceeds as follows: Panelists are screened for acuity relative to experienced persons. Dilution schedules are provided which simplify calculations of the eventual potency. Solutions are tested in 5% sucrose and negligible amounts of alcohol. Five panelists participate and concentrations are given in ascending order around the estimated threshold. Threshold is defined as the concentration at which three out of five judges respond positively. This method is difficult in practice and a number of additional variations have been tried to improve on the accuracy and pcision of the method (Govindarajan, 1987). Examples include the following: (1) substitution of other rules for 3/5, e.g., mean + SD of 20-30 judgments, (2) use of a triangle test, rather than simple yes/no at each concentration, (3) requiring recognition of pungency (Todd et al., 1977), (4) reduction of sucrose concentration in the carrier solution to 3%, and (5) use of a rating scale from 1 (definitely not detectable) to 6 (definitely detectable). This latter

6 Measurement of Sensory Thresholds

modification defined a detection threshold at mean scale value of 3.5. Almost all these methods specify mandatory rest periods between samples due to the long lasting nature of these sensations. The measurements are still difficult. One problem is that capsaicin, the active heat principle in red pepper, is prone to desensitize observers within a session and also regular consumers of hot spices also become less sensitive, leading to wide inpidual differences in sensitivity among panelists (Green, 1989; Lawless et al., 1985). Alternative procedures have been developed based on rating scales with fixed physical references (Gillette et al., 1984) and these have been endorsed by ASTM (2008c) as standard test methods. These rating scale procedures show good correlations with instrumental measures of capsaicin content in pepper samples and can be cross-referenced to Scoville units for those who pfer to do business in the traditional units (Gillette et al., 1984).

6.11 Conclusions Threshold measurements find three common uses in sensory analysis and flavor research. First, they can be used to compare the sensitivities of different panelists. Second, they can be used as an index of the biological potency of a flavor compound. Third, they can provide useful information regarding the maximum tolerable levels of an off-flavor or taint. A variety of different techniques have been used to find thresholds or have employed the threshold concept in practical flavor work. Examples of different threshold methods are given in Table 6.4. In spite of their practical applications, the usefulness of threshold measures is often questioned in sensory evaluation. One criticism is that thresholds are only one point on an intensity function and thus they do not tell us anything about abovethreshold responding. There are some good examples in which thresholds do not pdict or do not correlate very well with suprathreshold responses. For example, patients irradiated for cancer may lose their sense of taste temporarily and thresholds return to normal long before suprathreshold responsiveness is recovered (Bartoshuk, 1987). However, as we have seen both in the case of PTC tasting and in specific anosmias, insensitive inpiduals (as determined by their threshold) will also tend to be less responsive above

Appendix

143

Table 6.4 Threshold procedures Method Citations/examples

Response

Threshold

Ascending forced choice Ascending forced choice

ASTM E-679-79 Stevens et al. (1988)

3-AFC 2-AFC, 5 replicates

Semi-ascending paired difference

Lundahl et al. (1986) Rated difference from control

Adaptive, up-down transformed response rule

Reed et al. (1995)

2-AFC

Double Random Staircase CHARM analysis

Cornsweet (1962) Acree et al. (1986)

Yes/No Yes/No

Geometric mean of inpidual threshold Lowest correct set with confirmation at next concentration t-test for significant difference from zero with all blank trials (blind control scores) subtracted Average of responses ascending after one incorrect, descending after two correct at each level Average of reversal points Nonresponse on descending concentration runs (implied)

threshold. These correlations are strong when comparing inpiduals of very different sensitivities, but the threshold-suprathreshold parallel may not extend to all flavor compounds. A more complete understanding of the whole dynamic range of dose-response, as found in scaling studies, would be more informative. Other shortcomings need to be kept in mind by sensory workers who would use threshold measurements for making product decisions. First, thresholds are statistical constructs only. Signal detection theory warns us that the signal and noise perge in a continuous fashion and discontinuities in perception may be an idealized construction that is comfortable but unrealistic. There is no sudden transition from nondetection to 100% detection. Any modern concept of threshold must take into account that a range of values, rather than a single point is involved in the specification. Thresholds depend upon the conditions of measurement. For example, as the purity of the diluent increases, the threshold for taste will go down. So a measure of the true absolute threshold for taste (if such a thing existed) would require water of infinite purity. The threshold exists not in this abstract sense, but only

as a potentially useful construct of our methods and informational requirements. Finally, because of the problems mentioned above, sensory professionals need to keep the following principles firmly in mind when working with threshold procedures: First, changes in method will change the obtained values. Literature values cannot be trusted to extend to a new product or a new medium or if changes are made in test procedure. Next, threshold distributions do not always follow a normal bell curve. There are often high outliers and possibly cases of insensitivity due to inherited deficits like specific anosmia (Amoore, 1971; Brown et al., 1978; Lawless et al., 1994). Threshold values for an inpidual are prone to high variability and low test-retest reliability. An inpidual’s threshold measure on a given day is not necessarily a stable characteristic of that person (Lawless et al., 1995; Stevens et al., 1988). Practice effects can be profound, and thresholds may stabilize over a period of time (Engen, 1960; Rabin and Cain, 1986). However, group averaged thresholds are reliable (Brown et al., 1978; Punter, 1983) and provide a useful index of the biological activity of a stimulus.

Appendix: MTBE Threshold Data for Worked Example

Panelist 1 2 3 4 5 6

Concentration (μg/L) 2 3.5 + o o o o + o o + o o o

6 + o o o + +

10 + + o o o +

18 + + o o + +

30 + + o o + +

60 + + + + o +

100 + + + + + +

BET 4.6 7.7 42 42 77 4.6

log(BET) 0.663 0.886 1.623 1.623 1.886 0.663

144

Panelist 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 Prop. Corr.

6 Measurement of Sensory Thresholds

Concentration (μg/L) 2 3.5 6 o + + + + o o o + o o o + o + o o o + + + + + + o + + o o + + o + o o + + + + + o + + + + + + + + + o + + + o + + o + + + o o o o + o + o o + + + o o + + + + + + o o o o o o o o o + + + + o + + + o o o + + + + o + o o o o o + + o + + + + + o + o o + o o o o o o + o o o + + + + + + + o o o o + o o o o o + 0.44 0.49 0.61

10 o + o o + o + + o o + + + + + + o + + + o + o o + + o o + + + + o o + + o o o + o + + + + + + o + o + 0.58

18 + + + o + + + + + o + + + o + + + + o o + + + + o o + o o o + + + + + + o + + + o o o + + + + o + o o 0.65

30 + + + o + o + + + o + + + + + + + + + + o + + + o o o o + + + + + + + + o o + + + + + + + + + + + + + 0.77

60 + + + o + + + + + + + + + o + + + + + + o + + + + o + + + + + + + + + + + + + o + + + + + + + + + + o 0.89

100 + + + + + o + + + o + + + o + + + + + + + o + + o + + o o + + + + + + + + + + + o + + + + + + + + + + 0.86

BET 13 7.7 13 77 4.6 132 1.4 1.4 13 132 4.6 4.6 1.4 132 1.4 1.4 13 1.4 23 23 77 132 13 13 132 77 42 132 132 23 1.4 1.4 13 13 1.4 7.7 42 42 13 77 132 23 23 4.6 7.7 1.4 1.4 23 4.6 23 77 Mean (log(BET)) 10ˆ1.154=

log(BET) 1.114 0.886 1.114 1.886 0.663 2.121 0.146 0.146 1.114 2.121 0.663 0.663 0.146 2.121 0.146 0.146 1.114 0.146 1.362 1.362 1.886 2.121 1.114 1.114 2.121 1.886 1.623 2.121 2.121 1.362 0.146 0.146 1.114 1.114 0.146 0.886 1.623 1.623 1.114 1.886 2.121 1.362 1.362 0.663 0.886 0.146 0.146 1.362 0.663 1.362 1.886 1.154 14.24

References

References Acree, T. E. 1993. Bioassays for flavor. In: T. E. Acree and R. Teranishi (eds.), Flavor Science, Sensible Principles and Techniques. American Chemical Society Books, Washington, pp. 1-20. Acree, T. E., Barnard, J. and Cunningham, D. G. 1984. A procedure for the sensory analysis for gas chromatographic effluents. Food Chemistry, 14, 273-286. American Spice Trade Association 1968. Pungency of capsicum spices and oleoresins. American Spice Trade Association Official Analytical Methods, 21.0, 43-47. Amoore, J. E. 1971. Olfactory genetics and anosmia. In: L. M. Beidler (ed.), Handbook of Sensory Physiology. Springer, Berlin, pp. 245-256. Amoore, J. E. 1979. Directions for pparing aqueous solutions of primary odorants to diagnose eight types of specific anosmia. Chemical Senses and Flavor, 4, 153-161. Amoore, J. E.,Venstrom, D. and Davis, A. R. 1968. Measurement of specific anosmia. Perceptual and Motor Skills, 26, 143-164. Antinone, M. A., Lawless, H. T., Ledford, R. A. and Johnston, M. 1994. The importance of diacetyl as a flavor component in full fat cottage cheese. Journal of Food Science, 59, 38-42. ASTM. 1978. Compilation of Odor and Taste Threshold Values Data. American Society for Testing and Materials, Philadelphia. ASTM. 2008a. Standard practice for determining odor and taste thresholds by a forced-choice ascending concentration series method of limits, E-679-04. Annual Book of Standards, Vol. 15.08. ASTM International, Conshocken, PA, pp. 36-42. ASTM. 2008b. Standard practice for defining and calculating inpidual and group sensory thresholds from forcedchoice data sets of intermediate size, E-1432-04. ASTM International Book of Standards, Vol. 15.08. ASTM International, Conshocken, PA, pp.82-89. ASTM. 2008c. Standard test method for sensory evaluation of red pepper heat, E-1083-00. ASTM International Book of Standards, Vol. 15.08. ASTM International, Conshocken, PA, pp. 49-53. Ayya, N. and Lawless, H. T. 1992. Qualitative and quantitative evaluation of high-intensity sweeteners and sweetener mixtures. Chemical Senses, 17, 245-259. Bartoshuk, L. M. 1987. Psychophysics of taste. American Journal of Clinical Nutrition, 31, 1068-1077. Bartoshuk, L. M., Murphy, C. and Cleveland, C. T. 1978. Sweet taste of dilute NaCl: Psychophysical evidence for a sweet stimulus. Physiology and Behavior, 21, 609-613. Berglund, B. and Högman, L. 1992. Reliability and validity of odor measurements near the detection threshold. Chemical Senses, 17, 823-824. Blakeslee, A. F. 1932. Genetics of sensory thresholds: Taste for phenylthiocarbamide. Proceedings of the National Academy of Sciences, 18, 120-130. Brennand, C. P., Ha, J. K. and Lindsay, R. C. 1989. Aroma properties and thresholds of some branched chain and other minor volatile fatty acids occurring in milkfat and meat lipids. Journal of Sensory Studies, 4, 105-120.

145 Boring, E. G. 1942. Sensation and Perception in the History of Experimental Psychology. Appleton-CenturyCrofts, New York, pp. 41-42. Brown, D. G. W., Clapperton, J. F., Meilgaard, M. C. and Moll, M. 1978. Flavor thresholds of added substances. American Society of Brewing Chemists Journal, 36, 73-80. Bufe, B., Breslin, P. A. S., Kuhn, C., Reed, D., Tharp, C. D., Slack, J. P., Kim, U.-K., Drayna, D. and Meyerhof, W. 2005. The molecular basis of inpidual differences in phenylthiocarbamide and propylthiouracil bitterness perception. Current Biology, 15, 322-327. Cain, W. S. 1976. Olfaction and the common chemical sense: Some psychophysical contrasts. Sensory Processes, 1, 57-67. Cain, W. S. and Murphy, C. L. 1980. Interaction between chemoreceptive modalities of odor and irritation. Nature, 284, 255-257. Cornsweet, T. M. 1962. The staircase method in psychophysics. American Journal of Psychology, 75, 485-491. Cunningham, D. G., Acree, T. E., Barnard, J., Butts, R. M. and Braell, P. A. 1986. CHARM analysis of apple volatiles. Food Chemistry, 19, 137-147. Dale, M. S., Moylan, M. S., Koch, B. and Davis, M. K. 1997. Taste and odor threshold determinations using the flavor profile method. Proceedings of the American Water Works Association Water-Quality Technology Conference, Denver, CO. Dalton, P., Doolittle, N. and Breslin, P. A. S. 2002. Genderspecific induction of enhanced sensitivity to odors. Nature Neuroscience, 5, 199-200. Day, E. A., Lillard, D. A. and Montgomery, M. W. 1963. Autooxidation of milk chúng tôi Effect on flavor of the additive interactions of carbonyl compounds at subthreshold concentrations. Journal of Dairy Science, 46, 291-294. Diamond, J., Dalton, P., Doolittle, N. and Breslin, P. A. S. 2005. Gender-specific olfactory sensitization: Hormonal and cognitive influences. Chemical Senses, 30(suppl 1), i224-i225. Dravnieks, A. and Prokop, W. H. 1975. Source emission odor measurement by a dynamic forced-choice triangle olfactometer. Journal of the Air Pollution Control Association, 25, 28-35. Engen, T. E. 1960. Effects of practice and instruction on olfactory thresholds. Perceptual and Motor Skills, 10, 195-198. Finney, D. J. 1971. Probit Analysis, Third Edition. Cambridge University, London. Friedrich, J. E. and Acree, T. E. 2000. Design of a standard set of odorants to test inpiduals for specific anosmia. Frontiers in Flavour Science [Proc. 9th Weurman Flavour Res. Symp.], 230-234. Frijters, J. E. R. 1978. A critical analysis of the odour unit number and its use. Chemical Senses and Flavour, 3, 227-233. Gillette, M., Appel, C. E. and Lego, M. C. 1984. A new method for sensory evaluation of red pepper heat. Journal of Food Science, 49, 1028-1033. Green, B. G. 1989. Capsaicin sensitization and desensitization on the tongue produced by brief exposures to a low concentration. Neuroscience Letters, 107, 173-178. Govindarajan, V. S. 1987. Capsicum – production, technology, chemistry and quality. Part III. Chemistry of the color, aroma and pungency stimuli. CRC Critical Reviews in Food Science and Nutrition, 24, 245-311.

146 Guth, H. and Grosch, W. 1994. Identification of the character impact odorants of stewed beef juice by instrumental analysis and sensory studies. Journal of Agricultural and Food Chemistry, 42, 2862-2866. Harris, H. and Kalmus, H. 1949. The measurement of taste sensitivity to phenylthiourea (P.T.C.). Annals of Eugenics, 15, 24-31. Harvey, L. O. 1986. Efficient estimation of sensory thresholds. Behavior Research Methods, Instruments and Computers, 18, 623-632. James, W. 1913. Psychology. Henry Holt and Co., New York. Jesteadt, W. 1980. An adaptive procedure for subjective judgments. Perception and Psychophysics, 28(1), 85-88. Kittel, K. M and Acree, T. E. 2008. Investigation of olfactory deficits using gas-chromatography olfactometry. Manuscript submitted, available from the authors. Lawless, H. T. 1980. A comparison of different methods used to assess sensitivity to the taste of phenylthiocarbamide PTC. Chemical Senses, 5, 247-256. Lawless, H. T. 2010. A simple alternative analysis for threshold data determined by ascending forced-choice method of limits. Journal of Sensory Studies, 25, 332-346. Lawless, H. T., Antinone, M. J., Ledford, R. A. and Johnston, M. 1994. Olfactory responsiveness to diacetyl. Journal of Sensory Studies, 9(1), 47-56. Lawless, H. T., Rozin, P. and Shenker, J. 1985. Effects of oral capsaicin on gustatory, olfactory and irritant sensations and flavor identification in humans who regularly or rarely consumer chili pepper. Chemical Senses, 10, 579-589. Lawless, H. T., Thomas, C. J. C. and Johnston, M. 1995. Variation in odor thresholds for l-carvone and cineole and correlations with suprathreshold intensity ratings. Chemical Senses, 20, 9-17. Linschoten, M. R., Harvey, L. O., Eller, P. A. and Jafek, B. W. 1996. Rapid and accurate measurement of taste and smell thresholds using an adaptive maximum-likelihood staircase procedure. Chemical Senses, 21, 633-634. Lundahl, D. S., Lukes, B. K., McDaniel, M. R. and Henderson, L. A. 1986. A semi-ascending paired difference method for determining the threshold of added substances to background media. Journal of Sensory Studies, 1, 291-306. Masuoka, S., Hatjopolous, D. and O’Mahony, M. 1995. Beer bitterness detection: Testing Thurstonian and sequential sensitivity analysis models for triad and tetrad methods. Journal of Sensory Studies, 10, 295-306. Marin, A. B., Acree, T. E. and Barnard, J. 1988. Variation in odor detection thresholds determined by charm analysis. Chemical Senses, 13, 435-444. Marin, A. B., Barnard, J., Darlington, R. B. and Acree, T. E. 1991. Sensory thresholds: Estimation from dose-response curves. Journal of Sensory Studies, 6(4), 205-225. McBurney, D. H. and Collings, V. B. 1977. Introduction to Sensation and Perception. Prentice-Hall, Englewood Cliffs, NJ. Meilgaard, M., Civille, G. V. and Carr, B. T. 1991. Sensory Evaluation Techniques, Second Edition. CRC Press, Boca Raton, FL. Mela, D. 1989. Bitter taste intensity: The effect of tastant and thiourea taster status. Chemical Senses, 14, 131-135. Milo, C. and Grosch, W. 1993. Changes in the odorants of boiled trout (Salmo fario) as affected by the storage of the raw

6 Measurement of Sensory Thresholds material. Journal of Agricultural and Food Chemistry, 41, 2076-2081. Mojet, J., Christ-Hazelhof, E. and Heidema, J. 2001. Taste perception with age: Generic or specific losses in threshold sensitivity to the five basic tastes. Chemical Senses 26, 854-860. Morrison, D. G. 1978. A probability model for forced binary choices. American Statistician, 32, 23-25. Murphy, C., Quiñonez, C. and Nordin, S. 1995. Reliability and validity of electrogustometry and its application to young and elderly persons. Chemical Senses, 20(5), 499-515. O’Mahony, M. and Ishii, R. 1986. Umami taste concept: Implications for the dogma of four basic tastes. In: Y. Kawamura and M. R. Kare (eds.), Umami: A Basic Taste. Marcel Dekker, New York, pp. 75-93.. O’Mahony, M. and Odbert, N. 1985. A comparison of sensory difference testing procedures: Sequential sensitivity analysis and aspects of taste adaptation. Journal of Food Science, 50, 1055-1060. Pangborn, R. M. 1981. A critical review of threshold, intensity and descriptive analyses in flavor research. In: Flavor ’81. Walter de Gruyter, Berlin, pp. 3-32. Piggott, J. R. 1990. Relating sensory and chemical data to understand flavor. Journal of Sensory Studies, 4, 261-272. Prescott, J., Norris, L., Kunst, M. and Kim, S. 2005. Estimating a “consumer rejection threshold” for cork taint in white wine. Food Quality and Preference, 18, 345-349. Punter, P. H. 1983. Measurement of human olfactory thresholds for several groups of structurally related compounds. Chemical Senses, 7, 215-235. Rabin, M. D. and Cain, W. S. 1986. Determinants of measured olfactory sensitivity. Perception and Psychophysics, 39(4), 281-286. Reed, D. R., Bartoshuk, L. M., Duffy, V., Marino, S. and Price, R. A. 1995. Propylthiouracil tasting: Determination of underlying threshold distributions using maximum likelihood. Chemical Senses, 20, 529-533. Saliba, A. J., Bullock, J. and Hardie, W. J. 2009. Consumer rejection threshold for 1,8 cineole (eucalyptol) in Australian red wine. Food Qualithy and Preference, 20, 500-504. Schieberle, P. and Grosch, W. 1988. Identification of potent flavor compounds formed in an aqueous lemon oil/citric acid emulsion. Journal of Agricultural and Food Chemistry, 36, 797-800. Stevens, D. A. and O’Connell, R. J. 1991. Inpidual differences in threshold and quality reports of subjects to various odors. Chemical Senses, 16, 57-67. Stevens, D. A. and O’Connell, R. J. 1995. Enhanced sensitivity to adrostenone following regular exposure to pemenone. Chemical Senses, 20, 413-419. Stevens, J. C., Cain, W. S. and Burke, R. J. 1988. Variability of olfactory thresholds. Chemical Senses, 13, 643-653. Stocking, A. J., Suffet, I. H., McGuire, M. J. and Kavanaugh, M. C. 2001. Implications of an MTBE odor study for setting drinking water standards. Journal AWWA, March 2001, 95-105. Todd, P. H., Bensinger, M. G. and Biftu, T. 1977. Determination of pungency due to capsicum by gas-liquid chromatography. Journal of Food Science, 42, 660-665. Tuorila, H., Kurkela, R., Suihko, M. and Suhonen. 1981. The influence of tests and panelists on odour detection

References thresholds. Lebensmittel Wissenschaft und Technologie, 15, 97-101. USEPA (U.S. Environmental Protection Agency). 2001. Statistical analysis of MTBE odor detection thresholds in drinking water. National Service Center for Environmental Publications (NSCEP) # 815R01024, available from http://nepis.epa.gov. Van Gemert, L. 2003. Odour Thresholds. Oliemans, Punter & Partners, Utrecht, The Netherlands. Viswanathan, S., Mathur, G. P., Gnyp, A. W. and St. Peirre, C. C. 1983. Application of probability models to threshold determination. Atmospheric Environment, 17, 139-143.

147 Walker, J. C., Hall, S. B., Walker, D. B., Kendall-Reed, M. S., Hood, A. F. and Nio, X.-F. 2003. Human odor detectability: New methodology used to determine threshold and variation. Chemical Senses, 28, 817-826. Wetherill, G. B. and Levitt, H. 1965. Sequential estimation of points on a psychometric function. British Journal of Mathematical and Statistical Psychology, 18(1), 1-10. Wysocki, C. J., Dorries, K. M. and Beauchamp, G. K. 1989. Ability to perceive androstenone can be acquired by ostensibly anosmic people. Proceedings of the National Academy of Science, USA, 86, 7976-7989.

Chapter 7

Scaling

Abstract Scaling describes the application of numbers, or judgments that are converted to numerical values, to describe the perceived intensity of a sensory experience or the degree of liking or disliking for some experience or product. Scaling forms the basis for the sensory method of descriptive analysis. A variety of methods have been used for this purpose and with some caution, all work well in differentiating products. This chapter discusses theoretical issues as well as practical considerations in scaling. The vital importance of knowing the properties and limitations of a measuring instrument can hardly be denied by most natural scientists. However, the use of many different scales for sensory measurement is common within food science; but very few of these have ever been validated. . . . -(Land and Shepard, 1984, pp. 144-145)

Contents 7.1 7.2 7.3

7.4

7.5

7.6

Introduction . . . . . . . . . . . . . . . . . Some Theory . . . . . . . . . . . . . . . . Common Methods of Scaling . . . . . . . . . 7.3.1 Category Scales . . . . . . . . . . . . 7.3.2 Line Scaling . . . . . . . . . . . . . 7.3.3 Magnitude Estimation . . . . . . . . . Recommended Practice and Practical Guidelines . . . . . . . . . . . . . . . . . . 7.4.1 Rule 1: Provide Sufficient Alternatives . . . . . . . . . . . . . . 7.4.2 Rule 2: The Attribute Must Be Understood . . . . . . . . . . . . 7.4.3 Rule 3: The Anchor Words Should Make Sense . . . . . . . . . . . . . . . . . 7.4.4 To Calibrate or Not to Calibrate . . . . . 7.4.5 A Warning: Grading and Scoring are Not Scaling . . . . . . . . . . . . . . . . Variations-Other Scaling Techniques . . . . 7.5.1 Cross-Modal Matches and Variations on Magnitude Estimation . . . . . . . . . 7.5.2 Category-Ratio (Labeled Magnitude) Scales . . . . . . . . . . . . . . . . 7.5.3 Adjustable Rating Techniques: Relative Scaling . . . . . . . . . . . . . . . . 7.5.4 Ranking . . . . . . . . . . . . . . . 7.5.5 Indirect Scales . . . . . . . . . . . . Comparing Methods: What is a Good Scale? . . . . . . . . . . . . . . . . . . . .

Issues . . . . . . . . . . . . . . . . . . . . 7.7.1 “Do People Make Relative Judgments” Should They See Their Previous Ratings? . . . . . . . . . . . . . . . 7.7.2 Should Category Rating Scales Be Assigned Integer Numbers in Data Tabulation? Are They Interval Scales? . . . . . . . . . 7.7.3 Is Magnitude Estimation a Ratio Scale or Simply a Scale with Ratio Instructions? . . . . . . . . . . . . . 7.7.4 What is a “Valid” Scale? . . . . . . . . 7.8 Conclusions . . . . . . . . . . . . . . . . . Appendix 1: Derivation of Thurstonian-Scale Values for the 9-Point Scale . . . . . . . . . . . . . Appendix 2: Construction of Labeled Magnitude Scales . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . 7.7

149 151 152 152 155 156 158 159 159 159 159 160 160 160 162 164 165 166 167

168

168

169

169 169 170 171 172 174

7.1 Introduction People make changes in their behavior all the time based on sensory experience and very often this involves a judgment of how strong or weak something feels. We add more sugar to our coffee if it is not sweet enough. We adjust the thermostat in our home if it is too cold or too hot. If a closet is too dark to find your shoes you turn the light on. We apply more force to

H.T. Lawless, H. Heymann, Sensory Evaluation of Food, Food Science Text Series, DOI 10.1007/978-1-4419-6488-5_7, © Springer Science+Business Media, LLC 2010

149

150

chew a tough piece of meat if it will not disintegrate to allow swallowing. These behavioral decisions seem automatic and do not require a numerical response. But the same kinds of experiences can be evaluated with a response that indicates the strength of the sensation. What was subjective and private becomes public data. The data are quantitative. This is the basis of scaling. The methods of scaling involve the application of numbers to quantify sensory experiences. It is through this process of numerification that sensory evaluation becomes a quantitative science subject to statistical analysis, modeling, pdiction, and hard theory. However, as noted in the quote above, in the practical application of sensory test methods, the nature of this process of assigning numbers to experiences is rarely questioned and deserves scrutiny. Clearly numbers can be assigned to sensations by a panelist in a variety of ways, some by mere categorization, or by ranking or in ways that attempt to reflect the intensity of sensory experience. This chapter will illustrate these techniques and discuss the arguments that have been raised to substantiate the use of different quantification procedures. Scaling involves sensing a product or stimulus and then generating a response that reflects how the person perceives the intensity or strength of one or more of the sensations generated by that product. This process is based on a psychophysical model (see Chapter 2). The psychophysical model states that as the physical strength of the stimulus increases (e.g., the energy of a light or sound or the concentration of a chemical stimulus) the sensation will increase in some orderly way. Furthermore, panelists are capable of generating different responses to indicate these changes in what they experience. Thus a systematic relationship can be modeled of how physical changes in the real world result in changing sensations. Scaling is a tool used for showing differences and degrees of difference among products. These differences are usually above the threshold level or justnoticeable difference. If the products are very similar and there is a question of whether there is any difference at all, the discrimination testing methods are more suitable (Chambers and Wolf, 1996). Scaling is usually done in one of the two scenarios. In the first, untrained observers are asked to give responses to reflect changes in intensity and it is psumed that (1) they understand the attribute they are asked to scale, e.g., sweetness and (2) there is no need to train or calibrate them to use the scale. This is the kind of scaling done to study a

7

Scaling

Fig. 7.1 The two processes involved in scaling. The first, physiological process is the psychophysical translation of energy in the outside world into sensation, i.e., conscious experience. The second is the translation of that experience into some response. The psychophysical process can be modified by physiological processes such as adaptation and masking. The judgment function can be modified by cognitive processes such as contextual effects, number usage, and other response biases.

7.2 Some Theory

7.2 Some Theory Measurement theory tells us that numbers can be assigned to items in different ways. This distinction was popularized by S. S. Stevens, the major proponent of magnitude estimation (1951). At least four ways of assigning numbers to events exist in common usage. These are referred to as nominal scaling, ordinal scaling, interval scaling, and ratio scaling. In nominal scaling, numbers are assigned to events merely as labels. Thus gender may be coded as a “dummy variable” in statistical analysis by assigning a zero to males and a one to females; no assumption is made that these numbers reflect any ordered property

151

of the sexes. They merely serve as convenient labels. The meals at which a food might be eaten could be coded with numbers as categories-one for breakfast, two for lunch, three for supper, and four for snacks. The assignment of a number for analysis is merely a label, a category or pigeonhole. The appropriate analysis of such data is to make frequency counts. The mode, the most frequent response, is used as a summary statistic for nominal data. Different frequencies of response for different products or circumstances can be compared by chi-square analysis or other nonparametric statistical methods (Siegel, 1956; see Appendix B). The only valid comparisons between inpidual items with this scale is to say whether they belong to the same category or to different ones (an equal versus not equal decision). In ordinal scaling, numbers are assigned to recognize the rank order of products with regard to some sensory property, attitude, or opinion (such as pference). In this case increasing numbers assigned to the products repsent increasing amounts or intensities of sensory experience. So a number of wines might be rank ordered for perceived sweetness or a number of fragrances rank ordered from most pferred to least pferred. In this case the numbers do not tell us anything about the relative differences among the products. We cannot draw conclusions about the degree of difference perceived nor the ratio or magnitude of difference. In an analogy to the order of runners finishing in a race, we know who placed first, second, third, etc. But this order does indicate neither the finishing distances between contestants nor the differences in their elapsed times. In general, analyses of ranked data can report medians as the summary statistic for central tendency or other percentiles to give added information. As with nominal data, nonparametric statistical analyses (see Appendix B) are appropriate when ranking is done (Siegel, 1956). The next level of scaling occurs when the subjective spacing of responses is equal, so the numbers repsent equal degrees of difference. This is called interval-level measurement. Examples in the physical sciences would be the centigrade and Fahrenheit scales of temperature. These scales have arbitrary zero points but equal pisions between values. The scales are inter-convertible through a linear transformation, for example, ◦ C = 5/9 (◦ F-32). Few scales used in sensory science have been subjected to tests that would help establish whether they achieved an interval level of measurement and yet this level is often assumed.

152

7

Scaling

instructed to assign numbers in relative proportions that reflect the strength of their sensations (Stevens, 1956). However, ratio instructions are easy to give, but whether the scale has ratio properties in reflecting a person’s actual subjective experiences is difficult to determine, if not impossible. Because of these different measurement types with different properties, the sensory professional must be careful about two things. First, statements about differences or ratios in comparing the scores for two products should not be made when the measurement level is only nominal or ordinal. Second, it is risky to use parametric statistics for measurements that reflect only frequency counts or rankings (Gaito, 1980; Townsend and Ashby, 1980). Nonparametric methods are available for statistical analyses of such data.

7.3 Common Methods of Scaling Several different scaling methods have been used to apply numbers to sensory experience. Some, like magnitude estimation, are adapted from psychophysical research, and others, like category scaling have become popular through practical application and dissemination in a wide variety of situations. This section illustrates the common techniques of category scales, line marking, and magnitude estimation. The next section discusses the less frequently used techniques of hybrid category-ratio scales, indirect scales, and ranking as alternatives. Two other methods are illustrated. Intensity matching across sensory modalities, called cross-modality matching, was an important psychophysical technique and a pcedent to some of the category-ratio scales. Finally, adjustable rating techniques in which panelists make relative placements and are able to alter their ratings are also discussed.

7.3.1 Category Scales Perhaps the oldest method of scaling involves the choice of discrete response alternatives to signify increasing sensation intensity or degrees of liking and/or pference. The alternatives may be psented in a horizontal or vertical line and may offer choices of integer numbers, simple check boxes, or word phrases. Examples of simple category scales are shown in

7.3 Common Methods of Scaling

153

A) INTENSITY 1

2

3

4

5

6

7

8

9 Strong

Weak

B) Oxidized

not noticeable trace, not sure faint slight mild moderate definite strong very strong

D) Sweetness

Extremely sweet

Not sweet at all

E) Sweetness

R

Weaker

Stronger

F) Hedonic scale for children

Super Good

Really Good

Good

Maybe Good or Maybe Bad

Bad

Really Bad

Super Bad

Fig. 7.2 Examples of category scales. (a) a simple integer scale for sensation strength (after Lawless and Malone, 1986b); (b) a verbal scale for degree of oxidized flavor (after Mecredy et al., 1974; (c) a verbal scale for degree of difference from some reference or control sample (after Aust et al., 1985), (d) a simple

check-box scale for perceived intensity; (e) a simple check-box scale for difference in intensity from some reference sample, marked R (after Stoer and Lawless, 1993); (f) a facial scale suitable for use with children, after Chen et al. (1996).

Fig. 7.2. The job of the consumer or panelist is to choose the alternative that best repsents their reaction or sensation. In a category scale the number of alternative responses is limited. Seven to 15 categories are commonly used for intensity scaling depending upon the application and the number of gradations that the panelists are able to distinguish in the products. As panel training progresses, perceptual discrimination

of intensity levels will often improve and more scale points may be added to allow the panel to make finer distinctions. A key idea is to psent an easily understandable word like “sweetness” and ask the participant to evaluate the perceived intensity of that attribute. A second important factor concerns the verbal labels that appear along the alternatives. At the very least, the low and high ends of the scale must be labeled

154

with words that make sense, e.g., “not sweet at all” to “extremely sweet.” A wide variety of these scales have been used. A common version is to allow integer numerical responses of approximately nine points (e.g., Lawless and Malone, 1986a, b). Further gradations may be allowed. For example, Winakor et al. (1980) allowed options from 1 to 99 in rating attributes of fabric handfeel. In the Spectrum method (Meilgaard et al., 2006) a 15-point category scale is used, but allows intermediate points in tenths, rendering it (at least in theory) a 150point scale. In hedonic or affective testing, a bipolar scale is common, with a zero or neutral point of opinion at the center (Peryam and Girardot, 1952). These are often shorter than the intensity scales. For example, in the “smiling face” scale used with children, only three options may be used for very young respondents (Birch et al., 1980, 1982), although with older children as many as nine points may be used (Chen et al., 1996; Kroll, 1990). Lately there has been a move away from using labels or integers, in case these may be biasing to subjects. People seem to have favorite numbers or tendencies to use some numbers more than others (e.g., Giovanni and Pangborn, 1983). A solution to this problem is to use an unlabeled check-box scale as shown in Fig. 7.2. In early applications of category scaling, the procedure specifically instructed subjects to use the categories to repsent equal spacing. They might also be instructed to distribute their judgments over the available scale range, so the strongest stimulus was rated at the highest category and the weakest stimulus at the lowest category. This use of such explicit instructions surfaces from time to time. An example is in Anderson’s (1974) recommendation to show the subject specific examples of bracketing stimuli that are above and below the anticipated range of items in the set to be judged. A related method is the relative scaling procedure of Gay and Mead (1992) in which subjects place the highest and lowest stimuli at the scale endpoints (discussed below). The fact that there is an upper boundary to the allowable numbers in a category scaling task may facilitate the achievement of a linear interval scale (Banks and Coleman, 1981). A related issue concerns what kind of experience the high end anchor refers to. Muñoz and Civille (1998) pointed out that for descriptive analysis panels, the high end-anchor phrase could refer to different situations. For example, is the term “extremely sweet”

7

Scaling

7.3 Common Methods of Scaling

7.3.2 Line Scaling

155

a) very poor

excellent

AROMA

b) moderate

weak

strong

Pepper Heat

c)

O threshold

slight

moderate

strong

Sweetness

d) weaker

reference

stronger

overall opinion

e) dislike moderately

f)

X

least liked

neither

like moderately

X most liked

Fig. 7.3 Examples of line-marking scales: (a) with endpoints labeled (after Baten, 1946); (b) with indented “goal posts” (after Mecredy et al., 1974); (c) with additional points labeled as in ASTM procedure E-1083 (ASTM, 2008b); (d) a line for ratings relative to a reference as in Stoer and Lawless (1993); (e) hedonic scaling using a line; (f) the adjustable scale of Gay and Mead (1992) as pictured by Villanueva and D Silva (2009).

An important historical trend to use lines in descriptive analysis was instrumental in the popularization of line marking. Stone et al. (1974) recommended the use of line marking for Quantitative Descriptive Analysis (QDA), then a relatively new approach to specifying the intensities of all important sensory attributes. It was important to have a scaling method in QDA that approximated an interval scale, as analysis of variance was to become the standard statistical technique for comparing products in descriptive analysis. The justification for the application in QDA appears to rest on the pvious findings of Baten regarding the sensitivity of the method and the writings of Norman Anderson on functional measurement theory (Anderson, 1974). In his approach, Anderson used indented end anchors (see Weiss, 1972, for another example). Anderson also showed his subjects examples of the high and

156

7

Scaling

with a cursor that moves up and down via the computer mouse. Time-intensity methods are reviewed more fully in Chapter 8.

7.3.3 Magnitude Estimation 7.3.3.1 The Basic Techniques A popular technique for scaling in psychophysical studies has been the method of magnitude estimation. In this procedure, the respondent is instructed to assign numbers to sensations in proportion to how strong the sensation seems. Specifically, the ratios between the numbers are supposed to reflect the ratios of sensation magnitudes that have been experienced. For example, if product A is given the value of 20 for sweetness intensity and product B seems twice as sweet, B is given a magnitude estimate of 40. The two critical parts of the technique are the instructions given to the participant and the techniques for data analysis. Two primary variations of magnitude estimation have been used. In one method, a standard stimulus is given to the subject as a reference and that standard is assigned a fixed value such as 10. All subsequent stimuli are rated relative to this standard, sometimes called a “modulus.” It is often easier for panelists if the reference (i.e., the item used as the modulus) is chosen from somewhere near the middle of the intensity range. In the other variation of magnitude estimation, no standard stimulus is given and the participant is free to choose any number he or she wishes for the first sample. All samples are then rated relative to this first intensity, although in practice people probably “chain” their ratings to the most recent items in the series. Because people can choose different ranges of numbers in this “non-modulus” magnitude estimation, the data have to be treated to bring all judgments into the same range, an extra step in the analysis. Variations on magnitude estimation and guidelines for data analysis are found in ASTM Standard Test Method E 1697-05 (ASTM, 2008a). In the psychophysical laboratory, where magnitude estimation has found its primary usage, generally only one attribute is rated at a time. However, rating multiple attributes or profiling has been used in taste studies (McBurney and Bartoshuk, 1973; McBurney and Shick, 1971; McBurney et al., 1972) and this can

7.3 Common Methods of Scaling

naturally be extended to foods with multiple taste and aromatic attributes. Magnitude estimation has not been used very often for descriptive analysis, but in principle there is no reason why it could not be used for that purpose. Participants should be cautioned to avoid falling into pvious habits of using bounded category scales, e.g., limited ranges of numbers from zero to ten. This may be a difficult problem with pviously trained panels that have used a different scaling method, as people like to stick with a method they know and feel comfortable with. Panelists who show such behavior may not understand the ratio nature of the instructions. It is sometimes useful with a new panelist to have the participant perform a warm-up task to make sure they understand the scaling instructions. The warm-up task can involve estimation of the size or area of different geometric ps (Meilgaard et al., 2006) or the length of lines (McBurney et al., 1972). Sometimes it is desired to have panelists rate multiple attributes at the same time or to break down overall intensity into specific qualities. If this kind of “profiling” is needed, the geometric ps can include areas with different shading or the lines can be differently colored. A practice task is highly recommended so that the sensory scientist can check on whether the participant understands the task. Values of zero are allowed in this method as some of the products may in fact have or no sensation for a given attribute (like no sweetness in our example). Of course, the rating of zero should not be used for the reference material. While the value of zero is consistent with common sense for products with no sensation of some attributes, it does complicate the data analysis as discussed below. 7.3.3.2 Instructions The visual appearance of the ballot in magnitude estimation is not critical; it is the instructions and the participant’s comphension of the ratio nature of the judgments that are important. Some ballots even allow the subject/participant to view all pvious ratings. Here are sample instructions for the use of magnitude estimation with a reference sample or modulus with a fixed number assigned to it: Please taste the first sample and note its sweetness. This sample is given the value of “10” for its sweetness

157 intensity. Please rate all other samples in proportion to this reference. For example, if the next sample is twice as sweet, assign it a value of “20”, if half as sweet, assign it a value of “5” and if 3.5 times as sweet, assign in a value of 35. In other words, rate the sweetness intensity so that your numbers repsent the ratios among the intensities of sweetness. You may use any positive numbers including fractions and decimals.

The other major variation on this method uses no reference. In this case the instructions may read as follows: Please taste the first sample and note its sweetness. Please rate all other samples relative to this reference, applying numbers to the samples to repsent the ratios of sweetness intensity among the samples. For example, if the next sample was twice as sweet, you would give it a number twice as big as the rating assigned to the first sample, if half as sweet, assign it a number half as big and if 3.5 times as sweet, assign it a number 3.5 times as big. You may use any positive numbers including fractions and decimals.

7.3.3.3 Data Treatment In non-modulus methods, participants will generally choose some range of numbers they feel comfortable with. The ASTM procedure suggests having them pick a value between 30 and 100 for the first sample, and avoiding any number that seems small. If participants are allowed to choose their own number range, it becomes necessary to re-scale each inpidual’s data to bring them into a common range before statistical analysis (Lane et al., 1961). This will pvent subjects who choose very large numbers from having undue influence on measures of central tendency (means) and in statistical tests. This rescaling process has been referred to as “normalizing” (ASTM, 2008a) although it has nothing to do with the normal distribution or Z-scores. A common method for rescaling proceeds as follows: (1) Calculate the geometric mean of each inpidual’s ratings across their data set. (2) Calculate the geometric mean of the entire data set (of all subjects combined). (3) For each subject, construct a ratio of the grand geometric mean of the entire data set to each person’s geometric mean. The value of this ratio provides a post hoc inpidual rescaling factor for each subject. In place of the grand geometric mean,

158

any positive numerator may also be chosen in constructing this factor, e.g., a value of 100. (4) Multiply each data point for a given person by their inpidual rescaling factor. Do this for all participants using their own inpidual re-scaling factors. These re-scaled data are then analyzed. Note that due to the extra data treatment step in this method, it is simpler to use the modulus-based variation with a standard reference item. Magnitude estimation data are often transformed to logs before data analysis (Butler et al., 1987; Lawless, 1989). This is done primarily because the data tend to be positively skewed or log-normally distributed. There tends to be some high outlying values for any given sample. Perhaps this is not surprising because the scale is open-ended at the top, and bounded by zero at the bottom. Transformation into log data and/or taking geometric means psents some problems, however, when the data contain zeros. The log of zero is undefined. Any attempt to take a geometric mean by calculating the product of N items will yield a zero on multiplying. Several approaches have been taken to this problem. One is to assign a small positive value to any zeros in the data, perhaps one-half of the smallest rating given by a subject (ASTM, 2008a). The resulting analysis, however, will be influenced by this choice. Another approach is to use the median judgments in constructing the normalization factor for non-modulus methods. The median is less influenced by the high outliers in the data than the arithmetic mean.

7.3.3.4 Applications For practical purposes, the method of magnitude estimation may be used with trained panels, consumers, and even children (Collins and Gescheider, 1989). However, the data do tend to be a bit more variable than other bounded scaling methods, especially in the hands of untrained consumers (Lawless and Malone, 1986b). The unbounded nature of the scale may make it especially well suited to sensory attributes where an upper boundary might impose restrictions on the panelists’ ability to differentiate very intense sensory experiences in their ratings. For example, irritative or painful sensations such as chili pepper intensity might all be rated near the upper bound of a category scale

7

Scaling

of intensity, but an open-ended magnitude estimation procedure would allow panelists more freedom to differentiate and report variations among very intense sensations. With hedonic scaling of likes and dislikes, there is an additional decision in using magnitude estimation scaling. Two options have been adopted in using this technique, one employing a single continuum or unipolar scale for amount of liking and the other applying a bipolar scale with positive and negative numbers plus a neutral point (Pearce et al., 1986). In bipolar magnitude scaling of likes and dislikes, positive and negative numbers are allowed in order to signify ratios or proportions of both liking and disliking (e.g., Vickers, 1983). An alternative to positives and negatives is to have the respondent merely indicate whether the number repsents liking or disliking (Pearce et al., 1986). In unipolar magnitude estimation only positive numbers (and sometimes zeros) are allowed, with the lower end of the scale repsenting no liking and higher numbers given to repsent increasing proportions of liking (Giovanni and Pangborn, 1983; Moskowitz and Sidel, 1971). It is questionable whether a unipolar scale is a sensible response task for the participant, as it does not recognize the fact that a neutral hedonic response may occur, and that there are clearly two modes of reaction, one for liking and one for disliking. If one assumes that all items are on one side of the hedonic continuum-either all liked to varying degrees or all disliked to varying degrees then the one-sided scale makes sense. However, it is a rare situation with foods or consumer product testing in which at least some indifference or change of opinion was not visible in at least some respondents. So a bipolar scale fits common sense.

7.4 Recommended Practice and Practical Guidelines Both line scales and category scales may be used effectively in sensory testing and consumer work. So we will not expend much effort in recommending one of these two common techniques over another. Some practical concerns are given below to help the student or practitioner avoid some potential problems. The category-ratio or labeled magnitude scales may facilitate comparisons among different groups, and this issue is discussed below in Section 7.5.2.

7.4 Recommended Practice and Practical Guidelines

7.4.1 Rule 1: Provide Sufficient Alternatives One major concern is to provide sufficient alternatives to repsent the distinctions that are possible by panelists (Cox, 1980). In other words, a simple 3-point scale may not suffice if the panel is highly trained and capable of distinguishing among many levels of the stimuli. This is illustrated in the case of the flavor profile scale, which began with five points to repsent no sensation, threshold sensation, weak, moderate, and strong (Caul, 1957). It was soon discovered that additional intermediate points were desirable for panelists, especially in the middle range of the scale where many products would be found. However, there is a law of diminishing returns in allowing too many scale points-further elaboration allows better differentiation of products up to a point and then the gains diminish as the additional response choices merely capture random error variation (Bendig and Hughes, 1953). A related concern is the tendency, especially in consumer work, to simplify the scale by eliminating options or truncating endpoints. This brings in the danger caused by end-use avoidance. Some respondents may be reluctant to use the end categories, just in case a stronger or weaker item may be psented later in the test. So there is some natural human tendency to avoid the end categories. Truncating a 9-point scale to a 7-point scale may leave the evaluator with what is functionally only a 5-point scale for all practical purposes. So it is best to avoid this tendency to truncate scales in experimental planning.

7.4.2 Rule 2: The Attribute Must Be Understood Intensity ratings must be collected on an attribute which the participants understand and about which they have a general consensus and agreement as to its meaning. Terms like sweetness are almost universal but a term like “green aroma” might be interpted in different ways. In the case of a descriptive panel, a good deal of effort may be directed at using reference standards to illustrate what is meant by a specific term. In the case of consumer work, such training is not done, so if any intensity ratings are collected, they must be about simple terms about which people generally

159

agree. Bear in mind that most early psychophysics was done on simple attributes like the loudness of a sound or the heaviness of a weight. In the chemical senses, with their perse types of sensory qualities and fuzzy consumer vocabulary, this is not so straightforward. Other problems to avoid include mixing sensation intensity (strength) and hedonics (liking), except in the just-right scale where this is explicit. An example of a hedonically loaded sensory term is the adjective “fresh.” Whatever this means to consumers, it is a poor choice for a descriptive scale because it is both vague and connotes some degree of liking or goodness to most people. Vague terms are simply not actionable when it comes to giving feedback to product developers about what needs to be fixed. Another such vague term that is popular in consumer studies is “natural.” Even though consumers might be able to score products on some unknown basis using this word, the information is not useful as it does not tell formulators what to change if a product scores low. A similar problem arises with attempting to scale “overall quality.” Unless quality has been very carefully defined, it cannot be scaled.

7.4.3 Rule 3: The Anchor Words Should Make Sense In setting up the scales for descriptive analysis or for a consumer test, the panel leader should carefully consider the nature of the verbal end anchors for each scale as well as any intermediate anchors that may be needed. Should the scale be anchored from “very weak” to “very strong” or will there be cases in which the sensory attribute is simply not psent? If so, it makes sense to verbally anchor the bottom of the scale with “not at all” or “none.” For example, a sweetness scale could be anchored with “not at all sweet” and a scale for “degree of oral irritation” could be anchored using “none.”

160

7.4.5 A Warning: Grading and Scoring are Not Scaling In some cases pseudo-numerical scales have been set up to resemble category scales, but the examples cut

7

Scaling

across different sensory experiences, mixing qualities. An example is the pseudo-scale used for baked products, where the number 10 is assigned for perfect texture, 8 for slight dryness, 6 for gumminess, and 4 if very dry (AACC, 1986). Gumminess and dryness are two separate attributes and should be scaled as such. This is also an example of quality grading, which is not a true scaling procedure. When the numbers shift among different sensory qualities, this violates the psychophysical model for scaling the intensity of a single attribute. Although numbers may be applied to the grades, they cannot be treated statistically, as the average of “very dry ” (4) and slightly dry (8) is not gummy (6) (see Pangborn and Dunkley, 1964, for a critique of this in the dairy grading arena). The numbers in a quality-grading scheme do not repsent any kind of unitary psychophysical continuum.

7.5 Variations-Other Scaling Techniques An important idea in scaling theory is the notion that people may have a general idea of how weak or strong sensations are, and that they can compare different attributes of a product for their relative strength, even across different sensory modalities. So, for example, someone could legitimately say that this product tastes much more salty than it is sweet. Or that the trumpets are twice as loud as the flutes in a certain passage in a symphony. Given that this notion is correct, people would seem to have a general internal scale for the strength of sensations. This idea forms the basis for several scaling methods. It permits the comparison of different sensations cross-referenced by their numerical ratings, and even can be used to compare word responses. Methods derived from this idea are discussed next.

7.5.1 Cross-Modal Matches and Variations on Magnitude Estimation The method of magnitude estimation has a basis in earlier work such as fractionation methods and the method of sense ratios in the older literature (Boring,

7.5 Variations-Other Scaling Techniques

161

1942) where people would be asked to set one stimulus in a given sensation ratio to another. The notion of allowing any numbers to be generated in response to stimuli, rather than adjusting stimuli to repsent fixed numbers appeared somewhat later (Moskowitz, 1971; Richardson and Ross, 1930; Stevens, 1956). An important outcome of these studies was the finding that the resulting psychophysical function generally conformed to a power law of the following form:

brightness of the lights about in the same proportions as the loudness of the sounds. Stevens (1969) proposed that these experiments could validate the power law, since the exponents of the cross-modality matching function could be pdicted from the exponents derived from separate scaling experiments. Consider the following example: For one sensory attribute (using the log transform, Eq. (7.2)),

R = kI n

log R1 = n1 log I1 + log k1

(7.1)

and for a second sensory attribute

or after log transformation: log(R) = n log(I) + log(k)

(7.3)

(7.2)

where R was the response, e.g., perceived loudness (mean or geometric mean of the data) and I was the physical stimulus intensity, e.g., sound pssure, and k was a constant of proportionality that depends upon the units of measurement. The important characteristic value of any sensory system was the value for n, the exponent of the power function or slope of the straight line in a log-log plot (Stevens, 1957). The validity of magnitude estimation then came to hang on the validity of the power law-the methods and resulting functions formed an internally consistent theoretical system. Stevens also viewed the method as providing a direct window into sensation magnitude and did not question the idea that these numbers generated by subjects might be biased in some way. However, the generation of responses is properly viewed as combining at least two processes, the psychophysical transformation of energy into conscious sensation and the application of numbers to those sensations. This response process was not given due consideration in the early magnitude estimation work. Responses as numbers can exhibit nonlinear transformations of sensation (Banks and Coleman, 1981; Curtis et al., 1968) so the notion of a direct translation from sensation to ratings is certainly a dangerous oversimplification. Ratio-type instructions have been applied to other techniques as well as to magnitude estimation. A historically important psychophysical technique was that of cross-modality matching, in which the sensation levels or ratios would be matched in two sensory continua such as loudness and brightness. One continuum would be adjusted by the experimenter and the other by the subject. For example, one would try to make the

log R2 = n2 log I2 + log k2

(7.4)

Setting R1 = R2 in cross-modality matching gives n1 log I1 + log k1 = n2 log I2 + log k2

(7.5)

and rearranging, log I1 = (n2 /n1 ) log I2 + a constant

(7.6)

If one plots log I1 against log I2 from a crossmodality matching task, the slope of the function can be pdicted from the ratio of the slopes of the inpidual exponents (i.e., n2 /n1 , which you can derive from two separate magnitude estimation tasks). This pdiction holds rather well for a large number of compared sensory continua (Steven, 1969). However, whether it actually provides a validation for the power law or for magnitude estimation has been questioned (e.g., Ekman, 1964). For practical purposes, it is instructive that people can actually take disparate sensory continua and compare them using some generalized notion of sensory intensity. This is one of the underpinnings of the use of a universal scale in the Spectrum descriptive procedure (Meilgaard et al., 2006). In that method, different attributes are rated on 15-point scales that can (in theory) be meaningfully compared. In other words, a 12 in sweetness is twice as intense a sensation as a 6 in saltiness. Such comparisons seem to makes sense for tastes and flavors but may not cut across all other modalities. For example, it might seem less sensible to compare the rating given for the amount of chocolate chips in a cookie to the rating given for the cookie’s hardness-these seem like quite different experiences to quantify.

162

7

Scaling

or line scale? The idea of scaling word phrases takes shape in the labeled magnitude scales discussed next.

7.5.2 Category-Ratio (Labeled Magnitude) Scales A group of hybrid techniques for scaling has recently enjoyed some popularity in the study of taste and smell, for hedonic measurement and other applications. One of the problems with magnitude estimation data is that it does not tell in any absolute sense whether sensations are weak or strong, only giving the ratios among them. This group of scales attempts to provide ratio information, but combines it with common verbal descriptors along a line scale to provide a simple frame of reference. They are referred to as category-ratio scales, or more recently, labeled magnitude scales. They all involve a horizontal or vertical line with deliberately spaced labels and the panelists’ task is to make a mark somewhere along the line to indicate the strength of their perception or strength of their likes or dislikes. In general, these labeled line scales give data that are consistent with those from magnitude estimation (Green et al., 1993). An unusual characteristic of these scales is the verbal high endanchor phrase, which often refers to the “strongest imaginable.” The technique is based on early work by Borg and colleagues, primarily in the realm of perceived physical exertion (Borg, 1982, 1990; see Green et al., 1993). In developing this scale, Borg assumed that the semantic descriptors could be placed on a ratio scale and that they defined the level of perceptual intensity and that all inpiduals experienced the same perceptual range. Borg suggested that for perceived exertion, the maximal sensation is roughly equivalent across people for this sensory “modality” (Marks et al., 1983). For example, it is conceivable that riding a bicycle to the point of physical exhaustion produces a similar sensory experience for most people. So the scale came to have the highest label referring to the strongest sensation imaginable. This led to the development of the labeled magnitude scale (LMS) shown in Fig. 7.4. It is truly a hybrid method since the response is a vertical linemarking task but verbal anchors are spaced according to calibration using ratio-scaling instructions (Green

7.5 Variations-Other Scaling Techniques

Strongest imaginable

Very strong

(95.5)

(50.12)

Strong

(33.1)

Moderate

(16.2)

Weak Barely detectable

163

LMS

(5.8) (1.4)

Fig. 7.4 The labeled magnitude scale (LMS) of Green et al. (1993).

2007; Lawless et al., 2010a, b, c). A growing number of similar scales have been developed for various applications including oral pleasantness/unpleasantness (the “OPUS” scale, Guest et al., 2007), perceived satiety (the “SLIM” scale, Cardello et al., 2005), clothing fabric comfort (the “CALM” scale, Cardello et al., 2003), and odor dissimilarity (Kurtz et al., 2000). All of these scales depend upon a ratio scaling task to determine the spacing of the verbal descriptors and almost all use a Borg-type high end-anchor phrase. Others will surely be developed. Instructions to participants have differed in the use of these scales. In the first application of the LMS, Green et al. (1993) instructed subjects to first choose the most appropriate verbal descriptor, and then to “fine tune” their judgment by placing a mark on the line between that descriptor and the next most appropriate one. In current practice less emphasis may be placed on the consideration of the verbal labels and instructions may be given to simply make a mark “anywhere” on the line. A common observation with the hedonic versions of the scale is that some panelists will mark at or very near a verbal descriptor, seeming to use it as a category scale (Cardello et al., 2008; Lawless et al., 2010a). The proportion of people displaying this behavior may depend upon the physical length of the line (and not the instructions or examples that may be shown) (Lawless et al., 2010b). Results may depend in part on the nature of the high end-anchor example and the frame of reference of the subject in terms of the sensory modality they are thinking about. Green et al. (1996) studied the application of the LMS to taste and odor, using descriptors for the upper bound as “strongest imaginable” taste, smell, sweetness, etc. Steeper functions (a smaller response range) were obtained when mentioning inpidual taste qualities. This appears to be due to the omission of painful experiences (e.g., the “burn of hot peppers”) from the frame of reference when sensations were scaled relative to only taste. The steepening of the functions for the truncated frame of reference is consistent with the idea that subjects expanded their range of numbers as seen in other scaling experiments (e.g., Lawless and Malone, 1986b). The fact that subjects appear to adjust their perceptual range depending on instructions or frame of reference suggests that the scales have relative and not absolute properties, like most other scaling methods. Cardello et al. (2008) showed that the hedonic version of the scale (the LAM

164 Greatest imaginable like

Most imaginable

(+100)

Scaling (+100)

Like Extremely

(+74.2)

extremely

(+75.2)

Like Very Much

(+56.1)

very

(+56.1)

Like Moderately

(+36.2)

moderately

(+33.9)

Like Slightly

(+11.2)

slightly weakly

(+19.7) (+15.0)

PLEASANT

Fig. 7.5 Affective labeled magnitude scales, including the LAM scale (Cardello and Schutz, 2004) and the OPUS scale (Guest et al., 2007).

7

barely detectable Neither like nor dislike

(+6.5) (0)

(0)

(+6.5)

(-10.6)

barely detectable weakly slightly

(+15.0) (+19.7)

Dislike moderately

(-31.9)

moderately

(+33.9)

Dislike Very much

(-55.5)

very

(+56.1)

Dislike extremely

(-75.5)

extremely

(+75.2)

Greatest imaginable dislike

(-100)

LAM scale) will also show such range effects. A compssed range of responses is obtained when the frame of reference is greatest imaginable like (dislike) for an “experience of any kind” rather than something more delimited like “foods and beverages.” Apparently the compssion is not very detrimental to the ability of the LAM scale to differentiate products (Cardello et al., 2008, but see also Lawless et al., 2010a). Is this a suitable method for cross-subject comparisons? To the extent that Borg’s assumptions of common perceptual range and the similarity of the high end-anchor experience among people are true, the method might provide one approach to valid comparisons of the ratings among different respondents. This would facilitate comparisons of clinical groups or patients with sensory disorders or genetically different inpiduals such as anosmics, PTC/PROP taster groups. Bartoshuk and colleagues (1999, 2003,

UNPLEASANT

Dislike slightly

Most imaginable

(-100)

OPUS 2004a, b, 2006) have argued that the labeled magnitude scales should anchor their endpoints to “sensations of any kind” as such a reference experience would allow different inpiduals to use the scale in similar ways/and thus facilitate inter-inpidual comparisons. Scales with this kind of high end anchor have been termed “generalized” labeled magnitude scales (or gLMS). However, the sensory evaluation practitioner should be aware of the compssion effects that can occur with this kind of scale, which could potentially lead to lessened differentiation among products.

7.5.3 Adjustable Rating Techniques: Relative Scaling A few methods have been tried that allow consumers or panelists to change their ratings. An example is

7.5 Variations-Other Scaling Techniques

165

7.5.4 Ranking Another alternative to traditional scaling is the use of ranking procedures. Ranking is simply ordering the products from weakest to strongest on the stated

166

7.5.5 Indirect Scales A conservative approach to scaling is to use the variance in the data as units of measurement, rather than the numbers taken at face value. For example, we could

7

Scaling

ask how many standard deviations apart are the mean values for two products. This is a different approach to measurement than simply asking how many scale units separate the means on the response scale. On a 9-point scale, one product may receive mean rating of seven, and another nine, making the difference two scale units. If the pooled standard deviation is two units, however, they would only be one unit apart on a variability-based scale. As one example, Conner and Booth (1988) used both the slope and the variance of functions from just-right scales to derive a “tolerance discrimination ratio.” This ratio repsents a measure of the degree of difference along a physical continuum (such as concentration of sugar in lime drink) that observers find to make a meaningful change in their ratings of difference-from-ideal (or just-right). This is analogous to finding the size of a just-noticeable difference, but translated into the realm of hedonic scaling. Their insight was that it is not only the slope of the line that is important in determining this tolerance or liking-discrimination function, but also the variance around that function. Variability-based scales are the basis for scaling in Thurstone’s models for comparative judgment (Thurstone, 1927) and its extension into determining distances between category boundaries (Edwards, 1952). Since the scale values can be found from choice experiments as well as rating experiments, the technique is quite flexible. How this type of scaling can be applied to rating data is discussed below in the derivation of the 9-point hedonic scale words. When the scale values are derived from a choice method like the triangle test or paired comparison method, this is sometimes called “indirect scaling” (Baird and Noma, 1978; Jones, 1974). The basic operation in Thurstonian scaling of choice data is to convert the proportion correct in a choice experiment (or simply the proportions in a two-tailed test like paired pference) to Z-scores. The exact derivation depends upon the type of test (e.g., triangle versus 3-AFC) and the cognitive strategy used by the subject. Tables for Thurstonian scale values from various tests such as the triangle test were given by Frijters et al. (1980) and Bi (2006) and some tables are given in the Appendix. Mathematical details of Thurstonian scaling are discussed in Chapter 5. Deriving measures of sensory differences in such indirect ways psents several problems in applied sensory evaluation so the method has not been widely used. The first problem is one of economy in data

7.6 Comparing Methods: What is a Good Scale?

collection. Each difference score is derived from a separate discrimination experiment such as a paired comparison test. Thus many subjects must be tested to get a good estimate of the proportion of choice, and this yields just one scale value. In direct scaling, each participant gives at least one data point for each item tasted. Direct scaling allows for easy comparisons among multiple products, while the discrimination test must be done on one pair at a time. Thus the methods of indirect scaling are not cost-efficient. A second problem can occur if the products are too clearly different on the attribute in question, because then the proportion correct will approach 100%. At that point the scale value is undefined as they are some unknown number of standard deviations apart. So the method only works when there are small differences and some confusability of the items. In a study of many products, however, it is sometimes possible to compare only adjacent or similar items, e.g., products that differ in small degrees of some ingredient or process variable. This approach was taken by Yamaguchi (1967) in examining the synergistic taste combination of monosodium glutamate and disodium 5 inosinate. Many different levels of the two ingredients were tasted, but because the differences between some levels were quite apparent, an incomplete design was used in which only three adjacent levels were compared. Other applications have also used this notion of variability as a yardstick for sensory difference or sensation intensity. The original approach of Fechner in constructing a psychophysical function was to accumulate the difference thresholds or just-noticeable differences (JNDs) in order to construct the log function of psychophysical sensory intensity (Boring, 1942; Jones, 1974). McBride (1983a, b) examined whether JND-based scales might give similar results to category scales for taste intensity. Both types of scaling yielded similar results, perhaps not surprising since both tend to conform to log functions. In a study of children’s pferences for different odors Engen (1974) used a paired pference paradigm, which was well suited to the abilities of young children to respond in a judgment task. He then converted the paired pference proportions to Thurstonian scale values via Z-scores and was able to show that the hedonic range of children was smaller than that of adults. Another example of choice data that can be converted to scale values is best-worst scaling, in which

167

a consumer is asked to choose the best liked and least liked samples from a set of three or more items (Jaeger et al., 2008). With three products, it can be considered a form of a ranking task. When applied to sensory intensity, this is sometimes known as maximum-difference or “max-diff.” Best-worst scaling is also discussed in Section 13.7. Simple difference scores may be calculated based on the number of times an item is called best versus worst and these scores are supposed to have interval properties. If a multinomial logistic regression is performed on the data, they are theorized to have true ratio properties (Finn and Louviere, 1992). A practical problem with the method, however, is that so many products must be tasted and compared, rendering it difficult to perform with foods (Jaeger and Cardello, 2009). The sensory professional should bear in mind that in spite of their theoretical sophistication, the indirect methods are based on variability as the main determinant of degree of difference. Thus any influence, which increases variability, will tend to decrease the measured differences among products. In the wellcontrolled psychophysical experiment under constant standard conditions across sessions and days, this may not be important-the primary variability lies in the resolving power of the participant (and secondarily in the sample products). But in drawing conclusions across different days, batches, panels, factories, and such, one has a less pure situation to consider. Whether one considers the Thurstonian-type indirect measures comparable across different conditions depends upon the control of extraneous variation.

7.6 Comparing Methods: What is a Good Scale? A large number of empirical studies have been conducted comparing the results using different scaling methods (e.g., Birnbaum, 1982; Giovanni and Pangborn, 1983; Hein et al., 2008; Jaeger and Cardello, 2009; Lawless and Malone, 1986a, b; Lawless et al., 2010a; Marks et al., 1983; Moskowitz and Sidel, 1971; Pearce et al., 1986; Piggot and Harper, 1975; Shand et al., 1985; Vickers, 1983; Villanueva and Da Silva, 2009; Villanueva et al., 2005). Because scaling data are often used to identify differences between products, the ability to detect differences is one important

168

practical criterion for how useful a scaling method can be (Moskowitz and Sidel, 1971). A related criterion is the degree of error variance or similar measures such as size of standard deviations or coefficients of variation. Obviously, a scaling method with low interinpidual variability will result in more sensitive tests, more significant differences, and lower risk of Type II error (missing a true difference). A related issue is the reliability of the procedure. Similar results should be obtained upon repeated experimentation. Other practical considerations are important as well. The task should be user friendly and easy to understand for all participants. Ideally, the method should be applicable to a wide range of products and questions, so that the respondent is not confused by changes in response type over a long ballot or questionnaire. If panelists are familiar with one scale type and are using it effectively, there may be some liability in trying to introduce a new or unfamiliar method. Some methods, like category scales, line scales, and magnitude estimation, can be applied to both intensity and hedonic (like-dislike) responses. The amount of time required to code, tabulate and process the information may be a concern, depending upon computer-assisted data collection and other resources available to the experimenters. As in any method, validity or accuracy are also issues. Validity can only be judged by reference to some external criterion. For hedonic scaling, one might want the method to correspond to other behaviors such as choice or consumption (Lawless et al., 2010a). A related criterion is the ability of the scale to identify or uncover consumer segments with different pferences (Villanueva and Da Silva, 2009). Given these practical considerations, we may then ask how the different scaling methods fare. Most published studies have found about equal sensitivity for the different scaling methods, provided that the methods are applied in a reasonable manner. For example, Lawless and Malone (1986a, b) performed an extensive series of studies (over 20,000 judgments) with consumers in central location tests using different sensory continua including olfaction, tactile, and visual modalities. They compared line scales, magnitude estimation, and category scales. Using the degree of statistical differentiation among products as the criterion for utility of the methods, the scales performed about equally well. A similar conclusion was reached by Shand et al. (1985) for trained panelists. There

7

Scaling

was some small tendency for magnitude estimation to be marginally more variable in the hands of consumers as opposed to college students (Lawless and Malone, 1986b). Statistical differentiation increased over replicates, as would be expected as people came to understand the range of items to be judged (see Hein et al., 2008, for another example of improvement over replication in hedonic scaling). Similar findings for magnitude estimation and category scales in terms of product differentiation were found by Moskowitz and Sidel (1971), Pearce et al. (1986), Shand et al. (1985), and Vickers (1983) although the forms of the mathematical relations to underlying physical variables was often different (Piggot and Harper, 1975). In other words, as found by Stevens and Galanter (1957) there is often a curvilinear relationship between the data from the two methods. However, this has not been universally observed and sometimes simple linear relationships have been found (Vickers, 1983). Similar results for category scales and line scales were found by Mattes and Lawless (1985). Taken together, these empirical studies paint a picture of much more parity among methods than one might suppose given the number of arguments over the validity of scaling methods in the psychological literature. With reasonable spacing of the products and some familiarization with the range to be expected, respondents will distribute their judgments across the available scale range and use the scale appropriately to differentiate the products. A reasonable summary of the literature comparing scale types is that they work about equally well to differentiate products, given a few sensible pcautions.

7.7 Issues

responses (i.e., judgments that might fall in-between categories). The line scale offers a continuously graded choice of alternatives, limited only by the measurement abilities in data tabulation. Baten also noted that the line scale seemed to facilitate a relative comparison among the products. This was probably due to his placement of the scales one above the other on the ballot, so judges could see both markings at the same time. In order to minimize such contextual effects it is now more common to remove the prior ratings for products to achieve a more independent judgment of the products. However, whether that is ever achieved in practice is open to question-humans are naturally comparative when asked to evaluate items, as discussed in Chapter 9. Furthermore, there may be potential for increased discrimination in methods like the relative positioning technique. The naturally comparative nature of human judgment may be something we could benefit from rather than trying to fight this tendency by over-calibration.

7.7.2 Should Category Rating Scales Be Assigned Integer Numbers in Data Tabulation? Are They Interval Scales? There is also a strong suspicion that many numerical scaling methods may produce only ordinal data, because the spacing between alternatives is not subjectively equal. A good example is the common marketing research scale of “excellent-very good-good- fair-poor.” The subjective spacing between these adjectives is quite uneven. The difference between two products rated good and very good is a much smaller difference than that between products rated fair and poor. However, in analysis we are often tempted to assigned numbers one through five to these categories and take means and perform statistics as if the assigned numbers reflected equal spacing. This is a ptense at best. A reasonable analysis of the 5-point excellent to poor scale is simply to count the number of respondents in each category and to compare frequencies. Sensory scientists should not assume that any scale has interval properties in spite of how easy it is to tabulate data as an integer series.

169

7.7.4 What is a “Valid” Scale? An ongoing issue in psychophysics is what kind of scale is a true reflection of the subject’s actual sensations? From this perspective, a scale is valid when the numbers generated reflect a linear translation of subjective intensity (the private experience). It is well established that category scales and magnitude estimates, when given to the same stimuli, will form a curve when plotted against one another (Stevens and Galanter, 1957). Because this is not a linear relationship, one method or the other must result from a non-linear translation of the subjective intensities of the stimuli. Therefore, by this criterion, at least one scale must be “invalid.”

170

7

Anderson (1974, 1977) proposed a functional measurement theory to address this issue. In a typical experiment, he would ask subjects to do some kind of combination task, like judging the total combined intensity of two separately psented items (or the average lightness of two gray swatches). He would set up a factorial design in which every level of one stimulus was combined with every level of the other (i.e., a complete block). When plotting the response, a family of lines would be seen when the first stimulus continuum formed the X-axis and the second formed a family of lines. Anderson argued that only when the response combination rule was additive, and the response output function was linear, would a parallel plot be obtained (i.e., there would be no significant interaction term in ANOVA). This argument is illustrated in Fig. 7.6. In his studies using simple line and category scales, parallelism was obtained in a number of studies, and thus he reasoned that magnitude estimation was invalid by this criterion. If magnitude estimation is invalid, then its derivatives such as the LMS and LAM scales are similarly suspect.

S1

p1

S2

p2

P Psychophysical process

Response output process

Integration process

Response for Combination Judgment

R

(High)

(Med.)

Levels of Stimulus or Variable 2

(Low)

(Low)

(Med.)

(High)

Levels of Stimulus or Variable 1

Fig. 7.6 The functional measurement scheme of Anderson (1974).

Scaling

Others have found support for the validity of magnitude estimation in studies of binaural loudness summation (Marks, 1978). This argument continues and is difficult to resolve. A review of the matter was published by Gescheider (1988). For the purposes of sensory evaluation, the issue is not terribly important for two reasons. First, any scale that produces statistically significant differentiation of products is a useful scale. Second, the physical ranges over which category scaling and magnitude estimation produce different results is usually quite large in any psychophysical study. In most product tests, the differences are much more subtle and generally do not span such a wide dynamic range. The issue dissolves from any practical perspective.

7.8 Conclusions Much sound and fury has been generated over the years in the psychophysical literature concerning what methods yield valid scales. For the sensory practitioner, these issues are less relevant because the scale values do not generally have any absolute meaning. They are only convenient indices of the relative intensities or appeal of different products. The degree of difference may be a useful piece of information, but often we are simply interested in which product is stronger or weaker in some attribute, and whether the difference is both statistically significant and practically meaningful. Scaling provides a quick and useful way to get intensity or liking information. In the case of descriptive analysis, scaling allows collection of quantitative data on multiple attributes. The degree of variability or noise in the system is, to a large part, determined by whether the panelists have a common frame of reference. Thus reference standards for both the attribute terms and for the intensity anchors are useful. Of course, with consumer evaluations or a psychophysical study such calibration is not possible and usually not desired. The variability of consumer responses should offer a note of caution in the interptation of consumer scaling data. Students and sensory practitioners should examine their scaling methods with a critical eye. Not every task that assigns numbers will have useful scale properties like equal intervals. Bad examples abound

Appendix 1

in the commodity grading (quality scoring) literature (Pangborn and Dunkley, 1964). For example, different numbers may be assigned to different levels of oxidation, but that is scoring a physical condition based on inferences from sensory experience. It is not a report of the intensity of some experience itself. It is not tracking changes along a single perceptual continuum in the psychophysical sense. Scoring is not scaling. All hedonic scales seem to measure what they are intended to measure rather effectively, as long as no gross mistakes are made (Peryam, 1989, p. 23).

Appendix 1: Derivation of Thurstonian-Scale Values for the 9-Point Scale The choice of adjective words for the 9-point hedonic scale is a good example of how carefully a scale can be constructed. The long-standing track record of this tool demonstrates its utility and wide applicability in consumer testing. However, few sensory practitioners actually know how the adjectives were found and what criteria were brought to bear in selecting these descriptors (slightly, moderately, very much, and extremely like/dislike) from a larger pool of possible words. The goal of this section is to provide a shorthand description of the criteria and mathematical method used to select the words for this scale. One concern was the degree to which the term had consensual meaning in the population. The most serious concern was when a candidate word had an ambiguous or double meaning across the population. For example, the word “average” suggests an intermediate response to some people, but in the original study by Jones and Thurstone (1955) there were a group of people who equated it with “like moderately” perhaps since an average product in those days was one that people would like. These days, one can think of negative connotations to the word “average” as in “he was only an average student.” Other ambiguous or bimodal terms were “like not so much” and “like not so well.” Ideally, a term should have low variability in meaning, i.e., a low standard deviation, no bimodality, and little skew. Part of this concern with the normality of the distribution of psychological reactions to a word was the fact that the developers used Thurstone’s model

171

172

7

that marks the ordinate in equal standard deviations units according to the cumulative normal distribution. Either of these methods will tend to make the cumulative S-shaped curve for the item into a straight line. The X-axis value for each point is the “category z-value” for that bucket. 6. Fit a line to the data and interpolate the 50% point on the X-axis (the re-scaled category boundary estimates). These interpolated values for the median for each item now form the new scale values for the items. An example of this interpolation is shown in Fig. 7.7. Three of the phrases used in the original scaling study of Jones and Thurstone (1955) are pictured, three that were not actually chosen but for which we have approximate proportions and z-scores from their ps. The small vertical arrows on the X-axis show the scale values for the original categories of −4 to +3 (+4 has cumulative proportion of 100% and thus the z-score is infinite). Table 7.1 gives the values and

Scaling

proportions for each phrase and the original categories. The dashed vertical lines dropped from the intersection at the zero z-score (50% point) show the approximate mean values interpolated on the X-axis (i.e., about -1.1 for “do not care for it” and about +2.1 for “pferred.”). Note that “pferred” and “don’t care for it” have a linear fit and steep slope, suggesting a normal distribution and low standard deviation. In contrast, “highly unfavorable” has a lower slope and some curvilinearity, indicative of higher variability, skew, and/or pockets of disagreement about the connotation of this term. The actual scale values for the original adjectives are shown in Table 7.2, as found with a soldier population circa 1950 (Jones et al., 1955). You may note that the words are not equally spaced, and that the “slightly” values are closer to the neutral point than some of the other intervals, and the extreme points are a little farther out. This bears a good deal of similarity to the intervals found with the LAM scale as shown in the column where the LAM values are re-scaled to the same range as the 9-point Thurstonian Values.

Appendix 2: Construction of Labeled Magnitude Scales “

Fig. 7.7 An illustration of the method used to establish spacings and scale values for the 9-point hedonic scale using Thurstonian theory. Arrows on the X-axis show the scale points for the z-scores based on the complete distribution of the original -4 to +4 ratings. The Y-axis shows the actual z-scores based on the proportion of respondents using that category for each specific term. Re-plotted from data provided in Jones et al. (1955).

There are two primary methods for constructing labeled magnitude scales and they are very similar. Both require magnitude estimates from the participants to scale the word phrases used on the lines. In one case, just the word phrases are scaled, and in the second method, the word phrases are scaled among a list of common everyday experiences or sensations that most people are familiar with. The values obtained by the simple scaling of just the words will depend upon the words that are chosen, and extremely high examples (e.g., greatest imaginable liking for any experience) will tend to compss the values of the interior phrases (Cardello et al., 2008). Whether this kind of context effect will occur for the more general method of scaling amongst common experiences is not known. But the use of a broad frame of reference could be a stabilizing factor. Here is an example of the instructions given to subjects in construction of a labeled affective magnitude scale. Note that for hedonics, which are a bipolar continuum with a neutral point, it is necessary to collect a tone or valence (plus or minus) value as well as the overall “intensity” rating.

Appendix 2

173

Table 7.1 Examples of scaled phrases used in Fig. 7.7 Original “Preferred” category Proportion Z-score Proportion 4 3 2 1 0 −1 −2 −3 −4

1.000 0.999 0.983 0.882 0.616 0.383 0.185 0.068 0.008

Table 7.2 Actual 9-point scale phrase values and comparison to the LAM values

(undef.) 3.0 2.1 1.2 0.3 −0.3 −0.9 −1.5 −2.4

0.80 0.50 0.20 0.07 0.03

Z-score 0.84 0.00 −0.84 −1.48 −1.88

Descriptor

Scale value (9-point)

Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely

4.16 2.91 1.12 0.69 0.00 −0.59 −1.20 −2.49 −4.32

Next to each word label a response area appeared similar to this: Phrase: Like extremely

Words or phrases are psented in random order. After reading a word they must decide whether the word is positive, negative or neutral and place the corresponding symbol on the first line. If the hedonic tone was not a neutral one (zero value), they are instructed to give a numerical estimate using modulus-free magnitude estimation. The following is a sample of the instructions taken from Cardello et al. (2008): After having determined whether the phrase is positive or negative or neutral and writing the appropriate symbol (+, −, 0) on the first line, you will then assess the strength or magnitude of the liking or disliking reflected by the phrase. You will do this by placing a number on the second blank line (under “How Much”). For the first phrase that you rate, you can write any number you want on the line. We suggest you do not use a small number for this word/phrase. The reason for this is that subsequent words/phrases may reflect much lower levels of liking or disliking. Aside from this restriction you can use any numbers you want. For each subsequent

“Do not care for it” Proportion Z-score

“Highly unfavorable” Proportion Z-score

0.96 0.83 0.55 0.30 0.14

1.75 0.95 0.13 −0.52 −1.08

0.96 0.93 0.92 0.90 0.86 0.84 0.82 0.46

Interval

LAM value

LAM rescaled

Interval

1.26 1.79 0.43 0.69 0.59 0.61 1.29 1.83

74.2 56.1 36.2 11.2 0.0 −10.6 −31.9 −55.5 −75.5

4.20 3.18 2.05 0.63 0.00 −0.60 −1.81 −3.14 −4.28

1.02 1.13 1.52 0.63 0.60 1.21 1.33 1.14

1.75 1.48 1.41 1.28 1.08 0.99 0.92 −0.10

word/phrase your numerical judgment should be made proportionally and in comparison to the first number. That is, if you assigned the number 800 to index the strength of the liking/disliking denoted by the first word/phrase and the strength of liking/disliking denoted by the second word/phrase were twice as great, you would assign the number 1,600. If it were three times as great you would assign the number 2,400, etc. Similarly, if the second word/phrase denoted only 1/10 the magnitude of liking as the first, you would assign it the number 80 and so forth. If any word/phrase is judged to be “neutral” (zero (0) on the first line) it should also be given a zero for its magnitude rating.

In the cased of Cardello et al. (2008), positive and negative word labels were analyzed separately. Raw magnitude estimates were equalized for scale range using the procedure of Lane et al. (1961). All positive and negative magnitude estimates for a given subject were multiplied by an inpidual scaling factor. This factor was equal to the ratio of the grand geometric mean (of the absolute value of all nonzero ratings) across all subjects pided by the geometric mean for that subject. The geometric mean magnitude estimates for each phrase were then calculated based on this range-equated data. These means became the distance

174

from the zero point for placement of the phrases along the scale, usually accompanied by a short cross-hatch mark at that point.

7

Scaling

Basker, D. 1988. Critical values of differences among rank sums for multiple comparisons. Food Technology, 42(2), 79, 80-84. Baten, W. D. 1946. Organoleptic tests pertaining to apples and pears. Food Research, 11, 84-94. Bendig, A. W. and Hughes, J. B. 1953. Effect of number of verbal anchoring and number of rating scale categories upon transmitted information. Journal of Experimental Psychology, 46(2), 87-90. Bi, J. 2006. Sensory Discrimination Tests and Measurement. Blackwell, Ames, IA. Birch, L. L., Zimmerman, S. I. and Hind, H. 1980. The influence of social-affective context on the formation of children’s food pferences. Child Development, 51, 865-861. Birch, L. L., Birch, D., Marlin, D. W. and Kramer, L. 1982. Effects of instrumental consumption on children’s food pferences. Appetite, 3, 125-143. Birnbaum, M. H. 1982. Problems with so-called “direct” scaling. In: J. T. Kuznicki, R. A. Johnson and A. F. Rutkiewic (eds.), Selected Sensory Methods: Problems and Approaches to Hedonics. American Society for Testing and Materials, Philadelphia, pp. 34-48. Borg, G. 1982. A category scale with ratio properties for intermodal and interinpidual comparisons. In: H.-G. Geissler and P. Pextod (Eds.), Psychophysical Judgment and the Process of Perception. VEB Deutscher Verlag der Wissenschaften, Berlin, pp. 25-34. Borg, G. 1990. Psychophysical scaling with applications in physical work and the perception of exertion. Scandinavian Journal of Work and Environmental Health, 16, 55-58. Boring, E. G. 1942. Sensation and Perception in the History of Experimental Psychology. Appleton-Century-Crofts, New York. Brandt, M. A., Skinner, E. Z. and Coleman, J. A. 1963. The texture profile method. Journal of Food Science, 28, 404-409. Butler, G., Poste, L. M., Wolynetz, M. S., Agar, V. E. and Larmond, E. 1987. Alternative analyses of magnitude estimation data. Journal of Sensory Studies, 2, 243-257. Cardello, A. V. and Schutz, H. G. 2004. Research note. Numerical scale-point locations for constructing the LAM (Labeled affective magnitude) scale. Journal of Sensory Studies, 19, 341-346. Cardello, A. V., Lawless, H. T. and Schutz, H. G. 2008. Effects of extreme anchors and interior label spacing on labeled magnitude scales. Food Quality and Preference, 21, 323-334. Cardello, A. V., Winterhaler, C. and Schutz, H. G. 2003. Predicting the handle and comfort of military clothing fabrics from sensory and instrumental data: Development and application of new psychophysical methods. Textile Research Journal, 73, 221-237. Cardello, A. V., Schutz, H. G., Lesher, L. L. and Merrill, E. 2005. Development and testing of a labeled magnitude scale of perceived satiety. Appetite, 44, 1-13. Caul, J. F. 1957. The profile method of flavor analysis. Advances in Food Research, 7, 1-40. Chambers, E. C. and Wolf, M. B. 1996. Sensory Testing Methods. ASTM Manual Series, MNL 26. ASTM International, West Conshohocken, PA. Chen, A. W., Resurreccion, A. V. A. and Paguio, L. P. 1996. Age appropriate hedonic scales to measure the food pferences of young children. Journal of Sensory Studies, 11, 141-163.

References Chung, S.-J. and Vickers, 2007a. Long-term acceptability and choice of teas differing in sweetness. Food Quality and Preference 18, 963-974. Chung, S.-J. and Vickers, 2007b. Influence of sweetness on the sensory-specific satiety and long-term acceptability of tea. Food Quality and Preference, 18, 256-267. Coetzee, H. and Taylor, J. R. N. 1996. The use and adaptation of the paired comparison method in the sensory evaluation of hamburger-type patties by illiterate/semi-literate consumers. Food Quality and Preference, 7, 81-85. Collins, A. A. and Gescheider, G. A. 1989. The measurement of loudness in inpidual children and adults by absolute magnitude estimation and cross modality matching. Journal of the Acoustical Society of America, 85, 2012-2021. Conner, M. T. and Booth, D. A. 1988. Preferred sweetness of a lime drink and pference for sweet over non-sweet foods. Related to sex and reported age and body weight. Appetite, 10, 25-35. Cordinnier, S. M. and Delwiche, J. F. 2008. An alternative method for assessing liking: Positional relative rating versus the 9-point hedonic scale. Journal of Sensory Studies, 23, 284-292. Cox, E. P. 1980. The optimal number of response alternatives for a scale: A review. Journal of Marketing Research, 18, 407-422. Curtis, D. W., Attneave, F. and Harrington, T. L. 1968. A test of a two-stage model of magnitude estimation. Perception and Psychophysics, 3, 25-31. Edwards, A. L. 1952. The scaling of stimuli by the method of successive intervals. Journal of Applied Psychology, 36, 118-122. Ekman, G. 1964. Is the power law a special case of Fechner’s law? Perceptual and Motor Skills, 19, 730. Einstein, M. A. 1976. Use of linear rating scales for the evaluation of beer flavor by consumers. Journal of Food Science, 41, 383-385. El Dine, A. N. and Olabi, A. 2009. Effect of reference foods in repeated acceptability tests: Testing familiar and novel foods using 2 acceptability scales. Journal of Food Science, 74, S97-S105. Engen, T. 1974. Method and theory in the study of odor pferences. In: A. Turk, J. W. Johnson and D. G. Moulton (Eds.), Human Responses to Environmental Odors. Academic, New York. Finn, A. and Louviere, J. J. 1992. Determining the appropriate response to evidence of public concern: The case of food safety. Journal of Public Policy and Marketing, 11, 12-25. Forde, C. G. and Delahunty, C. M. 2004. Understanding the role cross-modal sensory interactions play in food acceptability in younger and older consumers. Food Quality and Preference, 15, 715-727. Frijters, J. E. R., Kooistra, A. and Vereijken, P. F. G. 1980. Tables of d’ for the triangular method and the 3-AFC signal detection procedure. Perception and Psychophysics, 27, 176-178. Gaito, J. 1980. Measurement scales and statistics: Resurgence of an old misconception. Psychological Bulletin, 87, 564-587. Gay, C., and Mead, R. 1992 A statistical appraisal of the problem of sensory measurement. Journal of Sensory Studies, 7, 205-228.

175 Gent, J. F. and Bartoshuk, L. M. 1983. Sweetness of sucrose, neohesperidin dihydrochalcone and sacchar in is related to genetic ability to taste the bitter substance 6-npropylthiouracil. Chemical Senses, 7, 265-272. Gescheider, G. A. 1988. Psychophysical scaling. Annual Review of Psychology, 39, 169-200. Giovanni, M. E. and Pangborn, R. M. 1983. Measurement of taste intensity and degree of liking of beverages by graphic scaling and magnitude estimation. Journal of Food Science, 48, 1175-1182. Gracely, R. H., McGrath, P. and Dubner, R. 1978a. Ratio scales of sensory and affective verbal-pain descriptors. Pain, 5, 5-18. Gracely, R. H., McGrath, P. and Dubner, R. 1978b. Validity and sensitivity of ratio scales of sensory and affective verbal-pain descriptors: Manipulation of affect by Diazepam. Pain, 5, 19-29. Green, B. G., Shaffer, G. S. and Gilmore, M. M. 1993. Derivation and evaluation of a semantic scale of oral sensation magnitude with apparent ratio properties. Chemical Senses, 18, 683-702. Green, B. G., Dalton, P., Cowart, B., Shaffer, G., Rankin, K. and Higgins, J. 1996. Evaluating the “Labeled Magnitude Scale” for measuring sensations of taste and smell. Chemical Senses, 21, 323-334. Greene, J. L., Bratka, K. J., Drake, M. A. and Sanders, T. H. 2006. Effective of category and line scales to characterize consumer perception of fruity fermented flavors in peanuts. Journal of Sensory Studies, 21, 146-154. Guest, S., Essick, G., Patel, A., Prajpati, R. and McGlone, F. 2007. Labeled magnitude scales for oral sensations of wetness, dryness, pleasantness and unpleasantness. Food Quality and Preference, 18, 342-352. Hein, K. A., Jaeger, S. R., Carr, B. T. and Delahunty, C. M. 2008. Comparison of five common acceptance and pference methods. Food Quality and Preference, 19, 651-661. Huskisson, E. C. 1983. Visual analogue scales. In: R. Melzack (Ed.), Pain Measurement and Assessment. Raven, New York, pp. 34-37. Jaeger, S. R.; Jørgensen, A. S., AAslying, M. D. and Bredie, W. L. P. 2008. Best-worst scaling: An introduction and initial comparison with monadic rating for pference elicitation with food products. Food Quality and Preference, 19, 579-588. Jaeger, S. R. and Cardello, A. V. 2009. Direct and indirect hedonic scaling methods: A comparison of the labeled affective magnitude (LAM) scale and best-worst scaling. Food Quality and Preference, 20, 249-258. Jones, F. N. 1974. History of psychophysics and judgment. In: E. C. Carterette and M. P. Friedman (Eds.), Handbook of Perception. Psychophysical Judgment and Measurement, Vol. 2. Academic, New York, pp. 11-22. Jones, L. V. and Thurstone, L. L. 1955. The psychophysics of semantics: An experimental investigation. Journal of Applied Psychology, 39, 31-36. Jones, L. V., Peryam, D. R. and Thurstone, L. L. 1955. Development of a scale for measuring soldier’s food pferences. Food Research, 20, 512-520. Keskitalo, K. Knaapila, A., Kallela, M., Palotie, A., Wessman, M., Sammalisto, S., Peltonen, L., Tuorila, H. and Perola, M. 2007. Sweet taste pference are partly genetically

176 determined: Identification of a trait locus on Chromosome 161-3 . American Journal of Clinical Nutrition, 86, 55-63. Kim, K.-O. and O’Mahony, M. 1998. A new approach to category scales of intensity I: Traditional versus rank-rating. Journal of Sensory Studies, 13, 241-249. King, B. M. 1986. Odor intensity measured by an audio method. Journal of Food Science, 51, 1340-1344. Koo, T.-Y., Kim, K.-O., and O’Mahony, M. 2002. Effects of forgetting on performance on various intensity scaling protocols: Magnitude estimation and labeled magnitude scale (Green scale). Journal of Sensory Studies, 17, 177-192. Kroll, B. J. 1990. Evaluating rating scales for sensory testing with children. Food Technology, 44(11), 78-80, 82, 84, 86. Kurtz, D. B., White, T. L. and Hayes, M. 2000. The labeled dissimilarity scale: A metric of perceptual dissimilarity. Perception and Psychophysics, 62, 152-161. Land, D. G. and Shepard, R. 1984. Scaling and ranking methods. In: J. R. Piggott (ed.), Sensory Analysis of Foods. Elsevier Applied Science, London, pp. 141-177. Lane, H. L., Catania, A. C. and Stevens, S. S. 1961. Voice level: Autophonic scale, perceived loudness and effect of side tone. Journal of the Acoustical Society of America, 33, 160-167. Larson-Powers, N. and Pangborn, R. M. 1978. Descriptive analysis of the sensory properties of beverages and gelatins containing sucrose or synthetic sweeteners. Journal of Food Science, 43, 47-51. Lawless, H. T. 1977. The pleasantness of mixtures in taste and olfaction. Sensory Processes, 1, 227-237. Lawless, H. T. 1989. Logarithmic transformation of magnitude estimation data and comparisons of scaling methods. Journal of Sensory Studies, 4, 75-86. Lawless, H. T. and Clark, C. C. 1992. Psychological biases in time intensity scaling. Food Technology, 46, 81, 84-86, 90. Lawless, H. T. and Malone, J. G. 1986a. The discriminative efficiency of common scaling methods. Journal of Sensory Studies, 1, 85-96. Lawless, H. T. and Malone, G. J. 1986b. A comparison of scaling methods: Sensitivity, replicates and relative measurement. Journal of Sensory Studies, 1, 155-174. Lawless, H. T. and Skinner, E. Z. 1979. The duration and perceived intensity of sucrose taste. Perception and Psychophysics, 25, 249-258. Lawless, H. T., Popper, R. and Kroll, B. J. 2010a. Comparison of the labeled affective magnitude (LAM) scale, an 11-point category scale and the traditional nine-point hedonic scale. Food Quality and Preference, 21, 4-12. Lawless, H. T., Sinopoli, D. and Chapman, K. W. 2010b. A comparison of the labeled affective magnitude scale and the nine point hedonic scale and examination of categorical behavior. Journal of Sensory Studies, 25, S1, 54-66. Lawless, H. T., Cardello, A. V., Chapman, K. W., Lesher, L. L., Given, Z. and Schutz, H. G. 2010c. A comparison of the effectiveness of hedonic scales and end-anchor compssion effects. Journal of Sensory Studies, 28, S1, 18-34. Lee, H.-J., Kim, K.-O., and O’Mahony, M. 2001. Effects of forgetting on various protocols for category and line scales of intensity. Journal of Sensory Studies, 327-342. Likert, R. 1932. Technique for the measurement of attitudes. Archives of Psychology, 140, 1-55.

7

Scaling

Lindvall, T. and Svensson, L. T. 1974. Equal unpleasantness matching of malodourous substances in the community. Journal of Applied Psychology, 59, 264-269. Mahoney, C. H., Stier, H. L. and Crosby, E. A. 1957. Evaluating flavor differences in canned foods. II. Fundamentals of the simplified procedure. Food Technology 11, Supplemental Symposium Proceedings, 37-42. Marks, L. E. 1978. Binaural summation of the loudness of pure tones. Journal of the Acoustical Society of America, 64, 107-113. Marks, L. E., Borg, G. and Ljunggren, G. 1983. Inpidual differences in perceived exertion assessed by two new methods. Perception and Psychophysic, 34, 280-288. Marks, L. E., Borg, G. and Westerlund, J. 1992. Differences in taste perception assessed by magnitude matching and by category-ratio scaling. Chemical Senses, 17, 493-506. Mattes, R. D. and Lawless, H. T. 1985. An adjustment error in optimization of taste intensity. Appetite, 6, 103-114. McBride, R. L. 1983a. A JND-scale/category scale convergence in taste. Perception and Psychophysics, 34, 77-83. McBride, R. L. 1983b. Taste intensity and the case of exponents greater than 1. Australian Journal of Psychology, 35, 175-184. McBurney, D. H. and Shick, T. R. 1971. Taste and water taste for 26 compounds in man. Perception and Psychophysics, 10, 249-252. McBurney, D. H. and Bartoshuk, L. M. 1973. Interactions between stimuli with different taste qualities. Physiology and Behavior, 10, 1101-1106. McBurney, D. H., Smith, D. V. and Shick, T. R. 1972. Gustatory cross-adaptation: Sourness and bitterness. Perception and Psychophysics, 11, 228-232. Mead, R. and Gay, C. 1995. Sequential design of sensory trials. Food Quality and Preference, 6, 271-280. Mecredy, J. M. Sonnemann, J. C. and Lehmann, S. J. 1974. Sensory profiling of beer by a modified QDA method. Food Technology, 28, 36-41. Meilgaard, M., Civille, G. V. and Carr, B. T. 2006. Sensory Evaluation Techniques, Fourth Edition. CRC, Boca Raton, FL. Moore, L. J. and Shoemaker, C. F. 1981. Sensory textural properties of stabilized ice cream. Journal of Food Science, 46, 399-402. Moskowitz, H. R. 1971. The sweetness and pleasantness of sugars. American Journal of Psychology, 84, 387-405. Moskowitz, H. R. and Sidel, J. L. 1971. Magnitude and hedonic scales of food acceptability. Journal of Food Science, 36, 677-680. Muñoz, A. M. and Civille, G. V. 1998. Universal, product and attribute specific scaling and the development of common lexicons in descriptive analysis. Journal of Sensory Studies, 13, 57-75. Newell, G. J. and MacFarlane, J. D. 1987. Expanded tables for multiple comparison procedures in the analysis of ranked data. Journal of Food Science, 52, 1721-1725. Olabi, A. and Lawless, H. T. 2008. Persistence of context effects with training and reference standards. Journal of Food Science, 73, S185-S189. O’Mahony, M., Park, H., Park, J. Y. and Kim, K.-O. 2004. Comparison of the statistical analysis of hedonic data using

References analysis of variance and multiple comparisons versus and R-index analysis of the ranked data. Journal of Sensory Studies, 19, 519-529. Pangborn, R. M. and Dunkley, W. L. 1964. Laboratory procedures for evaluating the sensory properties of milk. Dairy Science Abstracts, 26-55-62. Parducci, A. 1965. Category judgment: A range-frequency model. Psychological Review, 72, 407-418. Park, J.-Y., Jeon, S.-Y., O’Mahony, M. and Kim, K.-O. 2004. Induction of scaling errors. Journal of Sensory Studies, 19, 261-271. Pearce, J. H., Korth, B. and Warren, C. B. 1986. Evaluation of three scaling methods for hedonics. Journal of Sensory Studies, 1, 27-46. Peryam. D. 1989. Reflections. In: Sensory Evaluation. In Celebration of our Beginnings. American Society for Testing and Materials, Philadelphia, pp. 21-30. Peryam, D. R. and Girardot, N. F. 1952. Advanced taste-test method. Food Engineering, 24, 58-61, 194. Piggot, J. R. and Harper, R. 1975. Ratio scales and category scales for odour intensity. Chemical Senses and Flavour, 1, 307-316. Pokor´ny, J., Davídek, J., Prnka, V. and Davídková, E. 1986. Nonparametric evaluation of graphical sensory profiles for the analysis of carbonated beverages. Die Nahrung, 30, 131-139. Poulton, E. C. 1989. Bias in Quantifying Judgments. Lawrence Erlbaum, Hillsdale, NJ. Richardson, L. F. and Ross, J. S. 1930. Loudness and telephone current. Journal of General Psychology, 3, 288-306. Rosenthal, R. 1987. Judgment Studies: Design, Analysis and Meta-Analysis. University Press, Cambridge. Shand, P. J., Hawrysh, Z. J., Hardin, R. T. and Jeremiah, L. E. 1985. Descriptive sensory analysis of beef steaks by category scaling, line scaling and magnitude estimation. Journal of Food Science, 50, 495-499. Schutz, H. G. and Cardello, A. V. 2001.. A labeled affective magnitude (LAM) scale for assessing food liking/disliking. Journal of Sensory Studies, 16, 117-159. Siegel, S. 1956. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, New York. Sriwatanakul, K., Kelvie, W., Lasagna, L., Calimlim, J. F., Wels, O. F. and Mehta, G. 1983. Studies with different types of visual analog scales for measurement of pain. Clinical Pharmacology and Therapeutics, 34, 234-239. Stevens, J. C. and Marks, L. M. 1980. Cross-modality matching functions generated by magnitude estimation. Perception and Psychophysics, 27, 379-389. Stevens, S. S. 1951. Mathematics, measurement and psychophysics. In: S. S. Stevens (ed.), Handbook of Experimental Psychology. Wiley, New York, pp. 1-49.

177 Stevens, S. S. 1956. The direct estimation of sensory magnitudes-loudness. American Journal of Psychology, 69, 1-25. Stevens, S. S. 1957. On the psychophysical law. Psychological Review, 64, 153-181. Stevens, S. S. 1969. On pdicting exponents for cross-modality matches. Perception and Psychophysics, 6, 251-256. Stevens, S. S. and Galanter, E. H. 1957. Ratio scales and category scales for a dozen perceptual continua. Journal of Experimental Psychology, 54, 377-411. Stoer, N. L. and Lawless, H. T. 1993. Comparison of single product scaling and relative-to-reference scaling in sensory evaluation of dairy products. Journal of Sensory Studies, 8, 257-270. Stone, H., Sidel, J., Oliver, S., Woolsey, A. and Singleton, R. C. 1974. Sensory Evaluation by quantitative descriptive analysis. Food Technology, 28, 24-29, 32, 34. Teghtsoonian, M. 1980. Children’s scales of length and loudness: A developmental application of cross-modal matching. Journal of Experimental Child Psychology, 30, 290-307. Thurstone, L. L. 1927. A law of comparative judgment. Psychological Review, 34, 273-286. Townsend, J. T. and Ashby, F. G. 1980. Measurement scales and statistics: The misconception misconceived. Psychological Bulletin, 96, 394-401. Vickers, Z. M. 1983. Magnitude estimation vs. category scaling of the hedonic quality of food sounds. Journal of Food Science, 48, 1183-1186. Villanueva, N. D. M. and Da Silva, M. A. A. P. 2009. Performance of the nine-point hedonic, hybrid and selfadjusting scales in the generation of internal pference maps. Food Quality and Preference, 20, 1-12. Villanueva, N. D. M., Petenate, A. J., and Da Silva, M. A. A. P. 2005. Comparative performance of the hybrid hedonic scale as compared to the traditional hedonic, self-adjusting and ranking scales. Food Quality and Preference, 16, 691-703. Ward, L. M. 1986. Mixed-modality psychophysical scaling: Double cross-modality matching for “difficult” continua. Perception and Psychophysics, 39, 407-417. Weiss, D. J. 1972. Averaging: an empirical validity criterion for magnitude estimation. Perception and Psychophysics, 12, 385-388. Winakor, G., Kim, C. J. and Wolins, L. 1980. Fabric hand: Tactile sensory assessment. Textile Research Journal, 50, 601-610. Yamaguchi, S. 1967. The synergistic effect of monosodium glutamate and disodium 5 inosinate. Journal of Food Science 32, 473-477.

Chapter 8

Time-Intensity Methods

Contents 8.1 8.2 8.3

8.4

8.5

8.6

Introduction . . . . . . . . . . . . . . . . . A Brief History . . . . . . . . . . . . . . . Variations on the Method . . . . . . . . . . 8.3.1 Discrete or Discontinuous Sampling . . . 8.3.2 “Continuous” Tracking . . . . . . . . . 8.3.3 Temporal Dominance Techniques . . . . Recommended Procedures . . . . . . . . . . 8.4.1 Steps in Conducting a Time-intensity Study . . . . . . . . . . . . . . . . . 8.4.2 Procedures . . . . . . . . . . . . . . 8.4.3 Recommended Analysis . . . . . . . . Data Analysis Options . . . . . . . . . . . . 8.5.1 General Approaches . . . . . . . . . . 8.5.2 Methods to Construct or Describe Average Curves . . . . . . . . . . . . . . . . 8.5.3 Case Study: Simple Geometric Description . . . . . . . . . . . . . . 8.5.4 Analysis by Principal Components . . . Examples and Applications . . . . . . . . . . 8.6.1 Taste and Flavor Sensation Tracking . . . 8.6.2 Trigeminal and Chemical/Tactile Sensations . . . . . . . . . . . . . . 8.6.3 Taste and Odor Adaptation . . . . . . .

179 180 182 182 183 184 185 185 186 186 187 187 188 189 192 193 193 194 194

8.6.4 Texture and Phase Change . . 8.6.5 Flavor Release . . . . . . . 8.6.6 Temporal Aspects of Hedonics 8.7 Issues . . . . . . . . . . . . . . . 8.8 Conclusions . . . . . . . . . . . . References . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

195 195 196 197 198 198

8.1 Introduction Perception of aroma, taste, flavor, and texture in foods is a dynamic not a static phenomenon. In other words, the perceived intensity of the sensory attributes change from moment to moment. The dynamic nature of food sensations arises from processes of chewing, breathing, salivation, tongue movements, and swallowing (Dijksterhuis, 1996). In the texture profile method for instance, different phases of food breakdown were recognized early on as evidenced by the separation of characteristics into first bite, mastication, and residual phases (Brandt et al., 1963). Wine tasters often discuss

H.T. Lawless, H. Heymann, Sensory Evaluation of Food, Food Science Text Series, DOI 10.1007/978-1-4419-6488-5_8, © Springer Science+Business Media, LLC 2010

179

180

how a wine “opens in the glass,” recognizing that the flavor will vary as a function of time after opening the bottle and exposing the wine to air. It is widely believed that the consumer acceptability of different intensive sweeteners depends on the similarity of their time profile to that of sucrose. Intensive sweeteners with too long a duration in the mouth may be less pleasant to consumers. Conversely, a chewing gum with longlasting flavor or a wine with a “long finish” may be desirable. These examples demonstrate how the time profile of a food or beverage can be an important aspect of its sensory appeal. The common methods of sensory scaling ask the panelists to rate the perceived intensity of the sensation by giving a single (uni-point) rating. This task requires that the panelists must “time-average” or integrate any changing sensations or to estimate only the peak intensity in order to provide the single intensity value that is required. Such a single value may miss some important information. It is possible, for example, for two products to have the same or similar time-averaged profiles or descriptive specifications, but differ in the order in which different flavors occur or when they reach their peak intensities. Time-intensity (TI) methods provide panelists with the opportunity to scale their perceived sensations over time. When multiple attributes are tracked, the profile of a complex food flavor or texture may show differences between products that change across time after a product is first tasted, smelled, or felt. For most sensations the perceived intensity increases and eventually decreases but for some, like perceived toughness of meat, the sensations may only decrease as a function of time. For perceived melting, the sensation may only increase until a completely melted state is reached. When performing a TI study the sensory specialist can obtain a wealth of detailed information such as the following for each sample: the maximum intensity perceived, the time to maximum intensity, the rate and shape of the increase in intensity to the maximum point, the rate and shape of the decrease in intensity to half-maximal intensity and to the extinction point, and the total duration of the sensation. Some of the common time-intensity parameters are illustrated in Fig. 8.1. The additional information derived from time-intensity methods is especially useful when studying sweeteners or products like chewing gums and hand lotions that have a distinctive time profile.

8

Time-Intensity Methods

INTENSITY Other Parameters: DUR, Total duration = Tend – Tstart AUC, Area under the curve

TIME

Fig. 8.1 Example of a time-intensity curve and common curve parameters extracted from the record.

The remainder of this chapter will be devoted to an overview of the history and applications of this method, as well as recommended procedures and analyses. For the student who wants only the basic information, the following sections are key: variations on the method (Section 8.3), steps and recommended procedures (Section 8.4), data analysis options (Section 8.5), and conclusions (8.8).

8.2 A Brief History Holway and Hurvich (1937) published an early report of tracking taste intensity over time. They had their subjects trace a curve to repsent the sensations from a drop of either 0.5 or 1.0 M NaCl placed on the anterior tongue surface over 10 s. They noted several general effects that were later confirmed as common trends in other studies. The higher concentration led to a higher peak intensity, but the peak occurred later, in spite of a steeper rising slope. Most importantly they noted that taste intensity was not strictly a function of concentration: “while the concentration originally placed on the tongue is ‘fixed,’ the intensity varies in a definite manner from moment to moment. Saline intensity depends on time as well as concentration.” A review of studies of temporal factors in taste is found in Halpern (1991). A review of TI studies of the 1980s and early 1990s can be found in Cliff and Heymann (1993b). Sjostrom (1954) and Jellinek (1964) also made early attempts to quantify the temporal response to perceived sensory intensities. These authors asked their panelists to indicate their perceived bitterness of beer at 1 s intervals on a ballot, using a clock to indicate time.

8.2 A Brief History

They then constructed TI curves by plotting the x-y coordinates (time on the x-axis and perceived intensity on the y-axis) on graph paper. Once panelists had some experience with the method it was possible to ask them simultaneously to rate the perceived intensities of two different attributes at 1 s intervals. Neilson (1957) in an attempt to make the production of the TI curves easier, asked panelists to indicate perceived bitterness directly on graph paper at 2 s timed intervals. The clock could be distracting to the panelists and thus Meiselman (1968), studying taste adaptation and McNulty and Moskowitz (1974), evaluating oilin-water emulsions, improved the TI methodology by eliminating the clock. These authors used audible cues to tell the panelists when to enter perceived intensities on a ballot, placing the timekeeping demands on the experimenter rather than the participant. Larson-Powers and Pangborn (1978), in another attempt to eliminate the distractions of the clock or audible cues, employed a moving strip-chart recorder equipped with a foot pedal to start and stop the movement of the chart. Panelists recorded their responses to the perceived sweetness in beverages and gelatins sweetened with sucrose or synthetic sweeteners, by moving a pen along the cutter bar of the strip-chart recorder. The cutter bar was labeled with an unstructured line scale, from none to extreme. A cardboard cover was placed over the moving chart paper to pvent the panelists from watching the evolving curves and thus pventing them from using any visual cues to bias their responses. A similar setup was independently developed and used at the General Foods Technical Center in 1977 to track sweetness intensity (Lawless and Skinner, 1979). In this apparatus, the actual pen carriage of the chart recorder was grasped by the subject, eliminating the need for them to position a pen; also the moving chart was obscured by a line scale with a pointer attached to the pen carriage. In yet another laboratory at about the same time, Birch and Munton (1981) developed the “SMURF” version of TI scaling (short for “Sensory Measuring Unit for Recording Flux”). In the SMURF apparatus, the subject turned a knob graded from 1 to 10, and this potentiometer fed a variable signal to a strip-chart recorder out of sight of the panelist. The use of strip-chart recorders provided the first continuous TI data-collection methods and freed the panelists from any distractions caused by a clock or auditory signal. However, the methods required a fair degree of mental and physical

181

coordination by the participants. For example, in the Larson-Powers setup, the strip-chart recorder required the panelists to use a foot pedal to run the chart recorder, to place the sample in the mouth, and to move the pen to indicate the perceived intensity. Not all panelists were suitably coordinated and some could not do the evaluation. Although the strip-chart recorder made continuous evaluation of perceived intensities possible, the TI curves had to be digitized manually, which was extremely time consuming. The opportunity to use computers to time sample an analog voltage signal quite naturally led to online data collection to escape the problem of manual measurement of TI curves. To the best of our knowledge, the first computerized system was developed at the US Army Natick Food Laboratories in 1979 to measure bitter taste adaptation. It employed an electric sensor in the spout just above the subject’s tongue in order to determine the actual onset of stimulus arrival. Subthreshold amounts of NaCl were added to the stimulus and thus created a conductance change as the flow changed from the pliminary water rinse to the stimulus interface in the tube. A special circuit was designed to detect the conductance change and to connect the response knob to the visual line scale. Like the SMURF apparatus developed by Birch and Munton, the subject turned a knob controlling a variable resistor. The output of this potentiometer moved a pointer on a line scale for visual feedback while a parallel analog signal was sent to an analog-to-digital converter and then to the computer. The programming was done using FORTRAN subroutines on a popular lab computer of that era. The entire system is shown in Fig. 8.2. The appearance of desktop computers led to an explosion in the use of TI methodology in the 1980s and 1990s. A number of thesis research projects from U.C. Davis served as useful demonstrations of the method (Cliff, 1987; Dacanay, 1990; Rine, 1987) and the method was championed by Pangborn and coworkers (e.g., Lee and Pangborn, 1986). Several scientists (Barylko-Pikielna et al., 1990; Cliff, 1987; Guinard et al., 1985; Janusz et al., 1991; Lawless, 1980; Lee, 1985; Rine, 1987; Yoshida, 1986) developed computerized TI systems using a variety of hardware and software products. Computerized TI systems are now commercially available as part of data-collection software ensembles, greatly enhancing the ease and availability of TI data collection and data processing.

182

8

Time-Intensity Methods

Fig. 8.2 An early computerized system for time-intensity scaling used for tracking bitterness adaptation in a flow system for stimulating the anterior tongue. Heavy lines indicate the flow of stimulus solution, solid lines the flow of information, and dashed lines the experimenter-driven process control. Stimulus arrival at the tongue was monitored by conductivity sensors fitted into the glass tube just above the subject’s tongue. The subjects’

responses change on a line scale while the experimenter could view the conductivity and response voltage outputs on the computer’s display screen, which simultaneously were output to a data file. The system was programmed in FORTRAN subroutines controlling the clock sampling rate and analog-to-digital conversion routine. From Lawless and Clark (1992), reprinted with permission.

However, despite the availability of computerized systems, some research was still conducted using the simple time cueing at discrete intervals (e.g., Lee and Lawless, 1991; Pionnier et al., 2004), and the semi-manual strip-chart recorder method (Ott et al., 1991; Robichaud and Noble, 1990). A discussion of some common applications of TI methods is given in Section 8.6.

oldest approach is simply to ask the panelists to rate the intensity of sensation during different phases of consuming a food. This is particularly applicable to texture which may be evaluated in phases such as initial bite, first chew, mastication, and residual. An example of time pision during the texture evaluation of a baked product is shown in Table 8.1. When using a descriptive panel, it may be useful to have residual flavor or mouthfeel sensations rated at a few small intervals, e.g., every 30 s for 2 min, or immediately after tasting and then again after expectoration. For an example of this approach used with hot pepper “burn” see Stevens and Lawless (1986). Each measurement is then treated like a separate descriptive attribute and analyzed as a separate variable, with little or no attempt to reconstruct a time-connected record like the TI curve shown in Fig. 8.1. For researchers

8.3 Variations on the Method 8.3.1 Discrete or Discontinuous Sampling The sensory scientist has several options for collecting time-dependent sensory data. The methods for timerelated scaling can be pided into four groups. The

8.3 Variations on the Method

183

Table 8.1 Texture attributes at different phases of descriptive analysis Phase Attributes Surface

First bite

First chew Chew down

Residual

Roughness Particles Dryness Fracturability Hardness Particle size Denseness Uniformity of chew Moisture absorption Cohesiveness of mass Toothpacking Grittiness Oiliness Particles Chalky

interested in some simple aspect like “strength of bitter aftertaste” this method may suffice. Another related approach is to ask for repeated ratings of a single or just a few attributes at repeated smaller time intervals, usually cued by the panel leader or experimenter. These ratings are then connected and graphed on time axis. This is a simple procedure that can be used to track changes in the intensity of a flavor or texture attribute and requires no special equipment other than a stopwatch or other timing device. The panel must be trained to rate their sensations upon the time cue and to move rapidly through the list of attributes. The cue may be given verbally or on a computer screen. It is not known how many attributes can be rated in this way, but with faster time cueing and shorter intervals, obviously fewer attributes may be rated. This method also requires some faith in the assumption that the attributes are being rated close to the actual time when the cue is given. The accuracy with which panelists can do this is unknown, but given that there is a reaction time delay in any perceptual judgment, there must be some inherent error or delayrelated variance built into the procedure. An example of the repeated, discrete time interval method with verbal cueing and multiple attributes can be found in studies of sweetener mixtures (e.g., Ayya and Lawless, 1992) and astringency (Lee and Lawless, 1991). The time record is treated as a connected series and time is analyzed as one factor (i.e., one independent variable) in the statistical analysis.

Word anchors Smooth-rough None-many Oily-dry Crumbly-brittle Soft-hard Small-large Airy-dense Even-uneven None-much Loose-cohesive None-much None-much Dry-oily None-much Not chalky-very chalky

184

trained to move a mouse diagonally and a visual scale included pointers on both horizontal and vertical scales to repsent the intensity of the inpidual attributes. With a slowly changing product like chewing gum, with a sampling time that is not too frequent (every 9-15 s in this case), the technique would seem to be within the capabilities of human observers to either rapidly shift their attention or to respond to the overall pattern of the combined flavors. However, as currently used, most TI tracking methods must repeat the evaluation in order to track additional attributes. Ideally, this could lead to a composite profile of all the dynamic flavor and texture attributes in a product and how they changed across time. Such an approach was proposed by DeRovira (1996), who showed how the descriptive analysis spider-web plot of multiple attributes could be extended into the time dimension to produce a set of TI curves and thus to characterize an entire profile.

8

Time-Intensity Methods

8.4 Recommended Procedures

attributes are rated in each trial, (2) that it is simple and easy to do, requiring little or no training, and (3) that it provides a picture of enhanced differences relative to the TI records. Because panelists are forced to respond to only one attribute at a time, differences in the temporal profile may be accentuated. However, at the time of this publication, no standard procedure seemed to be agreed upon. The technique requires specialized software to collect the information, but at least one of the major sensory software systems has implemented a TDS option. Attributes are assumed to have a score of zero before they become dominant, and some attributes may never be rated. This would seem to necessarily lead to an incomplete record. Different panelists are contributing at different times to different attributes, so statistical methods for comparison of differences between products using the raw data set are difficult. However, products can be statistically compared using the simplified summary scores (summed intensity by duration measures, but losing time information) or by comparison of the proportion of responders (losing intensity information) as in Labbe et al. (2009). Qualitative comparisons can be made from inspecting the curves, such as “this product is initially sweet, then becoming more astringent, compared to product X which is initially sour, then fruity.” Is information provided by TDS and by traditional TI tracking methods different? One study found a strong resemblance of the constructed time curves for both methods, providing virtually redundant information for some attributes (Le Reverend et al., 2008). In another study, correlations with TI parameters were high for intensity maxima versus dominance proportion maxima, as might be expected since a higher number of persons finding an attribute dominant should be related to the mean intensity in TI. Correlations with other time-dependent parameters such as time to Imax (Tmax ) and duration measures were low, due to the different information collected in TDS and the limited attention to one attribute at a time (Pineau et al., 2009).

8.4 Recommended Procedures 8.4.1 Steps in Conducting a Time-intensity Study The steps in conducting a time-intensity study are similar to those in setting up a descriptive analysis procedure. They are listed in Table 8.2. The first

185 Table 8.2 Steps in conducting a time-intensity study 1. Determine project objectives: Is TI the right method? 2. Determine critical attributes to be rated. 3. Establish products to be used with clients/researchers. 4. Choose system and/or TI method for data collection. a. What is the response task? b. What visual feedback is provided to the panelists? 5. Establish statistical analysis and experimental design. a. What parameters are to be compared? b. Are multivariate comparisons needed? 6. Recruit panelists. 7. Conduct training sessions. 8. Check panelist performance. 9. Conduct study. 10. Analyze data and report.

important question is to establish whether TI methods are appropriate to the experimental research objective. Is this a product with just one or a few critical attributes that are likely to vary in some important way in their time course? Is this difference likely to impact consumer acceptance of the product? What are these critical attributes? Next, the product test set should be determined with the research team, and this will influence the experimental design. The TI method itself must be chosen, e.g., discrete point, continuous tracking, or TDS. The sensory specialist should by this time have an idea of what the data set will look like and what parameters can be extracted from the TI records for statistical comparisons such as intensity maxima, time to maxima, areas under the curve, and total duration. Many of the TI curve parameters are often correlated, so there is little need to analyze more than about ten parameters. Practice is almost always essential. You cannot assume that a person sitting down in a test booth will know what to do with the TI system and feel comfortable with the mouse or other response device. A protocol for training TI panelists was outlined by Peyvieux and Dijksterhuis (2001) and this protocol or similar versions have been widely adopted. It is also wise that some kind of panel checking be done to make sure the panelists are giving reliable data (see Bloom et al., 1995; Peyvieux and Dijksterhuis, 2001) and to examine the reasonableness of their data records. At this time the researchers and statistical staff should also decide how to handle missing data or records that may have artifacts or be incomplete. As in any sensory study, extensive planning may save a lot of headaches and problems, and this is especially true for TI methods.

186

8.4.2 Procedures If only a few attributes are going to be evaluated, the continuous tracking methods are appropriate and will provide a lot of information. This usually requires the use of computer-assisted data collection. Many, if not all, of the commercial software packages for sensory evaluation data collection have TI modules. The start and stop commands, sampling rate, and inter-trial intervals can usually be specified. The mouse movement will generally produce some visual feedback such as the motion of a cursor or line indicator along a simple line scale. The display often looks like a vertical or horizontal thermometer with the cursor position clearly indicated by bar or line that rises and falls. The computer record can be treated as raw data for averaging across panelists. However, statistical analysis by simple averaging raises a number of issues (discussed in Section 8.5). A simple approach is to pull characteristic curve parameters off each record for purposes of statistical comparisons such as intensity maximum (Imax ), time to maximum (Tmax ), and area under the curve (AUC). These are sometimes referred to as “scaffolding parameters” as they repsent the fundamental structure of the time records. Statistical comparison of these parameters can provide a clear understanding of how different products are perceived with regard to the onset of sensations, time course of rising and falling sensations, total duration, and total sensory impact of that flavor or textural aspect of the products. If a computer-assisted software package is not available or cannot be programmed, the research can always choose to use the cued/discontinuous method (e.g., with a stopwatch and verbal commands). This may be suitable for products in which multiple attributes must be rated in order to get the full picture. However, given the widespad availability of commercial sensory data-collection systems in major food and consumer product companies, it is likely that a sensory professional will have access to a continuous tracking option. The starting position of the cursor on the visible scale or computer screen should be considered carefully. For most intensity ratings it makes sense to start at the lower end, but for hedonics (like/dislike) the cursor should begin at the neutral point. For meat tenderness or product melting, the track is usually unidirectional, so the cursor should start at “not tender” or

8

Time-Intensity Methods

“tough” for meat and “not melted” for a product that melts. If the cursor is started at the wrong end of a unidirectional tracking situation, a falsely bi-directional record may be obtained due to the initial movement. An outline of panel training for TI studies was illustrated in a case study by Peyvieux and Dijksterhuis (2001) and the sensory specialist should consider using this or a similar method to insure good performance of the panelists. These authors had panelists evaluate flavor and texture components of a complex meat product. The prospective panelists were first introduced to the TI method, and then given practice with basic taste solutions over several sessions. The basic tastes were considered simpler than the complex product and more suitable for initial practice. Panelist consistency was checked and a panelist was considered reliable if they could produce two out of three TI records on the same taste stimulus that did not differ more than 40% of the time. A vertical line scale was used and an important specification was when to move the cursor back to zero (when there was no flavor or when the sample was swallowed for the texture attribute of juiciness). Problems were noted with (1) nontraditional curve shapes such as having no return to zero, (2) poor replication by some panelists, (3) unusable curves due to lack of a landmark such as no Imax . The authors also conducted a traditional profiling (i.e., descriptive analysis) study before the TI evaluations to make sure the attributes were correctly chosen and understood by the panelists. If TI panelists are already chosen from an existing descriptive panel, this step may not be needed. The authors conducted several statistical analyses to check for consistent use of the attributes, to look for oddities in curve shapes by some panelists, and to examine inpidual replicates. Improvements in consistency and evidence of learning and practice were noted.

8.4.3 Recommended Analysis For purposes of comparing products, the simplest approach is to extract the curve parameters such as Imax , Tmax , AUC, and total duration from each record. Some sensory software systems will generate these measures automatically. Then these curve parameters can be treated like any data points in any sensory evaluation and compared statistically. For three or

8.5 Data Analysis Options

more products, analysis would be by ANOVA and then planned comparisons of means (see Appendix C). Means and significance of differences can be reported in graphs or tables for each curve attribute and product. If a time by intensity curve is desired, the curves can be averaged in the time direction by choosing points at specific time intervals. This averaging method is not without its pitfalls; however, a number of alternative methods are given in the next section. An example of how to produce a simplified averaged curve is given below in the case study of the trapezoidal method (Lallemand et al., 1999).

8.5 Data Analysis Options 8.5.1 General Approaches Two common statistical approaches have been taken to perform hypothesis testing on TI data. Perhaps the most obvious test is simply to treat the raw data at whatever time intervals were sampled as the input data to analyses of variance (ANOVA) (e.g., Lee and Lawless, 1991). This approach results in a very large ANOVA with at least three factors-time, panelists, and the treatments of interest. Time and panelists effects may not be of primary interest but will always show large F-ratios due to the fact that people differ and the sensations change over time. This is not news. Another common pattern is a time-by-treatment interaction since all curves will tend to converge near baseline at later time intervals. This is also to be

187

Table 8.3 Parameters extracted from time-intensity curves Parameter Other names Definition Peak intensity Imax , Ipeak Height of highest point on TI record Total duration DUR, Dtotal Time from onset to return to baseline Self-explanatory Area under the curve AUC, Atotal Plateau Dpeak Time difference between reaching maximum and beginning descent Area under plateau Apeak Self-explanatory Area bounded by onset of decline and reaching baseline Area under descending phase Ptotal Rising slope Ri Rate of increase (linear fit) or slope of line from onset to peak intensity Rate of decrease (linear fit) or slope of line from initial declining point to baseline. Declining slope Rf Extinction Time at which curve terminates at baseline Time to reach peak intensity Time to peak Tmax , Tpeak Time to half peak Half-life Time to reach half maximum in decay portion Modified from Lundahl (1992) Other shape parameters are given in Lundahl (1992), based on a half circle of equivalent area under the curve and piding the half circle into rising and falling phase segments

188

8

2006), an effect that is sometimes described as an inpidual “signature.” Examples of inpidual signatures are shown in Fig. 8.3. The causes of these inpidual patterns are unknown but could be attributed to differences in anatomy, differences in physiology such as salivary factors (Fischer et al., 1994), different types of oral manipulation or chewing efficiency (Brown et al., 1994; Zimoch and Gullet, 1997), and inpidual habits of scaling. Some of this information may be lost when analyzing only the extracted parameters.

INTENSITY

Judge 1 Judge 2 Judge 3

0

20

40

60 80 TIME (sec)

100

120

140

Fig. 8.3 Examples of time-intensity records showing characteristic signatures or shapes. Judge 1 shows a record with multiple plateaus, a common occurrence. Judge 2 shows a smooth and continuous curve. Judge 3 shows a steep rise and fall.

A third approach is to fit some mathematical model or set of equations to each inpidual record and then use the constants from the model as the data for comparisons of different products (Eilers and Dijksterhuis, 2004; Garrido et al., 2001; Ledauphin et al., 2005, 2006; Wendin et al., 2003). Given the increasing activity and ingenuity in the area of sensometrics, it is likely that such models will continue to be developed. The sensory scientist needs to ask how useful they are in the product testing arena and whether the model fitting is useful in differentiating products. Various approaches to modeling and mathematical description of TI curves are discussed in the next section.

8.5.2 Methods to Construct or Describe Average Curves The analysis of TI records has produced a sustained interest and response from sensometricians, who have

Time-Intensity Methods

8.5 Data Analysis Options

189

INTENSITY

JUDGE 1 JUDGE 2 AVERAGE

TIME

Fig. 8.4 Two curves with different peak times, if averaged, can lead to a double-peaked curve that resembles neither original data record.

style” either by simple visual inspection of the curves or by a clustering analysis or other statistical methods (van Buuren, 1992; Zimoch and Gullet, 1997). Then these subgroups can be analyzed separately. The analysis may proceed using the simple fixed-time averaging of curve heights or one of the other methods described next. An alternative approach is to average in both the intensity and time directions, by setting each inpidual’s maximum of the mean time to maximum across all curves, and then finding mean times to fixed percentages of maximum in the rising and falling phases of each curve. This procedure was originally published by Overbosch et al. (1986) and subsequent modifications were proposed by Liu and MacFie (1990). The steps in the procedure are shown in Fig. 8.5. Briefly, the method proceeds as follows: In the first step, the geometric mean value for the intensity maximum is found. Inpidual curves are multiplicatively scaled to have this Imax value. In the second step, the geometric mean time to Imax is calculated. In the next steps, geometric mean times are calculated for fixed percentage “slices” of each curve, i.e., at fixed percentages of Imax . For example, the rising and falling phases are “sliced” at 95% of Imax and 90% of Imax and the geometric mean time values to reach these heights are found. This procedure avoids the kind of double-peaked curve that can arise from simple averaging of two distinctly different curve shapes as shown in Fig. 8.4. The method results in several desirable properties that do not necessarily occur with simple averaging at fixed times. First, the Imax value from the mean curve is the geometric mean of the Imax of the inpidual curves. Second, the Tmax value from the mean curve is the geometric mean of the Tmax of the inpidual

curves. Third, the endpoint is the geometric mean of all endpoint times. Fourth, all judges contribute to all segments of the curve. With simple averaging at fixed times, the tail of the curve may have many judges returned to zero and thus the mean is some small value that is a poor repsentation of the data at those points. In statistical terms, the distribution of responses at these later time intervals is positively skewed and left-censored (bound by zero). One approach to this problem is to use the simple median as the measure of central tendency (e.g., Lawless and Skinner, 1979). In this case the summary curve goes to zero when over half the judges go to zero. A second approach is to use statistical techniques designed for estimating measures of central tendency and standard deviations from leftcensored positively skewed data (Owen and DeRouen, 1980). Overbosch’s method works well if all inpidual curves are smoothly rising and falling with no plateaus or multiple peaks and valleys, and all data begin and end at zero. In practice, the data are not so regular. Some judges may begin to fall after the first maximum and then rise again to a second peak. Due to various common errors, the data may not start and end at zero, for example, the record may be truncated within the allowable time of sampling. To accommodate these problems, Liu and MacFie (1990) developed a modification of the above procedure. In their procedure, Imax and four “time landmarks” were averaged, namely starting time, time to maximum, time at which the curve starts to descend from Imax and ending time. The ascending and descending phases of each curve were then pided into about 20 time interval slices. At each time interval, the mean I value is calculated. This method then allows for curves with multiple rising and falling phases and a plateau of maximum intensity that is commonly seen in some judges’ records.

8.5.3 Case Study: Simple Geometric Description A simple and elegant method for comparing curves and extracting parameters by a geometric approximation was described by Lallemand et al. (1999). The authors used the method with a trained texture panel to evaluate different ice cream formulations. The labor-intensive nature of TI studies was illustrated in the fact that

190

8

JUDGE 1 JUDGE 2

Time-Intensity Methods

JUDGE 1 JUDGE 2

A Imax 1

B Geo. mean Imax

Imax 2

JUDGE 1 JUDGE 2

Tmax1

C

JUDGE 1 JUDGE 2

D

Geo. mean Tmax

Tmax2

JUDGE 1 JUDGE 2

Geo. mean 95% Tmax

T1 @ 95% Imax

E

F Geo. mean T @95% Imax

Geo. mean Tmax,Imax

T2 @ 95% Imax

Fig. 8.5 Steps in the data analysis procedure recommended by Overbosch et al. (1986). (a) Two hypothetical time-intensity records from two panelists showing different intensity maxima at different times. (b) The geometric mean value for the intensity maximum is found. Inpidual curves may then by multiplicatively scaled to have this Imax value. (c) The two Tmax values. (d) The geometric mean time to maximum (Tmax ) is calculated.

(e) Geometric mean times are calculated for fixed percentage “slices” of each curve, i.e., at fixed percentages of Imax . The rising phase is “sliced” at 95% of Imax and the time values determined. A similar value will be determined at 95% of maximum for the falling phase. (f) The geometric mean times at each percent of maximum are plotted to generate the composite curve.

12 products were evaluated on 8 different attributes in triplicate sessions, requiring about 300 TI curves from each panelist! Texture panelists were given over 20 sessions of training, although only a few of the final sessions were specifically devoted to practice

with the TI procedure. Obviously, this kind of extensive research program requires a significant time and resource commitment. Data were collected using a computer-assisted rating program, where mouse movement was linked to

8.5 Data Analysis Options

191

the position of a cursor on a 10 cm 10-point scale. The authors noted a number of issues with the data records due to “mouse artifacts” or other problems. These included sudden unintended movement of the mouse leading to false peaks or ratings after the end of the sensation, mouse blockage leading to unusable records, and occasional inaccurate positioning by panelists causing data that did not reflect their actual perceptions. Such ergonomic difficulties are not uncommon in TI studies although they are rarely reported or discussed. Even with this highly trained panel, from 1 to 3% of the records needed to be discarded or manually corrected due to artifacts or inaccuracies. Sensory professionals should not assume that just because they have a computer-assisted TI system, the human factors in mouse and machine interactions will always work smoothly and as planned. Examples of response artifacts are shown in Fig. 8.6. T1 T2

T3

False Plateau

INTENSITY

“Mouse artifacts”

TIME (sec)

Fig. 8.6 Response artifacts in TI records. The solid line shows some perhaps unintended mouse movement (muscle spasm?) near the peak intensity. The dashed line shows a bump in the mouse after sensation ceased and returned to zero. The dotted line illustrates an issue in determining at what point the intensity plateau has ended. The short segment between T1 and T2 may have simply been an adjustment of the mouse after the sudden rise, when the panelist felt they overshot the mark. The actual end of the plateau might more reasonably be considered to occur at T3. (see Lallemand et al. 1999).

Lallemand and coworkers noted that TI curves often took a shape in which the sensation rose to a plateau near peak intensity for a period during which intensity ratings changed very little and then fell to the baseline. They reasoned that a simple geometric approximation by a trapezoid shape might suffice for extracting curve parameters and finding the area under the curve (not unlike the trapezoidal approximation method used for integration in calculus). In principle, four points could

be defined that would describe the curve: the onset time, the time at the intensity maximum or beginning of the plateau, the time at which the plateau ended and the decreasing phase began, and the time at which sensation stopped. These landmarks are those originally proposed by Lui and MacFie (1990). In practice, these points turned out to be more difficult to estimate than expected, so some compromises were made. For example, some records would show a gradually decreasing record during the “plateau” and before the segment with a more rapidly falling slope was evident. How much of a decrease would justify the falling phase or conversely, how little of a decrease would be considered still part of the plateau (see Fig. 8.6)? Also, what should one do if the panelist did not return to zero sensation or unintentionally bumped the mouse after reaching zero? In order to solve these issues, the four points were chosen at somewhat interior sections of the curve, namely the times at 5% of the intensity maximum for the onset and endpoint of the trapezoid, and the times at 90% of the intensity maximum for the beginning and end of the plateau. This approximation worked reasonably well, and its application to a hypothetical record is shown in Fig. 8.7. Given the almost 3,000 TI curves in this one study, the trapezoid points were not mapped by hand or by eye, but a special program written to extract the points. However, for smaller experiments it should be quite feasible to do this kind of analysis “by hand” on any collection of graphed records. The establishment of the four trapezoid vertices now allows extraction of the six basic TI curve parameters for statistical analysis (5 and 90% of maximum intensity points, the four times at those points), as well as the intensity maximum from the original record, and derived (secondary) parameters such as rising and falling slopes and the total area under the curve. Note that the total area becomes simply the sum of the two triangles and the rectangle described by the plateau. These are shown in the lower section of Fig. 8.7. A composite trapezoid can be drawn from the averaged points. The utility and validity of the method was illustrated in one sample composite record, showing the fruity flavor intensity from two ice creams differing in fat content. Consistent with what might be expected from the principles of flavor release, the higher fat sample had a slower and more delayed rise to the peak (plateau) but a longer duration. This would be pdicted if the higher fat level was better able to sequester

192

8

Time-Intensity Methods

with their single-point intensity estimates. Consistent with the latter notion, the profiling scores could be better modeled by a combination of several of the TI parameters. The simplicity and validity of this analysis method suggests that it should find wider application in industrial settings.

Intensity trapezoidal approxiamtion to T-I curve

A,D = points at 5% of maximum B,C = points at 90% of maximum

8.5.4 Analysis by Principal Components Time DERIVED PARAMETERS;

B

C

Rd Ri Ai

Am

Ad

A

D Time Di

Dm

Dd

Fig. 8.7 The trapezoidal method of Lallemand et al. (1999) for assessing curve parameters on TI records. The upper panel shows the basic scheme in which four points are found when the initial 5% of the intensity maximum (Imax ) occurs, when 90% of Imax is first reached on the ascending segment, when the plateau is finished at 90% of Imax on the descending phase and the endpoint approximation at 5% of Imax on the descending phase. The lower panel shows the derived parameters, namely Ri , Ai , and Di for the rate (slope), area, and duration of the initial rising phase; Am and Dm for the area and duration of the middle plateau section; and Rd , Ad , and Dd for the rate (slope), area, and duration of the falling phase. A total duration can be found from the sum of Di , Dm , and Dd . The total area is given by the sum of the A parameters or by the formula for the area of a trapezoid: Total area = (I90 -I5 ) (2Dm + Di + Dd )/2. (Height times the sum of the two parallel segments, then pided by 2).

a lipophilic or nonpolar flavor compound and thus delay the flavor release. They also examined the correlation with a traditional texture descriptive analysis and found very low correlations of inpidual TI parameters with texture profiling mean scores. This would be expected if the TI parameters were contributing unique information or if the texture profilers were integrating a number of time-dependent events in coming up

Another analysis uses principal components analysis (PCA, discussed in Chapter 18) (van Buuren, 1992). Briefly, PCA is a statistical method that “bundles” groups of correlated measurements and substitutes a new variable (a factor or principal component) in place of the original variables, thus simplifying the picture. In studying the time-intensity curves for bitterness or different brands of lager beer, van Buuren noticed that inpiduals once again produced their own characteristic “style” of curve shape. Most people showed a classic TI curve shape, but some subjects were classified as “slow starters,” with a delayed peak and some showed a tendency to persist and not come back down to baseline within the test period. Submission of the data to PCA allowed the extraction of a “principal curve” which captured the majority trend. This showed a peaked TI curve and a gradual return to baseline. The second principal curve captured the shape of the minority trends, with slow onset, a broad peak and slow decline without reaching baseline. The principal curves were thus able to extract judge trends and provide a cleaned-up view of the primary shape of the combined data (Zimoch and Gullet, 1997). Although a PCA program may extract a number of principal components, not all may be practically meaningful (for an example, see Reinbach et al., 2009), and the user should examine each one for the story it tells. Reasonable questions are whether the component reflects something important relative to the simple TI curve parameters, and whether it shows any patterns related to inpidual differences among panelists. Dijksterhuis explored the PCA approach in greater detail (Dijksterhuis, 1993; Dijksterhuis and van den Broek, 1995; Dijksterhuis et al., 1994). Dijksterhuis (1993) noted that the PCA method as applied by van Buuren was not discriminating of different bitter stimuli. An alternative approach was “non-centered PCA” in which curve height information was retained during

8.6 Examples and Applications

data processing, rather than normalizing curves to a common scale. The non-centered approach works on the raw data matrix. Stimuli or treatments were better distinguished. The first principal curve tends to look like the simple average, while the second principal curve contains rate information such as accelerations or inflection points (Dijksterhuis et al., 1994). This could be potentially useful information in differentiating the subtle patterns of TI curves for different flavors. The PCA approach also involves the possibility of generating weights for different assessors that indicate the degree to which they contribute to the different principal curves. This could be an important tool for differentiating outliers in the data or panelists with highly unusual TI signatures (Peyvieux and Dijksterhuis, 2001).

8.6 Examples and Applications A growing number of studies have used TI methods for the evaluations of flavor, texture, flavor release, hedonics, and basic studies of the chemical senses. A short review of these studies follows in this section, although the reader is cautioned that the list is not exhaustive. We have cited a few of the older studies to give credit to the pioneers of this field as well as some of the newer applications. The examples are meant to show the range of sensory studies for which TI methods are suitable.

8.6.1 Taste and Flavor Sensation Tracking A common application of continuous time-intensity scaling is tracking the sensation rise and decay from important flavor ingredients, such as sweeteners (Swartz, 1980). An early study of Jellinek and Pangborn reported that addition of salt to sucrose extended the time-intensity curve and made the taste “more rounded” in their words (Jellinek, 1964). One of the salient characteristics of many intensive or non-carbohydrate sweeteners is their lingering taste that is different from that of sucrose. Time-intensity tracking of sweet tastes was an active area of study (Dubois and Lee, 1983; Larson-Powers and Pangborn, 1978; Lawless and Skinner, 1979; Yoshida, 1986) and

193

194

8

Cliff and Noble, 1990; Matysiak and Noble, 1991). Using these techniques, enhancement of sweetness by fruity volatiles has been observed, and differences in the interactions were seen for different sweetening agents.

8.6.2 Trigeminal and Chemical/Tactile Sensations Reactions to other chemical stimuli affecting irritation or tactile effects in the mouth have been a fertile area for time-related sensory measurements. Compounds such as menthol produce extended flavor sensations and the time course is concentration dependent (Dacanay, 1990; Gwartney and Heymann, 1995). A large number of studies of the burning sensation from hot pepper compounds have used time-intensity scaling, using repeated ratings at discrete time intervals as well as continuous tracking (Cliff and Heymann, 1993a; Green, 1989; Green and Lawless, 1991; Lawless and Stevens, 1988; Stevens and Lawless, 1986). Given the slow onset and extended time course of the sensations induced by even a single sample of a food containing hot pepper, this is a highly appropriate application for time-related judgments. The repeated ingestion paradigm has also been used to study the short- and long-term desensitization to irritants such as capsaicin and zingerone (Prescott and Stevenson, 1996). The temporal profile of different irritative spice compounds is an important point of qualitative differentiation (Cliff and Heymann, 1992). Reinbach and colleagues used TI tracking to study the oral heat from capsaicin in various meat products (Reinbach et al., 2007, 2009) as well as the interactions of oral burn with temperature (see also Baron and Penfield, 1996). When examining the decay curves following different pepper compounds, different time courses can be fit by different decay constants in a simple exponential curve of the form R = Ro e−kt

Time-Intensity Methods

from peak during the decay portion of the time curve (Lawless, 1984). Another chemically induced tactile or feeling factor in the mouth that has been studied by TI methods is astringency. Continuous tracking has been applied to the astringency sensation over repeated ingestions (Guinard et al., 1986). Continuous tracking can provide a clear record of how sensations change during multiple ingestions, and how flavors may build as subsequent sips or tastings add greater sensations on an existing background. An example of this is shown in Fig. 8.8 where the saw-tooth curve shows an increase and builds astringency over repeated ingestions (Guinard et al., 1986). Astringency has also been studied using repeated discrete-point scaling. For example, Lawless and coworkers were able to show differences in the time profiles of some sensory sub-qualities related to astringency, namely dryness, roughness in the mouth, and puckery tightening sensations (Lawless et al., 1994; Lee and Lawless, 1991) depending on the astringent materials being evaluated.

Fig. 8.8 Continuous tracking record with multiple ingestions producing a “sawtooth” curve record. The abscissa shows the time axis in seconds and the ordinate the mean astringent intensity from 24 judges. The dashed curve is from a 15 ml sample of a base wine with 500 gm/l added tannin and the solid curve from the base wine with no added tannin. Sample intake and expectoration are indicated by stars and arrows, respectively. From Guinard et al. (1986), reprinted with permission of the American Society of Enology and Viticulture.

(8.1)

or

8.6.3 Taste and Odor Adaptation ln R = ln Ro − kt

(8.2)

where Ro is the peak intensity and t is time, k is the value determining how rapid the sensation falls off

The measurement of a flavor sensation by tracking over time has a close parallel in studies of taste and odor adaptation (Cain, 1974; Lawless and Skinner,

8.6 Examples and Applications

1979; O’Mahony and Wong, 1989). Adaptation may be defined as the decrease in responsiveness of a sensory system to conditions of constant stimulation providing a changing “zero point” (O’Mahony, 1986). Adaptation studies have sometimes used discrete, single bursts of stimulation (e.g., Meiselman and Halpern, 1973). An early study of adaptation used a flowing system through the subject’s entire mouth using pipes inserted through a dental impssion material that was held in the teeth (Abrahams et al., 1937). Disappearance of salt taste was achieved in under 30 s for a 5% salt solution, although higher concentrations induced painful sensations over time that were confusing to the subjects and sometimes masked the taste sensation. By flowing a continuous stream over a section of the subject’s tongue or stabilizing the stimulus with wet filter paper, taste often disappears in under a few minutes (Gent, 1979; Gent and McBurney, 1978; Kroeze, 1979; McBurney, 1966). Concentrations above the adapting level are perceived as having the characteristic taste of that substance, e.g., salty for NaCl. Concentrations below that level, to which pure water is the limiting case, take on other tastes so that water after NaCl, for example, is sour-bitter (McBurney, and Shick, 1971). Under other conditions adaptation may be incomplete (Dubose et al., 1977; Lawless and Skinner, 1979; Meiselman and Dubose, 1976). Pulsed flow or intermittent stimulation causes adaptation to be much less complete or absent entirely (Meiselman and Halpern, 1973).

8.6.4 Texture and Phase Change Tactile features of foods and consumer products have been evaluated using time-related measurements. Phase change is an important feature of many products that undergo melting when eaten. These include both frozen products like ice cream and other dairy desserts and fatty products with melting points near body temperature such as chocolate. Using the chartrecording method for time-intensity tracking, Moore and Shoemaker (1981) evaluated the degree of coldness, iciness, and sensory viscosity of ice cream with different degrees of added carboxymethyl cellulose. The added carbohydrate shifted the time for peak intensity of iciness and extended the sensations of coldness on the tongue. Moore and Shoemaker also

195

8.6.5 Flavor Release A potentially fertile area for the application of timerelated sensory measurements is in flavor release from foods during eating. Not only are texture changes

196

8

Time-Intensity Methods

cheese systems (Pionnier et al., 2004) using a discrete point TI method.

8.6.6 Temporal Aspects of Hedonics Since the pleasantness or appeal of a sensory characteristic is largely dependent on its intensity level, it is not surprising that one’s hedonic reaction to a product might shift over time as the strength of a flavor waxes and wanes. Time-related shifts in food likes and dislikes are well known. In the phenomenon known as alliesthesia, our liking for a food depends a lot on whether we are hungry or replete (Cabanac, 1971). The delightful lobster dinner we enjoyed last night may not look quite so appealing as leftovers at lunchtime the next day. Wine tasters may speak of a wine that is “closed in the glass, open on the palate, and having a long finish.” Accompanying such a description is the implicit message that this particular wine got better over the course of the sensory experience. Given the shorter time span of flavor and texture sensations in the mouth, we can ask whether there are shifts in liking and disliking. This has been measured in several studies. Taylor and Pangborn (1990) examined liking for chocolate milk with varying degrees of milk fat. Different inpidual trends were observed in liking for different concentrations, and this affected the degree of liking expssed over time. Another example of hedonic TI scaling was in a study of the liking/disliking for the burning oral sensation from chili (hot ) pepper (Rozin et al., 1982). They found different patterns of temporal shifting in liking as the burn rose and declined. Some subjects liked the burn at all time intervals, some disliked the burn at all time intervals, and some shifted across neutrality as strong burns became more tolerable. This method was revisited by Veldhuizen et al. (2006) who used a simple bipolar line scale for pleasantness and had subjects evaluated both intensity and hedonic reactions to a citrus beverage flowed over the tongue from a computer-controlled delivery system (see Fig. 8.2 for an early example of this kind of device). Note that with a bipolar hedonic scale, the mouse and cursor positions must begin at the center of the scale and not the lower end as with intensity scaling. The authors found a delayed pleasantness response compared to the intensity tracking, a similar time to maximum, but an unexpectedly quicker offset of response for pleasantness tracking. Some panelists

8.7 Issues

produced a double-peaked pleasantness response, as the sensation could rise in pleasantness, but then become too intense, but become more pleasant again as adaptation set in and the perceived strength decreased.

8.7 Issues Sensory scientists who wish to use time-intensity methods for any particular study need to weigh the potential for obtaining actionable information against the cost and time involved in collecting these data. Some orientation or training is required (Peyvieux and Dijksterhuis, 2001) and in some published studies, the training and practice is quite extensive. For example, Zimoch and Gullet (1997) trained their meat texture panel for 12 h. Panelists must be trained to use the response device and sufficiently practiced to feel comfortable with the requirements of the task in terms of maintaining focused attention to momentary sensation changes. With the use of online data collection the tabulation and processing of information is generally not very labor intensive; but without computer-assisted collection the time involved can be enormous. Even with computerized systems, the data collection is not foolproof. Responses may be truncated or fail to start at zero in some records (Liu and MacFie, 1990; McGowan and Lee, 2006) making automatic averaging of records infeasible. In one study of melting behavior (Lawless et al., 1996), some subjects mistakenly returned the indicating cursor to zero as soon as the product was completely melted, instead of leaving the cursor on maximum, producing truncated records. Such unexpected events remind us to never assume that panelists are doing what you think they should be doing. A fundamental issue is information gain. In the case where changes in duration are observed at equal maximum intensities, it can be argued that the traditional scaling might have missed important sensory differences. For example, TI can capture information such as when the TI curves cross over, e.g., the interesting case when a product with a lower peak intensity has a longer duration (e.g., Lallemand et al., 1999; Lawless et al., 1996). However, this pattern is not often seen. Usually products with stronger peak height have longer durations. In general there is a lot of redundant information in TI parameters. Lundahl (1992) studied the correlation of 15 TI parameters associated with TI

197

curves’ shapes, sizes, and rates of change. Curve size parameters were highly correlated and usually loaded on the first principal component of a PCA, capturing most of the variance (see also Cliff and Noble, 1990). Curve size parameters, including peak height, were correlated with simple category ratings of the same beverages. So an open question for the sensory scientist is whether there is any unique information in the TI parameters extracted from the records and whether there is information gain over what would be provided by more simple direct scaling using a single intensity rating. A potential problem in time-intensity methods is that factors affecting response behavior are not well understood. In TI measurements, there are dynamic physical processes (chewing, salivary dilution) leading to changes in the stimulus and resulting sensations (Fischer et al., 1994). A second group of processes concerns how the participant translates the conscious experience into an overt response, including a decision mechanism and motoric activation (Dijksterhuis, 1996). The notion that TI methods provide a direct link from the tongue of the subject to the hand moving the mouse is a fantasy. Even in continuous tracking, there must be some decision process involved. There is no information as to how often a panelist in the continuous procedure reflects upon the sensation and decides to change the position of the response device. Decisions are probably not continuous even though some records from some subjects may look like smooth curves. An indication that response tendencies are important is when the conditions of stimulation are held constant, but the response task changes. For example, using the graphic chart-recorder method, Lawless and Skinner (1979) found median durations for sucrose intensity that were 15-35% shorter than the same stimuli rated using repeated category ratings. Why would the different rating methods produce apparently different durations? Very different patterns may be observed when taste quality and intensity are tracked. Halpern (1991) found that tracked taste quality of 2 mM sodium saccharin had a delayed onset (by 400 ms) compared with tracked intensity. This might be understandable from the point of requiring a more complex decision process in the case of tracking intensity. However, it still alerts us to the fact that the behavior probably trails the actual experience by some unknown amount. What is more surprising in Halpern’s data is that tracked quality also stopped well before tracked intensity (by

198

600 ms). Can it be possible that subjects are still experiencing a taste of some definable intensity and yet the quality has disappeared? Or is there another response generation process at work in this task? A third area where potential response biases can operate is in contextual effects. Clark and Lawless (1994) showed that common contextual affects like successive contrast also operated with TI methods, as they do with other scaling tasks. Also, some ratings could be enhanced when a limited number of scales were used by subjects. As observed in singlepoint scaling, enhancement of sweetness by fruity flavors tends to occur when only sweetness is rated. When the fruity flavor is also rated, the sweetness enhancement often disappears (Frank et al., 1989), an effect sometimes referred to as halo dumping or simply “dumping.” Using the discrete-point version of TI scaling, so that multiple attributes could be rated, Clark and Lawless showed a similar effect. This is potentially troublesome for the continuous tracking methods, since they often limit subjects to responding to only one attribute at a time. This may explain in part why sweetness enhancement by flavors can occur so readily in TI studies (e.g., Matysiak and Noble, 1991). A final concern is the question of whether the bounded response scales often used in TI measurement produces any compssion of the differences among products. In analog tracking tasks, there is a limit as to how far the joystick, mouse, lever, dial, or other response device can be moved. With some practice, judges learn not to bump into the top. Yet the very nature of the tracking response encourages judges to sweep a wide range of the response scale. If this were done on every trial, it would tend to attenuate the differences in maximum tracked intensity between products. As an example, Overbosch et al. (1986, see Fig. 2) showed curves for pentanone where doubling the concentration changed peak heights by only about 8%. A similar sort of compssion is visible in Lawless and Skinner’s (1979) data for sucrose, compared to the psychophysical data in the literature.

8.8 Conclusions In most cases TI parameters show similar statistical differentiation as compared to traditional scales, but this is not universally the case (e.g., Moore and Shoemaker, 1981).

8

Time-Intensity Methods

Many sensory evaluation researchers have supported increased application of time-intensity measurements for characterization of flavor and texture sensations. In particular, the method was championed by Lee and Pangborn, who argued that the methods provide detailed information not available from single estimates of sensation intensity (Lee, 1989; Lee and Pangborn, 1986). TI methods can provide rate-related, duration, and intensity information not available from traditional scaling. However, the utility of the methods must be weighed against the enhanced cost and complexity in data collection and analysis. In deciding whether to apply TI methods over conventional scaling, the sensory scientist should consider the following criteria: (1) Is the attribute or system being studied known to change over time? Simply eating the food can often settle this issue; in many cases it is obvious. (2) Will the products differ in sensory time course as a function of ingredients, processing, packaging, or other variables of interest? (3) Will the time variation occur in such a way that it will probably not be captured by direct single ratings? (4) Is some aspect of the temporal profile likely to be related to consumer acceptability? (5) Does the added information provided by the technique outweigh any additional costs or time delays in panel training, data acquisition, and data analysis? Obviously, when more answers are positive on these criteria, a stronger case can be made for choosing a TI method from the available set of sensory evaluation tools.

References Abrahams, H., Krakauer, D. and Dallenbach, K. M. 1937. Gustatory adaptation to salt. American Journal of Psychology, 49, 462-469. Ayya, N. and Lawless, H. T. 1992. Qualitative and quantitative evaluation of high-intensity sweeteners and sweetener mixtures. Chemical Senses, 17, 245-259. Baron, R. F. and Penfield, M. P. 1996. Capsaicin heat intensity – concentration, carrier, fat level and serving temperature effects. Journal of Sensory Studies, 11, 295-316. Barylko-Pikielna, N., Mateszewska, I. and Helleman, U. 1990. Effect of salt on time-intensity characteristics of bread. Lebensmittel Wissenschaft und Technologie, 23, 422-426.

References Birch, G. G. and Munton, S. L. 1981. Use of the “SMURF” in taste analysis. Chemical Senses, 6, 45-52. Bloom, K., Duizer, L. M. and Findlay, C. J. 1995. An objective numerical method of assessing the reliability of time- intensity panelists. Journal of Sensory Studies, 10, 285-294. Bonnans, S. and Noble, A. C. 1993. Effect of sweetener type and of sweetener and acid levels on temporal perception of sweetness, sourness and fruitiness. Chemical Senses, 18, 273-283. Brandt, M. A., Skinner, E. Z. and Coleman, J. A. 1963. Texture profile method. Journal of Food Science, 28, 404-409. Brown, W. E., Landgley, K. R., Martin, A. and MacFie, H. J. 1994. Characterisation of patterns of chewing behavior in human subjects and their influence on texture perception. Journal of Texture Studies, 15, 33-48. Butler, G., Poste, L. M., Mackie, D. A., and Jones, A. 1996. Time-intensity as a tool for the measurement of meat tenderness. Food Quality and Preference, 7, 193-204. Cabanac, M. 1971. Physiological role of pleasure. Science, 173, 1103-1107. Cain, W. S. 1974. Perception of odor intensity and time-course of olfactory adaptation. ASHRAE transactions, 80, 53-75. Clark, C. C. and Lawless, H. T. 1994. Limiting response alternatives in time-intensity scaling: An examination of the Halo-Dumping effect. Chemical Senses, 19, 583-594. Cliff, M. 1987. Temporal perception of sweetness and fruitiness and their interaction in a model system. MS Thesis, University of California, Davis, USA. Cliff, M. and Heymann, H. 1992. Descriptive analysis of oral pungency. Journal of Sensory Studies, 7, 279-290. Cliff, M. and Heymann, H. 1993a. Time-intensity evaluation of oral burn. Journal of Sensory Studies, 8, 201-211. Cliff, M. and Heymann, 1993b. Development and use of time- intensity methodology for sensory evaluation: A review. Food Research International, 26, 375-385. Cliff, M. and Noble, A. C. 1990. Time-intensity evaluation of sweetness and fruitiness in a model solution. Journal of Food Science, 55, 450-454. Dacanay, L. 1990. Thermal and concentration effects on temporal sensory attributes of L – menthol. M.S. Thesis, University of California, Davis, USA. de Roos, K. B. 1990. Flavor release from chewing gums. In: Y. Bessiere and A. F. Thomas (eds.), Flavour Science and Technology. Wiley, Chichester, pp. 355-362. DeRovira, D. 1996. The dynamic flavor profile method. Food Technology, 50, 55-60. Dijksterhuis, G. 1993. Principal component analysis of time- intensity bitterness curves. Journal of Sensory Studies, 8, 317-328. Dijksterhuis, G. 1996. Time-intensity methodology: Review and pview. Proceedings, COST96 Meeting: Interaction of Food Matrix with Small Ligands Influencing Flavour and Texture, Dijon, France, November 20, 1995. Dijksterhuis, G. and van den Broek, E. 1995. Matching the shape of time-intensity curves. Journal of Sensory Studies, 10, 149-161. Dijksterhuis, G., Flipsen, M. and Punter, P. H. 1994. Principal component analysis of time-intensity data. Food Quality and Preference, 5, 121-127. DuBois, G. E. and Lee, J. F. 1983. A simple technique for the evaluation of temporal taste properties. Chemical Senses, 7, 237-247.

200 DuBois (eds.), Sweeteners: Discovery, Molecular Design and Chemoreception. ACS Symposium Series #450. American Chemical Society, Washington, DC, pp. 277-289. Jellinek, G. 1964. Introduction to and critical review of modern methods of sensory analysis (odor taste and flavor evaluation) with special emphasis on descriptive analysis. Journal of Nutrition and Dietetics, 1, 219-260. Jellinek, G. 1985. Sensory Evaluation of Food, Theory and Practice. Ellis Horwood, Chichester, England. Kroeze, J. H. A. 1979. Masking and adaptation of sugar sweetness intensity. Physiology and Behavior, 22, 347-351. Kuo, Y.-L., Pangborn, R. M. and Noble, A. C. 1993. Temporal patterns of nasal, oral and retronasal perception of citral and vanillin and interactions of these odorants with selected tastants. International Journal of Food Science and Technology, 28, 127-137. Labbe, D., Schlich, P., Pineau, N., Gilbert, F. and Martin, N. 2009. Temporal dominance of sensations and sensory profiling: A comparative study. Food Quality and Preference, 20, 216-221. Lallemand, M., Giboreau, A., Rytz, A. and Colas, B. 1999. Extracting parameters from time-intensity curves using a trapezoid model: The example of some sensory attributes of ice cream. Journal of Sensory Studies, 14, 387-399. Larson-Powers, N. and Pangborn, R. M. 1978. Paired comparison and time-intensity measurements of the sensory properties of beverages and gelatins containing sucrose or synthetic sweeteners. Journal of Food Science, 43, 41-46. Lawless, H. T. 1980. A computerized system for assessing taste intensity over time. Paper psented at the Chemical Senses and Intake Society, Hartford, CT, April 9, 1980. Lawless, H. T. 1984. Oral chemical irritation: Psychophysical properties. Chemical Senses, 9, 143-155. Lawless, H. T. and Clark, C. C. 1992. Psychological biases in time-intensity scaling. Food Technology, 46(11), 81, 84-86, 90. Lawless, H. T. and Skinner, E. Z. 1979. The duration and perceived intensity of sucrose taste. Perception and Psychophysics, 25, 249-258. Lawless, H. T. and Stevens, D. A. 1988. Responses by humans to oral chemical irritants as a function of locus of stimulation. Perception and Psychophysics, 43, 72-78. Lawless, H. T., Corrigan, C. L. and Lee, C. L. 1994. Interactions of astringent substances. Chemical Senses, 19, 141-154. Lawless, H. T., Tuorila, H., Jouppila, K., Virtanen, P. and Horne, J. 1996. Effects of guar gum and microcrystalline cellulose on sensory and thermal properties of a high fat model food system. Journal of Texture Studies 27, 493-516. Le Reverend, F. M., Hidrio, C., Fernandes, A. and Aubry, V. 2008. Comparison between temporal dominance of sensation and time intensity results. Food Quality and Preference, 19, 174-178. Leach, E. J. and Noble, A. C. 1986. Comparison of bitterness of caffeine and quinine by a time-intensity procedure. Chemical Senses, 11, 339-345. Ledauphin, S., Vigneau, E. and Causeur, D. 2005. Functional approach for the analysis of time intensity curves using B-splines. Journal of Sensory Studies, 20, 285-300. Ledauphin, S., Vigneau, E. and Qannari, E. M. 2006. A procedure for analysis of time intensity curves. Food Quality and Preference, 17, 290-295.

8

Time-Intensity Methods

Lee, C. B. and Lawless, H. T. 1991. Time-course of astringent materials. Chemical Senses, 16, 225-238. Lee, W. E. 1985. Evaluation of time-intensity sensory responses using a personal computer. Journal of Food Science, 50, 1750-1751. Lee, W. E. 1986. A suggested instrumental technique for studying dynamic flavor release from food products. Journal of Food Science, 51, 249-250. Lee, W. E. 1989. Single-point vs. time-intensity sensory measurements: An informational entropy analysis. Journal of Sensory Studies, 4, 19-30. Lee, W. E. and Pangborn, R. M. 1986. Time-intensity: The temporal aspects of sensory perception. Food Technology, 40, 71-78, 82. Liu, Y. H. and MacFie, H. J. H. 1990. Methods for averaging time-intensity curves. Chemical Senses, 15, 471-484. Lundahl, D. S. 1992. Comparing time-intensity to category scales in sensory evaluation. Food Technology, 46(11), 98-103. Lynch, J., Liu, Y.-H., Mela, D. J. and MacFie, H. J. H. 1993. A time-intensity study of the effect of oil mouthcoatings on taste perception. Chemical Senses, 18, 121-129. Matysiak, N. L. and Noble, A. C. 1991. Comparison of temporal perception of fruitiness in model systems sweetened with aspartame, aspartame + acesulfame K blend or sucrose. Journal of Food Science, 65, 823-826. McBurney, D. H. 1966. Magnitude estimation of the taste of sodium chloride after adaptation to sodium chloride. Journal of Experimental Psychology, 72, 869-873. McBurney, D. H. and Shick, T. R. 1971. Taste and water taste of 26 compounds for man. Perception and Psychophysics, 11, 228-232. McGowan, B. A. and Lee, S.-Y. 2006. Comparison of methods to analyze time-intensity curves in a corn zein chewing gum study. Food Quality and Preference 17, 296-306. McNulty, P. B. 1987. Flavour release-elusive and dynamic. In: J. M. V. Blanshard and P. Lillford (eds.), Food Structure and Behavior. Academic, London, pp. 245-258. McNulty, P. B. and Moskowitz, H. R. 1974. Intensity -time curves for flavored oil-in-water emulsions. Journal of Food Science, 39, 55-57. Meiselman, H. L. 1968. Magnitude estimation of the time course of gustatory adaptation. Perception and Psychophysics, 4, 193-196. Meiselman, H. L. and Dubose, C. N. 1976. Failure of instructional set to affect completeness of taste adaptation. Perception and Psychophysics, 19, 226-230. Meiselman, H. L. and Halpern, B. P. 1973. Enhancement of taste intensity through pulsatile stimulation. Physiology and Behavior, 11, 713-716. Moore, L. J. and Shoemaker, C. F. 1981. Sensory textural properties of stabilized ice cream. Journal of Food Science, 46, 399-402, 409. Neilson, A. J. 1957. Time-intensity studies. Drug and Cosmetic Industry, 80, 452-453, 534. O’Keefe, S. F., Resurreccion, A. P., Wilson, L. A. and Murphy, P. A. 1991. Temperature effect on binding of volatile flavor compounds to soy protein in aqueous model systems. Journal of Food Science, 56, 802-806. O’Mahony, M. 1986. Sensory adaptation. Journal of Sensory Studies, 1, 237-257.

References O’Mahony, M. and Wong, S.-Y. 1989. Time-intensity scaling with judges trained to use a calibrated scale: Adaptation, salty and umami tastes. Journal of Sensory Studies, 3, 217-236. Ott, D. B., Edwards, C. L. and Palmer, S. J. 1991. Perceived taste intensity and duration of nutritive and non-nutritive sweeteners in water using time-intensity (T-I) evaluations. Journal of Food Science, 56, 535-542. Overbosch, P. 1987. Flavour release and perception. In: M. Martens, G. A. Dalen and H. Russwurm (eds.), Flavour Science and Technology. Wiley, New York, pp. 291-300. Overbosch, P., Van den Enden, J. C., and Keur, B. M. 1986. An improved method for measuring perceived intensity/time relationships in human taste and smell. Chemical Senses, 11, 315-338. Owen, W. J. and DeRouen, T. A. 1980. Estimation of the mean for lognormal data containing zeroes and left-censored values, with application to the measurement of worker exposure to air contaminants. Biometrics, 36, 707-719. Pangborn, R. M. and Koyasako, A. 1981. Time-course of viscosity, sweetness and flavor in chocolate desserts. Journal of Texture Studies, 12, 141-150. Pangborn, R. M., Lewis, M. J. and Yamashita, J. F. 1983. Comparison of time-intensity with category scaling of bitterness of iso-alpha-acids in model systems and in beer. Journal of the Institute of Brewing, 89, 349-355. Peyvieux, C. and Dijksterhuis, G. 2001. Training a sensory panel for TI: A case study. Food Quality and Preference, 12, 19-28. Pineau, N., Schlich, P., Cordelle, S., Mathonniere, C., Issanchou, S., Imbert, A., Rogeaux, M., Eteviant, P. and Köster, E. 2009. Temporal dominance of sensations: Construction of the TDS curves and comparison with time- intensity. Food Quality and Preference, 20, 450-455. Pionnier, E., Nicklaus, S., Chabanet, C., Mioche, L., Taylor, A. J., LeQuere, J. L. and Salles, C. 2004. Flavor perception of a model cheese: relationships with oral and physico-chemical parameters. Food Quality and Preference, 15, 843-852. Prescott, J. and Stevenson, R. J. 1996. Psychophysical responses to single and multiple psentations of the oral irritant zingerone: Relationship to frequency of chili consumption. Physiology and Behavior, 60-617-624. Reinbach, H. C., Toft, M. and Møller, P. 2009. Relationship between oral burn and temperature in chili spiced pork patties evaluated by time-intensity. Food Quality and Preference, 20, 42-49. Reinbach, H. C., Meinert, L., Ballabio, D., Aayslyng, M. D., Bredie, W. L. P., Olsen, K. and Møller, P. 2007. Interactions between oral burn, meat flavor and texture in chili spiced pork patties evaluated by time-intensity. Food Quality and Preference, 18, 909-919. Rine, S. D. 1987. Computerized analysis of the sensory properties of peanut butter. M. S. Thesis, University of California, Davis, USA.

Chapter 9

Context Effects and Biases in Sensory Judgment

Abstract Human judgments about a sensation or a product are strongly influenced by items that surround the item of interest, either in space or in time. This chapter shows how judgments can change as a function of the context within which a product is evaluated. Various contextual effects and biases are described and categorized. Some solutions and courses of action to minimize these biases are psented. By such general principles of action as these everything looked at, felt, smelt or heard comes to be located in a more or less definite position relatively to other collateral things either actually psented or only imagined as possibly there. – James (1913, p. 342)

Contents

9.6.3 Positional or Order Bias . . . . . . Antidotes . . . . . . . . . . . . . . . . 9.7.1 Avoid or Minimize . . . . . . . . . 9.7.2 Randomization and Counterbalancing 9.7.3 Stabilization and Calibration . . . . 9.7.4 Interptation . . . . . . . . . . . 9.8 Conclusions . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . 9.7

9.1 9.2

9.3

9.4

9.5

9.6

Introduction: The Relative Nature of Human Judgment . . . . . . . . . . . . . . . . . . Simple Contrast Effects . . . . . . . . . . . 9.2.1 A Little Theory: Adaptation Level . . . . 9.2.2 Intensity Shifts . . . . . . . . . . . . 9.2.3 Quality Shifts . . . . . . . . . . . . . 9.2.4 Hedonic Shifts . . . . . . . . . . . . 9.2.5 Explanations for Contrast . . . . . . . . Range and Frequency Effects . . . . . . . . . 9.3.1 A Little More Theory: Parducci’s Range and Frequency Principles . . . . . . . . 9.3.2 Range Effects . . . . . . . . . . . . . 9.3.3 Frequency Effects . . . . . . . . . . . Biases . . . . . . . . . . . . . . . . . . . . 9.4.1 Idiosyncratic Scale Usage and Number Bias . . . . . . . . . . . 9.4.2 Poulton’s Classifications . . . . . . . . 9.4.3 Response Range Effects . . . . . . . . 9.4.4 The Centering Bias . . . . . . . . . . Response Correlation and Response Restriction . . . . . . . . . . . . . . . . . 9.5.1 Response Correlation . . . . . . . . . 9.5.2 “Dumping” Effects: Inflation Due to Response Restriction in Profiling . . . 9.5.3 Over-Partitioning . . . . . . . . . . . Classical Psychological Errors and Other Biases . . . . . . . . . . . . . . . . . . . . 9.6.1 Errors in Structured Sequences: Anticipation and Habituation . . . . . . . . . . . . 9.6.2 The Stimulus Error . . . . . . . . . .

203 206 206 207 207 208 209 210 210 210 211 212 212 213 214 215 216 216 217 218 218 218 219

. . . . . . . .

. . . . . . . .

219 219 219 220 221 222 222 223

9.1 Introduction: The Relative Nature of Human Judgment This chapter will discuss context effects and common biases that can affect sensory judgments. Context effects are conditions in which the judgment about a product, usually a scaled rating, will shift depending upon factors such as the other products that are evaluated in the same tasting session. A mediocre product evaluated in the context of some poor-quality items may seem very good in comparison. Biases refer to tendencies in judgment in which the response is influenced in some way to be an inaccurate reflection of the actual sensory experience. In magnitude estimation ratings, for example, people have a tendency to use numbers that are multiples of 2, 5, and 10, even though they can use any number or fraction they wish. At the

H.T. Lawless, H. Heymann, Sensory Evaluation of Food, Food Science Text Series, DOI 10.1007/978-1-4419-6488-5_9, © Springer Science+Business Media, LLC 2010

203

204

end of the chapter, some solutions to these problems are offered, although a sensory scientist should realize that we can never totally eliminate these factors. In fact, they are of interest and deserve study on their own for what they can tell us about human sensory and cognitive processes. An axiom of perceptual psychology has it that humans are very poor absolute measuring instruments but are very good at comparing things. For example, we may have difficulty estimating the exact sweetness level of our coffee, but we have little trouble in telling whether more sugar has been added to make it sweeter. The question arises, if people are prone to making comparisons, how can they give ratings when no comparison is requested or specified? For example, when asked to rate the perceived firmness of a food sample, how do they judge what is firm versus what is soft? Obviously, they must either choose a frame of reference for the range of firmness to be judged or be trained with explicit reference standards to understand what is high and low on the response scale. In other words, they must relate this sensory judgment to other products they have tried. For many items encountered in everyday life, we have established frames of reference based on our experiences. We have no trouble forming an image of a “large mouse running up the trunk of a small elephant” because we have established frames of reference for what constitutes the average mouse and the average elephant. In this case the judgment of large and small is context dependent. Some people would argue that all judgments are relative. This dependence upon a frame of reference in making sensory judgments demonstrates the influence of contextual factors in biasing or changing how products are evaluated. We are always prone to see things against a background or pvious experience and evaluate them accordingly. A 40◦ (Fahrenheit) day in Ithaca, New York, in January seems quite mild against the background of the northeastern American winter. However, the same 40◦ C temperature will feel quite cool on an evening in August in the same location. This principle of frame of reference is the source of many visual illusions, where the same physical stimulus causes very different perceptual impssions, due to the context within which it is embedded. Examples are shown in Fig. 9.1. A simple demonstration of context is the visual afterimage effect that gave rise to Emmert’s law (Boring, 1942). In 1881, Emmert formalized a

9

Context Effects and Biases in Sensory Judgment

A

B

C Fig. 9.1 Examples of contextual effects from simple visual illusions. (a) The dumbell version of the Muller-Lyer illusion. (b) The Ebbinghaus illusion. (c) Illusory contours. In this case the contexts induce the perceptions of shapes.

principle of size constancy based on the following effect: Stare for about 30 s at a brightly illuminated colored paper rectangle (it helps to have a small dot to aid in fixation in the center) about a meter away. Then shift your gaze to a white sheet on the table in front of you. You should see the rectangle afterimage in a complementary color and somewhat smaller in size as compared to the original colored rectangle. Next, shift your gaze to a white wall some distance off. The afterimage will now appear much larger, as the brain finds a fixed visual angle at greater distance to repsent larger physical objects. Since the mind does not immediately recognize that the afterimage is just a creation of the visual sensory system, it projects it at the distance of the surface upon which it is “seen.” The more distant frame of reference, then, demands a larger size perception. The close link between sensory judgments and context psents problems for anyone who wants to view ratings as absolute or comparable across different times, sessions, or settings. Even when the actual sensory impssion of two items is the same, we can shift the frame of reference and change the overt behavior of the person to produce a different response. This problem (or principle of sensory function) was glossed

9.1 Introduction: The Relative Nature of Human Judgment

over by early psychophysical scientists. In psychological terms, they used a simple stimulus-response (S-R) model, in which response was considered a direct and unbiased repsentation of sensory experiences. Certain biases were observed, but it was felt that suitable experimental controls could minimize or eliminate them (Birnbaum, 1982; Poulton, 1989). A more modern view is that there are two or three distinct processes contributing to ratings. The first is a psychophysical process by which stimulus energy is translated into physiological events that result in a subjective experience of some sensory intensity. The second, equally important process is the function by which the subjective experience is translated into the observed response, i.e., how the percept is translated onto the rating scale (see Fig. 9.2). Many psychophysical researchers now consider a “judgment function” to

A)

Response

Stimulus

B) Stimulus

Response

Sensation Judgment Process

Psychophysical Process C) Stimulus

Sensation

Psychophysical Process

ENCODED SENSATION, “PERCEPT”

Perceptual Process

Response

Judgment Process

Fig. 9.2 Models for sensory-response processes. (a) The simple stimulus-response model of twentieth-century behavioral psychology. (b) Two processes are involved in sensation and response, a psychophysical process and then a response output or a judgment process in which the participant decides what response to give for that sensation. (c) A more complex model in which the sensation may be transformed before the response is generated. It may exist in short-term memory as an encoded percept, different from the sensation. Contextual effects of simultaneous or sequential stimuli can influence the stimulus-response sequence in several ways. Peripheral physiological effects such as adaptation or mixture inhibition may change the transduction process or other early stages of neural processing. Other stimuli may give rise to separate percepts that are integrated into the final response. Contextual factors may also influence the frame of reference that determines how the response output function will be applied. In some models, an additional step allows transformation of the percept into covert responses that are then translated as a separate step into the overt response R.

205

be an important part of the sequence from stimulus to response (Anderson, 1974, Birnbaum, 1982, McBride and Anderson, 1990; Schifferstein and Frijters, 1992; Ward, 1987). This process is also sometimes referred to as a response output function. A third intermediate step is the conversion of the raw sensory experience into some kind of encoded percept, one that is available to memory for a short time, before the judgment is made (Fig. 9.2c). Given this framework, there are several points at which stimulus context may influence the sensory process. First, of course, the actual sensation itself may change. Many sensory processes involve interaction effects of simultaneous or sequential influences of multiple items. An item may be perceived differently due to the direct influence of one stimulus upon another that is nearby in time or space. Simultaneous color contrast is an example in color vision and some types of inhibitory mixture interactions and masking in taste and smell are similarly hard wired. Quinine with added salt is less bitter than quinine tasted alone, due to the ways that sodium ions inhibit bitterness transduction. Sensory adaptation weakens the perception of a stimulus because of what has pceded it. So the psychophysical process itself is altered by the milieu in which the stimulus is observed, sometimes because of physical effects (e.g., simple buffering of an acid) or physiological effects (e.g., neural inhibition causing mixture suppssion) in the peripheral sensory mechanisms. A second point of influence is when the context shifts the frame of reference for the response output function. That is, two sensations may have the same subjective intensity under two conditions, but because of the way the observer places them along the response continuum (due to different contexts), they are rated differently. A number of studies have shown that contextual factors such as the distribution of stimuli along the physical continuum affect primarily (although not exclusively) the response output function (Mellers and Birnbaum, 1982, 1983). A third process is sometimes added in which the sensation itself is translated into an implicit response or encoded image that may also be affected by context (Fig. 9.2c). This would provide another opportunity to influence the process if contextual factors affect this encoding step. Contextual change can be viewed as a form of bias. Bias, in this sense, is a process that causes a shift or a change in response to a constant sensation. If one situation is viewed as producing a true

206

and accurate response, then the contextual conditions that cause shifts away from this accurate response are “biased.” However, bias need only have a negative connotation if there is a reason to psume that one condition of judgment is more accurate than all others. A broader view is to accept the idea that all judgments are a function of observing conditions and therefore all judgments are biased from one another in different ways. Fortunately, many of these biases and the conditions that cause them are pdictable and well understood, and so they can be eliminated or minimized. At the very least, the sensory practitioner needs to understand how these influences operate so as to know when to expect changes in judgments and ratings. An important endpoint is the realization that few, if any, ratings have any absolute meaning. You cannot say that because a product received a hedonic rating of 7.0 today, it is better than the product that received a rating of 6.5 last week. The context may have changed.

9.2 Simple Contrast Effects By far the most common effect of sensory context is simple contrast. Any stimulus will be judged as more intense in the psence of a weaker stimulus and as less intense in the psence of a stronger stimulus, all other conditions being equal. This effect is much easier to find and to demonstrate than its opposite, convergence or assimilation. For example, an early sensory worker at the Quartermaster Corp., Kamenetsky (1957), noticed that the acceptability ratings for foods seemed to depend upon what other foods were psented during an evaluation session. Poor foods seemed even worse when pceded by a good sample. Convergence is more difficult to demonstrate, although under some conditions a group of items may seem more similar to each other when they are in the psence of an item that is very different from that group (Zellner et al., 2006).

9.2.1 A Little Theory: Adaptation Level As we noted above, a 40◦ day in January (in New York) seems a lot warmer than the same temperature in August. These kinds of effects are pdicted Helson’s

9

Context Effects and Biases in Sensory Judgment

theory of adaptation level. Helson (1964) proposed that we take as a frame of reference the average level of stimulation that has pceded the item to be evaluated. The mild temperature in the middle of a hot and humid summer seems a lot more cool and refreshing than is the mild temperature after a period of cold and icy weather. So we refer to our most recent experiences in evaluating the sensory properties of an item. Helson went on to elaborate the theory to include both immediate and distant pdecessors. That is, he appciated the fact that more recent items tend to have a stronger effect on the adaptation level. Of course, mere reference to the mean value of experience is not always sufficient to induce a contrast effect-it is more influential if the mean value comes to be centered near the middle of the response scale, an example of a centering bias, discussed below (Poulton, 1989). The notion of adaptation, a decrease in responsiveness under conditions of constant stimulation, is a major theme in the literature on sensory processes. Physiological adaptation or an adjustment to the ambient level of stimulation is obvious in light/dark adaptation in vision. The thermal and tactile senses also show profound adaptation effects-we become easily adjusted to the ambient room temperature (as long as it is not too extreme) and we become unaware of tactile stimulation from our clothing. So this mean reference level often passes from consciousness or becomes a new baseline from which deviations in the environment become noticeable. Some workers have even suggested that this improves discrimination-that the difference threshold is smallest right around the adaptation level or physiological zero, in keeping with Weber’s law (McBurney, 1966). Examples of adaptation effects are discussed in Chapter 2 for the senses of taste and smell. In the chemical, thermal, and tactile senses, adaptation is quite profound. However, we need not invoke the concept of neural adaptation to a pceding item or a physiological effect to explain all contrast effects. It may be simply that more or less extreme stimuli change our frame of reference or the way in which the stimulus range and response scales are to be mapped onto one another. The general principle of context is that human observers act like measuring instruments that constantly recalibrate themselves to the experienced frame of reference. What we think of as a small horse may depend upon whether the frame of reference includes Clydesdales, Shetland ponies, or tiny phistoric equine species. The

9.2 Simple Contrast Effects

207

following examples show simple effects of context on intensity, sensory quality, and hedonics or acceptability. Most of these examples are cases of perceptual contrast or a shift in judgment away from other stimuli psented in the same session.

9.2.2 Intensity Shifts Figure 9.3 shows a simple contrast effect of soups with varying salt levels psented in different contexts (Lawless, 1983). The central stimulus in the series was psented either with two lower or two higher concentrations of salt added to a low sodium soup. Ratings of saltiness intensity were made on a simple nine-point category scale. In the lower context, the central soup received a higher rating, and in the higher context, it received a lower rating, analogous to our perception of a mild day in winter (seemingly warmer) versus a mild day in summer (seemingly cooler). Note that the shift is quite dramatic, about two points on the nine-point scale or close to 25% of scale range. A simple classroom demonstration can show a similar shift for the tactile roughness of sandpapers varying in grit size. In the context of a rougher sample, a medium sample will be rated lower than it is in the context of a smoother sample. The effects of simple contrast are not limited to taste and smell.

Mean Category Rating

7 6 5

Low Context High Context

4

3

2

.12

.18

.25

.35

.50

NaCl Concentration (log scale)

Fig. 9.3 Saltiness ratings of soups with added NaCl. The sample at 0.25 M was evaluated in two contexts, one with higher concentrations and one with lower concentrations. The shift is typical of a simple contrast effect of contrast. Replotted from Lawless (1983). Copyright ASTM, used with permission.

Contrast effects are not always observed. In some psychophysical work with long series of stimuli, some item-to-item correlations have been observed. The effects of immediately pceding versus remotely pceding stimuli have been measured and a positive correlation among adjacent responses in the series was found. This can be taken as evidence for a type of assimilation, or underestimation of differences (Ward, 1979, 1987; but see also Schifferstein and Frijters, 1992).

9.2.3 Quality Shifts Visual examples such as color contrast were well known to early psychologists like William James: “Meanwhile it is an undoubted general fact that the psychical effect of incoming currents does depend on what other currents may be simultaneously pouring in. Not only the perceptibility of the object which the current brings before the mind, but the quality of it is changed by the other currents.” (1913, p. 25). A gray line against a yellow background may appear somewhat bluish, and the same line against a blue background may seem more yellowish. Paintings of the renowned artist Josef Albers made excellent use of color contrast. Similar effects can be observed for the chemical senses. During a descriptive panel training period for fragrance evaluation, the terpene aroma compound dihydromyrcenol was psented among a set of woody or pine-like reference materials. The panelists complained that the aroma was too citrus-like to be included among the woody reference materials. However, when the same odor was placed in the context of citrus reference materials, the same panelists claimed that it was far too woody and pine-like to be included among the citrus examples. This contextual shift is shown in Fig. 9.4. In a citrus context, the item is rated as more woody in character than when placed in a woody context. Conversely, ratings for citrus intensity decrease in the citrus context and increase in the woody context. The effect is quite robust and is seen whether or not a rest period is included to undo the potential effects of sensory adaptation. It even occurs when the contextual odor follows the target item and judgments are made after both are experienced (Lawless et al., 1991)! This striking effect is discussed further in Section 9.2.5.

208

9

Context Effects and Biases in Sensory Judgment

CITRUS CONTEXT

MEAN INTENSITY RATING, DIHYDROMYRCENOL

WOODY CONTEXT Citrus/woody scales

Intensity/pleasantness

used during exposure

scales during exposure

9 8 7 6 5 4 3 2 1 CITRUS RATING WOODY RATING

CITRUS RATING WOODY RATING

ODOR CHARACTERISTIC Fig. 9.4 Odor quality contrast noted for the ambiguous terpene aroma compound dihydromyrcenol. In a citrus context, woody ratings increase and citrus character decreases. In a woody context, the woody ratings decrease. The group using different

scales did not rate citrus and woody character during the contextual exposure phase, only overall intensity and pleasantness were rated. From Lawless et al. (1991) by permission of Oxford University Press.

Context effects can also alter how items are identified and characterized. When people categorize speech sounds, repeated exposure to one type of simple phoneme changes the category boundary for other speech sounds. Repeated exposure to the sound of the phoneme “bah,” which has an early voice onset time, can shift the phoneme boundary so that speech sounds near the boundary are more likely classified as “pah” sounds (a later voice onset) (Eimas and Corbit, 1973). Boundary-level examples are shifted across the boundary and into the next category. This shift resembles a kind of contrast effect.

an item of poor quality and less appealing if it followed something of better quality. The effect was known to Beebe-Center (1932), who also attributed it to Fechner in 1898. This kind of contrast has been observed for tastes (Riskey et al., 1979; Schifferstein, 1995), odors (Sandusky and Parducci, 1965), and art (Dolese et al., 2005). Another effect observed in these kinds of experiments is that a contrasting item causes other, generally lower rated stimuli to become more similar or less discriminable, an effect termed condensation (Parker et al., 2002; Zellner et al., 2006;). In the study by Zellner et al. (2006), p-exposure to a good-tasting juice reduced the magnitude of pference ratings among less appealing juices. Mediocre items were both worse and more similar. An example of hedonic shifting was found in a study on the optimization of the saltiness of tomato juice and also the sweetness of a fruit beverage using the method of adjustment (Mattes and Lawless, 1985). When trying to optimize the level of sweetness or saltiness in this study, subjects worked in two directions. In an ascending series, they would concentrate a dilute solution by mixing the beverage with a more concentrated

9.2.4 Hedonic Shifts Changes in the pference or acceptance of foods can be seen as a function of context. Hedonic contrast was a well-known effect to early workers in food acceptance testing (Hanson et al., 1955; Kamenetzky, 1959). An item seems more appealing if it followed

9.2 Simple Contrast Effects

version having the same color, aroma, and other flavor materials (i.e., only sweetness or saltiness was different). In a second descending series, they would be given a very intense sample as the starting point and then dilute down to their pferred level. This effect is shown in Fig. 9.5. The adjustment stops too soon and the discrepancy is remarkable, nearly a concentration range of 2:1. The effect was also robust-it could not be attributed to sensory adaptation or lack of discrimination and persisted even when subjects were financially motivated to try to achieve the same endpoints in both trials. This is a case of affective contrast. When compared to a very sweet or salty starting point, a somewhat lower item seems just about right, but when starting with a relatively sour fruit beverage

Fig. 9.5 Optimized concentrations of salt in tomato juice and sucrose in a fruit beverage. In the trials labeled D, the concentration was diluted from a concentrated version of the test sample. In the trials marked A, the concentration was increased from a dilute version of the test sample. Concentrations of other ingredients were held constant. The contextual shift is consistent with reaching the apparent optimum too soon as if the apparent optimum was shifted in contrast to the starting point. From Mattes and Lawless (1985) with permission.

209

or bland tomato juice, just a little bit of sugar or salt helps quite a bit. The stopping point contrasts with the starting material and seems to be better than it would be perceived in isolation. In an ascending or descending sequence of products, a change in responses that happens too soon is called an “error of anticipation.”

9.2.5 Explanations for Contrast At first glance, one is tempted to seek a physiological explanation for contrast effects, rather than a psychological or a judgmental one. Certainly sensory adaptation to a series of intense stimuli would cause any subsequent test item to be rated much lower. The importance of sensory adaptation in the chemical senses of taste and smell lends some credence to this explanation. However, a number of studies have shown that pcautions against sensory adaptation may be taken, such as sufficient rinsing or time delays between stimuli and yet the context effects persist (Lawless et al., 1991; Mattes and Lawless, 1985; Riskey, 1982). Furthermore, it is difficult to see how sensory adaptation to low-intensity stimuli would cause an increase in the ratings for a stronger item, as adaptation necessarily causes a decrement in physiological responsiveness compared to a no-stimulation baseline. Perhaps the best evidence against a simple adaptation explanation for contrast effects is from the reversed-pair experiments in which the contextual item follows the to-be-rated target item and therefore can have no physiologically adapting effect on it. This paradigm calls for a judgment of the target item from memory after the psentation of the contextual item, in what has been termed a reversed-pair procedure (Diehl et al., 1978). Due to the reversed order, the context effects cannot be blamed on physiological adaptation of receptors, since the contextual item follows rather than pcedes the item to be rated. Reversed-pair effects are seen for shifts in odor quality of aroma compounds like dihydromyrcenol and are only slightly smaller in magnitude than the contextual shift caused when the contextual item comes first (Lawless et al., 1991). The reversed-pair situation is also quite capable of causing simple contrast effects in sensory intensity. A sweetness shift was observed when a higher or a lower sweetness item was interpolated between the tasting and rating (from memory)

210

of a normal-strength fruit beverage (Lawless, 1994). Looking back at Fig. 9.2, it seems more likely that the effect changes the response function. However, not all workers in the field agree. In particular, Marks (1994) has argued that the contextual shifts are much like an adaptation process and that for auditory stimuli this is a peripheral event. It is possible that what changes is not the sensation/experience, but to some encoded version of the sensation, or to some kind of implicit response, not yet verbalized. If a person, when rating, is evaluating some memory trace of the experience, it is possible that this memory, for example, could be altered.

9.3 Range and Frequency Effects Two of the most common factors that can affect ratings are the sensory range of the products to be evaluated and the frequency with which people use the available response options. These factors were nicely integrated into a theory that helped to explain shifts in category ratings. They are also general tendencies that can affect just about any ratings or responses.

9.3.1 A Little More Theory: Parducci’s Range and Frequency Principles Parducci (1965, 1974) sought to go beyond Helson’s (1964) simple idea that people respond to the mean or the average of their sensory experiences in determining the frame of reference for judgment. Instead, they asserted that the entire distribution of items in a psychophysical experiment would influence the judgments of a particular stimulus. If this distribution was denser (bunched up) at the low ends and a lot of weak items were psented, product ratings would shift up. Parducci (1965, 1974) proposed that behavior in a rating task was a compromise between two principles. The first was the range principle. Subjects use the categories to sub-pide the available scale range and will tend to pide the scale into equal perceptual segments. The second was the frequency principle. Over many judgments, people like to use the categories an equal number of times (Parducci, 1974). Thus it is not only

9

Context Effects and Biases in Sensory Judgment

the average level that is important but also how stimuli may be grouped or spaced along the continuum that would determine how the response scale was used. Category scaling behavior could be pdicted as a compromise between the effects of the range and frequency principles (Parducci and Perrett, 1971).

9.3.2 Range Effects The range effect has been known for some time, both in category ratings and other judgments including ratio scaling (Engen and Levy, 1958; Teghtsoonian and Teghtsoonian, 1978). When expanding or shrinking the overall range of products, subjects will map their experiences onto the available categories (Poulton, 1989). Thus short ranges produce steep psychophysical functions and wide ranges produce flatter functions. An example of this can be seen in two published experiments on rating scales (Lawless and Malone, 1986a, b). In these studies, four types of response scales and a number of visual, tactile, and olfactory continua were used to compare the abilities of consumers to use the different scales to differentiate products. In the first study, the consumers had no trouble in differentiating the products and so in the second study, the stimuli were spaced more closely on the physical continua so that the task would be more challenging. However, when the experimenters closed the stimuli in, the range principle took over, and participants used more of the rating scale than expected. In Fig. 9.6, ratings for perceived thickness of (stirred) silicone samples are shown in the wide and narrow stimulus ranges. Note the steepening of the response function. For the same one log unit change in physical viscosity, the range of responses actually doubled from the wide range to the narrower range. Another kind of stimulus range effect occurs with anchor stimuli. Sarris (1967) pviously showed a strong effect of anchor stimuli on the use of rating scales, unless the anchors were very extreme, at which point their influence would tend to diminish, as if they had become irrelevant to the judgmental frame of reference. Sarris and Parducci (1978) found similar effects of both single and multiple end anchors that generally take the form of a contrast effect. For example, a low anchor stimulus, whether rated or unrated, will cause stronger stimuli to receive higher ratings than

9.3 Range and Frequency Effects

211

Fig. 9.6 A simple range effect. When products are psented over a wide range, a shallow psychophysical function is found. Over a narrow range, a steeper psychophysical function will be observed. This is in part due to the tendency of subjects to map the products (once known) onto the available scale range. From Lawless and Malone (1986b), with permission.

they would if no anchor were psented, unless the “anchor” is so extreme as to seem irrelevant. Sarris and Parducci (1978) provided the following analogy: A salesman will consider his commissions to be larger if coworkers receive less than he does. However, he is unlikely to extend or shift his scale of judgment by hearing about others who are in an entirely different bracket of income (1978, p. 39). Whether an outlying product is similar enough to have an influence (or whether it is a “horse of a different color”) should be of concern in product comparisons of perse items.

9.3.3 Frequency Effects The frequency effect is the tendency of people to try to use the available response options about the same number of times across a series of products or stimuli to be rated. The frequency effect can cause shifts that look like simple contrast and also a local steepening of the psychophysical function around points where stimuli were closely spaced or very numerous (compared with less “dense” portions of the stimulus range). The frequency principle dictates that when judging many samples, products that are numerous or bunched at the low or high ends of the distributions tend to be spad out into neighboring categories. This is illustrated in the two panels of Fig. 9.7. The upper panel shows four hypothetical experiments and how products might be bunched in different parts of the range. In the upper left panel, we see how a normal replicated psychophysical experiment would be conducted with

equal psentations of each stimulus level. The common outcome of such a study using category ratings would be a simple linear function of the log of stimulus intensity. However, if the stimulus psentations were more frequent at the high end of the distribution, i.e., negative skew, the upper categories would be overused, and subjects would begin to distribute their judgments into lower categories. If the samples were bunched at the lower end, the lower response categories would be overused and subjects would begin to move into higher categories. If the stimuli were bunched in the midrange, the adjacent categories would be used to take on some of the middle stimuli, pushing extreme stimuli into the ends of the response range, as shown in the panel for a quasi-normal distribution. Such behavior is relevant to applied testing situations. For example, in rating the psence of off-flavors or taints, there may be very few examples of items with high values on the scale and lots of weak (or zero) sensations. The frequency effect may explain why the low end of the scale is used less often than anticipated, and higher mean values are obtained than one would deem appropriate. Another example is screening a number of flavor or fragrance candidates for a new product. A large number of good candidates are sent for testing by suppliers or a flavor development group. Presumably these have been p-tested or at least have received a favorable opinion from a flavorist or a perfumer. Why do they then get only mediocre ratings from the test panel? The high end of the distribution is overrepsented (justifiably so and perhaps on purpose), so the tendency for the panel is to drop into lower categories. This may partly explain why in-house testing

212

9

Number of Stimuli Presented

Rectangular Distribution

Quasinormal

Positive Skew

Negative Skew

Predicted Rating

number of zones or categories. The effect of grouping or spacing products also intensifies as the exposure to the distributions increases. Lawless (1983) showed that the shift that occurred with a negative skew (bunching at the upper end) into lower response categories would intensify as the exposure to the skewed distribution went from none to a single exposure to three exposures. Thus the contextual effects do not suddenly appear but will take hold of the subjects’ behavior as they gain experience with the sample set.

9.4 Biases

Rectangular Distribution

Positive Skew

Quasi-normal Distribution

Context Effects and Biases in Sensory Judgment

Negative Skew

Stimulus Intensity (log scale)

Fig. 9.7 Predictions from the Parducci range-frequency theory. Distributions of stimuli that are concentrated at one part of the perceptual range (upper quartet) will show local steepening of the psychophysical functions (lower quartet). This is due to subjects’ tendencies to use categories with equal frequency, the resulting shifting into adjacent categories from those that are overused.

panels are sometimes more critical or negative than consumers when evaluating the same items. Although the great majority of experiments on the range and frequency effects have been performed with simple visual stimuli, there are also examples from taste evaluation (Lee et al., 2001; Riskey et al., 1979; Riskey, 1982; Schifferstein and Frijters, 1992; Vollmecke, 1987). Schifferstein and Frijters found similar effects of skewed distributions with line-marking responses as seen in pvious studies with category ratings. Perhaps line marking is not a response scale with infinite pisions, but panelists sub-pide the line into discrete sub-ranges as if they were using a limited

9.4.1 Idiosyncratic Scale Usage and Number Bias People appear to have pferred ranges or numbers on the response scale that they feel comfortable using. Giovanni and Pangborn (1983) noted that people using magnitude estimation very often used numbers that were multiples of 2 and 5 (or obviously 10), an effect that is well known in the psychophysical literature (Baird and Noma, 1978). With magnitude estimation, the idiosyncratic usage of a favorite range of numbers causes a correlation of the power function exponents across different sensory continua for an inpidual (Jones and Marcus, 1961; Jones and Woskow, 1966). This correlation can be explained if people are more or less expansive (versus restrictive) in their response output functions, i.e., in how they apply numbers to their sensations in magnitude estimation studies. Another version of such personal idiosyncrasy is the common observation in time-intensity scaling that people show a kind of personal “signature” or a characteristic curve shape (Dijksterhuis, 1993; McGowan and Lee, 2006). Another version of self-induced response restriction can be seen when people use only selected portions of the scale in a line-marking rating task. On a line scale with verbal labels, people may choose to make markings only near the verbal labels, rather than distributing them across the response scale. This was first observed by Eng (1948) with a simple hedonic line scale labeled Like Very Highly at one end, Dislike Very Highly at the other, and Neither Like nor Dislike at the center. In a group of 40 consumers, 24 used only the three labeled parts of the scale, and Eng deleted them

9.4 Biases

from the data analysis! This kind of behavior was also noted with the labeled affective magnitude scale (LAM scale) by Cardello et al. (2008) with both Army laboratory and student groups. Lawless et al. (2010a) found a very high frequency (sometimes above 80%) of people making marks within ±2 mm of a phrase mark on the LAM scale in a multi-city consumer central location test. Lawless et al. (2010b) found that instructions did not seem to change this behavior much but that expanding the physical size of the scale on the ballot (from about 120 to 200 mm) decreased the “categorical” behavior somewhat. Categorical rating behavior can also be seen as a step function in time-intensity records (rather than a smooth continuous curve). Finding product differences against the background of these inpidual response tendencies can be facilitated by within-subject experimental designs. Each participant is used as his or her own baseline in comparisons of products, as in dependent t-tests or repeated measure analysis of variance in complete block designs. Another approach is to compute a difference score for a comparison of products in each inpidual’s data, rather than merely averaging across people and looking at differences between mean values.

9.4.2 Poulton’s Classifications Poulton (1989) published extensively on biases in ratings and classified them. Biases in Poulton’s system go beyond Parducci’s theory but are documented in the psychophysical literature. These include centering biases, contraction biases, logarithmic response bias with numerical ratings, and a general transfer bias that is seen when subjects carry the context from a pvious session or study into a new experiment. The centering bias is especially relevant to just-right scales and is discussed in a later section. The response range bias is also a special case and follows this section. The contraction biases are all forms of assimilation, the opposite of contrast. According to Poulton, people may rate a stimulus relative to a reference or a mean value that they hold in memory for similar types of sensory events. They tend to judge new items as being close (perhaps too close) to this reference value, causing underestimation of high values and overestimation of low values. There may also be

213

overestimation of an item when it follows a stronger standard stimulus or underestimation when it follows a weaker standard stimulus, a sort of local contraction effect. Poulton also classifies the tendency to gravitate toward the middle of the response range as a type of contraction effect, called a response contraction bias. While all of these effects undoubtedly occur, the question arises as to whether contrast or assimilation is a more common and potent process in human sensory judgment. While some evidence for response assimilation has been found in psychophysical experiments through sequential analysis of response correlations (Ward, 1979), contrast seems much more to be the rule with taste stimuli (Schifferstein and Frijters, 1992) and foods (Kamenetzky, 1959). In our experience, assimilation effects are not as pvalent as contrast effects, although assimilation has certainly been observed in experiments on consumer expectation (e.g., Cardello and Sawyer, 1992). In that case, the assimilation is not toward other actual stimuli but toward expected levels. The logarithmic response bias can be observed with open-ended response scales that use numbers, such as magnitude estimation. There are several ways to view this type of bias. Suppose that a series of stimuli have been arranged in increasing magnitude and they are spaced in subjectively equal steps. As the intensity increases, subjects change their strategy as they cross into ranges of numerical responses where there are more digits. For example, they might be rating the series using numbers like 2, 4, 6, and 8, but then when they get to 10, they will continue by larger steps, perhaps 20, 40, 60, 80. In Poulton’s view they proceed through the larger numerical responses “too rapidly.” A converse way of looking at this problem is that the perceived magnitude of the higher numbers is in smaller arithmetic steps as numbers get larger. For example, the difference between one and two seems much larger compared to the difference between 91 and 92. Poulton also points out that in addition to contraction of stimulus magnitude at very high levels, the converse is also operating and that people seem to illogically expand their subjective number range when using responses smaller than the number 3. One obvious way to avoid the problems in number bias is to avoid numbers altogether or to substitute line scaling or cross-modality matching to line length as a response instead of numerical rating techniques like magnitude estimation (Poulton, 1989).

214

Transfer bias refers to the general tendency to use pvious experimental situations and remembered judgments to calibrate oneself for later tasks. It may involve any of the biases in Poulton’s or Parducci’s theories. The situation is common when subjects are used in multiple experiments or when sensory panelists are used repeatedly in evaluations (Ward, 1987). People have memories and a desire to be internally consistent. Thus the ratings given to a product on one occasion may be influenced by ratings given to similar products on pvious occasions. There are two ways to view this tendency. One is that the judgments may not shift appropriately when the panelists’ sensory experience, perception, or opinion of the product has in fact changed. On the other hand, one of the primary functions of panelist training and calibration in descriptive analysis is to build up exactly those sorts of memory references that may stabilize sensory judgments. So there is a positive light to this tendency as well. An open question for sensory evaluation is whether exposure to one continuum of sensory intensities or one type of product will transfer contextual effects to another sensory attribute or a related set of products (Murphy, 1982; Parducci et al., 1976; Rankin and Marks, 1991). And if so, how far does the transfer extend?

9.4.3 Response Range Effects One of Poulton’s biases was called the “response range equalizing bias” in which the stimulus range is held constant but the response range changes and so do the ratings. Ratings expand or contract so that the entire range is used (minus any end-category avoidance). This is consistent with the “mapping” idea mentioned for stimulus range effects (stimuli are mapped onto the available response range). Range stabilization is implicit in the way some scaling studies have been set up and in the instructions given to subjects. This is similar to the use of physical reference standards in some descriptive analysis training (Muñoz and Civille, 1998) and is related to Sarris’s work on anchor stimuli (Sarris and Parducci, 1978). In Anderson’s work with 20-point category scales and line marking, high and low examples or end anchors are given to subjects to show them the likely range of the stimuli to be encountered. The range of responses is known since

9

Context Effects and Biases in Sensory Judgment

it is visible upon the page of the response sheet or has been p-familiarized in a practice session (Anderson, 1974). Thus it is not surprising that subjects distribute their responses across the range in a nicely graded fashion, giving the appearance that there is a reasonably linear use of the scale. Anderson noted that there are end effects that work against the use of the entire range (i.e., people tend to avoid using the endpoints) but that these can be avoided by indenting the response marks for the stimulus end anchors, for example, at points 4 and 16 on the 20-point category scale. This will provide psychological insulation against the end effects by providing a comfort zone for unexpected or extreme stimuli at the ends of the scale while leaving sufficient gradations and room to move within the interior points. The “comfort zone” idea is one reason why early workers in descriptive analysis used line scales with indented vertical marks under the anchor phrases. An exception to the response range mapping rule is seen when anchor phrases or words on a scale are noted and taken seriously by participants. An example is in Green’s work on the labeled magnitude scale, which showed a smaller response range when it was anchored to “greatest imaginable sensation” that included all oral sensations including pain, as opposed to a wider range when the greatest imaginable referred only to taste (Green et al., 1996). This also looks like an example of contrast in which the high-end anchor can evoke a kind of stimulus context, at least in the participant’s mind. If the image evoked by the high-end phrase is very extreme, it acts like a kind of stimulus that compsses ratings into a smaller range of the scale. A similar kind of response compssion was seen with the LAM scale when it was anchored to greatest imaginable liking for “sensations of any kind” as opposed to a more delimited frame such as “foods and beverages” (Cardello et al., 2008). A sensory scientist should consider how the high anchor phrase is interpted, especially if he or she wants to avoid any compssion of ratings along the response range. As Muñoz and Civille (1998) pointed out, the use of a descriptive analysis scale also depends a lot on the conceptualization of the high extreme. Does “extremely strong” refer to the strongest possible taste among all sensations and products, the strongest sensation in this product type, or just how strong this particular attribute can become in this particular product? The strongest sweetness in this product might be more intense than the strongest saltiness. The definition needs to be a

9.4 Biases

215

deliberate choice of the panel leader and an explicit instruction to the panelist to give them a uniform frame of reference.

9.4.4 The Centering Bias The centering bias arises when subjects become aware of the general level of stimulus intensity they are likely to encounter in an experiment and tend to match the center or midpoint of the stimulus range with the midpoint of the response scale. Poulton (1989) distinguished a stimulus centering bias from a response centering bias, but this distinction is primarily a function of how experiments are set up. In both cases, people tend to map the middle of the stimulus range onto the middle of the response range and otherwise ignore the anchoring implications of the verbal labels on the response scale. Note that the centering bias works against the notion that respondents can use unbalanced scales with any effectiveness. For example, the “Excellent-very good-good-fair-poor” scale commonly used in marketing research with consumers is unbalanced. The problem with unbalanced scales is that over many trials, the respondents will come to center their responses on the middle category, regardless of its verbal label. The centering bias is an important problem when there is a need to interpolate some value on a psychophysical function or to find an optimal product in just-right scaling. Poulton gives the example of McBride’s method for considering bias in the justabout-right (JAR) scale (McBride, 1982; see also Johnson and Vickers, 1987). In any series of products to be tested, say for just-right level of sweetness, there is a tendency to center the series so that the middle product will come out closest to the just-right point. The function shifts depending upon the range that is tested. One way to find the true just-right point would be to actually have the experimental series centered on that value, but then of course you would not need to do the experiment. McBride gives a method for interpolation across several experiments with different ranges. The point at which the just-right function and the median of the stimulus series will cross shows the unbiased or true just-right level. This method of interpolation is shown in Fig. 9.8. In this method, you

Fig. 9.8 Adjusting for the centering bias in just-right ratings. Three series of sucrose concentrations in lemonade were tested, a low series (2-8%), a middle series (6-14%), and a high range (10-22%). In the upper panel, the method of Poulton is used to interpolate the unbiased just-right point from the series where the midpoint concentration would correspond to the just-right item. In the lower panel, the method of McBride is used to interpolate the just-right point from a series in which the average response would correspond to the just-right point. When the average response would be just right (zero on this scale), the hypothetical stimulus range would have been centered on the just-right level. Replotted from Johnson and Vickers (1987), with permission.

psent several ranges of the products in separate sessions and plot how the judgments of the JAR point shift up and down. You can then interpolate to find the range in which the just-right point would have been from the center product in the series. This obviously takes more work to do the test a couple of times, but it could avoid a mistaken estimate of the JAR level.

216

9.5 Response Correlation and Response Restriction

9

9.5.1 Response Correlation A simple example of a halo effect is shown in Fig. 9.9. In this case, a small amount of vanilla extract was added to low-fat milk, near the threshold of perceptibility. Ratings were then collected from 19 milk consumers for sweetness, thickness, creaminess and liking for the spiked sample, and for a control milk. In spite of the lack of relationship between vanilla aroma and sweet taste and between vanilla and texture characteristics, the introduction of this one positive aspect was sufficient to cause apparent enhancement in sweetness, creaminess, and thickness ratings. Apparent enhancement of sweetness is an effect long known for ethyl maltol, a caramelization product that has an odor similar to heated sugar (Bingham

ADDED VANILLA CONTROL MILK

13

MEAN RATING + 1 S.E.M.

11

Early experimental psychologists like Thorndike (1920) noted that one very positive attribute of a person could influence judgments on other, seemingly unrelated characteristics of that inpidual. In personnel evaluations of military officers, Thorndike noted a moderate positive correlation among the inpidual rated factors. People evaluate others like this in real life. If achievement in sports is influential in our assessment of a person, we might suppose a gifted athlete to also be kind to children, generous to charities, etc., even though there is no logical relationship between these characteristics. People like to have cognitive structures that form consistent wholes and are without conflicts or contradictions (called cognitive dissonance) that can make us uncomfortable. The halo effect has also been described as a carry-over from one positive product to another (Amerine et al., 1965), but its common usage is in reference to a positive correlation of unrelated attributes (Clark and Lawless, 1994). Of course, there can also be negative or horns effects, in which one salient negative attribute causes other, unrelated attributes to be viewed or rated negatively. If a product makes a mess in the microwave, it might be rated negatively for flavor, appearance, and texture as well.

Context Effects and Biases in Sensory Judgment

*Different from

* *

9

control, paired t-test N = 19.

*

7

5

3

1

SWEET

CREAMY THICK SCALED ATTRIBUTE

LIKE

Fig. 9.9 Adding a just perceivable level of vanilla extract to low-fat milk causes increases in rated sweetness, thickness, creaminess, and liking, an example of the Halo effect. From Lawless and Clark (1994), with permission.

et al., 1990). When maltol is added to various products, sweetness ratings may rise compared to products lacking this flavor. However, the effect seems to be a case of the misattribution of olfactory stimulation to the taste sense. Murphy and Cain (1980) showed that citral (a lemon odor) could enhance taste ratings, but only when the nostrils were open, which allows diffusion of the odor into the nose and stimulation of the olfactory receptors (i.e., retronasal smell). When the nostrils are pinched shut, the diffusion is effectively eliminated and the enhancement disappears. Murphy and Cain interpted this as convincing evidence that there was no true enhancement of taste intensity by citral, but only olfactory referral, a kind of confusion between taste and smell. Studies with other odors have also shown that the taste enhancement effect from volatile flavors can be eliminated by nose pinching (Frank and Byram, 1988) even for maltol (Bingham et al., 1990). The maltol effect is also minimized by training subjects who then learn to more effectively separate or localize their odor experiences from taste (Bingham et al., 1990). The sweetness enhancement may arise as a function of conditioning or experience with the pairing of sweet tastes with some odors in foods (Stevenson et al., 1995). Several lessons can be learned from the vanilla halo effect shown in Fig. 9.9. First, untrained consumers

9.5 Response Correlation and Response Restriction

9.5.2 “Dumping” Effects: Inflation Due to Response Restriction in Profiling It is part of the folklore of consumer testing that if there is one very negative and salient attribute of a product, it will influence other attributes in a negative direction, an example of a horns effect. The effect is even worse when the salient negative attribute is omitted from the questionnaire. Omission could be due to some oversight or failure to anticipate the outcome in a consumer test or simply that it was not observed in the laboratory conditions of pliminary phases of testing. In this case, consumers will find a way to dump their frustration from not being able to report their dissatisfaction by giving negative ratings on other scales or reporting negative opinions of other even unrelated attributes. In other words, restricting responses or failure to ask a relevant question may change ratings on a number of other scales. A common version of this restriction effect can be seen in sweetness enhancement. Frank et al. (1993) found that the enhancement of sweet ratings in the psence of a fruity odor was stronger when ratings were restricted to sweetness only. When

217

both sweetness and fruitiness ratings were allowed, no enhancement of sweetness was observed. Exactly the same effect was seen for sweetness and fruitiness ratings and for sweetness and vanilla ratings (Clark and Lawless, 1994). So allowing the appropriate number of attributes can address the problem of illusory enhancement. Schifferstein (1996) gave the example of hexenol, a fresh green aroma, which when added to a strawberry flavor mixture caused mean ratings in several other scales to increase. The enhancement of the other ratings occurred only when the “green” attribute was omitted from the ballot. When the “green” attribute was included in the ballot, the response was correctly assigned to that scale, and there was no apparent enhancement in the other attributes in the aroma profile. There is good news and bad news in these observations. From a marketing perspective, ratings can be easily obtained from consumers that will show apparent sweetness enhancements if the questionnaires cleverly omit the opportunity to report on sensations other than sweetness. However, the nose pinch conditions and the use of complete sets of attributes show us that these volatile odorants such as maltol are not sweet taste enhancers but they are sweet rating enhancers. That is, they are not affecting the actual perception of sweet taste intensity but are changing the response output function or perhaps broadening the concept of sweetness to go beyond taste and include pleasant aromas as well. It would not be wise to try to use maltol to sweeten your coffee. Are there other situations in sensory testing where the dumping effect can show up? One area in which responses are usually restricted to one attribute at a time is in time-intensity scaling (Chapter 8). In a common version of this technique, the subject moves a pointer, a mouse, or some other response device to provide a continuous record of sensory intensity for a specified attribute. Usually, just one attribute at a time is rated since it is very difficult to attend continuously or even by rapid shifting of attention to more than one attribute. This would seem to be a perfect opportunity for the dumping tendency to produce illusory enhancements (e.g., Bonnans and Noble, 1993). This idea was tested in experiments with repeated category ratings across time, a time-intensity procedure that allows for ratings of multiple attributes. These studies showed sweetness enhancement in sweet-fruity mixtures when sweetness alone was rated, but little or

218

no enhancement when both sweetness and fruit intensity were rated over time (Clark and Lawless, 1994). This is exactly parallel to the sweetness enhancement results seen by Frank and colleagues. Workers using single-attribute time-intensity ratings should be wary of apparent enhancements due to response restriction.

9.5.3 Over-Partitioning In the data of van der Klaauw and Frank (1996), one can also see cases in which having too many attributes causes a deflation in ratings. As in the dumping examples, their usual paradigm was to compare the sweetness ratings of a simple sucrose solution to the same concentration with a fruity odor added. When rating sweetness only, the rating is higher than when rating sweetness and fruitiness, the common dumping effect. But when total intensity and six additional attributes were rated, the sweetness rating was significantly lower than either of the other two conditions. In another example, including a bitterness rating (in addition to the sweetness and fruitiness ratings) lowered the sweetness rating compared to rating sweetness (the highest condition) and also compared to rating sweetness and fruitiness (an intermediate sweetness rating was obtained). This effect appears to be a deflation due to people over-partitioning their sensations into too many categories. The specific choices may be important in this effect. Adding only a bitter or a bitter and a floral rating had little or no effect and dumping inflation was still observed probably because there was no fruity rating. In the study by Clark and Lawless (1994), the control condition (sweetener only) showed some evidence of a decrement when the attributes for volatiles were also available. Even more dramatic was the complete reversal of sweet enhancement to inhibition when a large number of response categories were provided for simple mixtures (Frank et al., 1993). Although this effect has not been thoroughly studied, it serves to warn the sensory scientist that the number of choices given to untrained consumers may affect the outcome and that too many choices may be as dangerous as too few. Whether this effect might be seen with trained panels remains an open question. It is sometimes difficult to pdetermine the correct number of attributes to rate in order to guard against

9

Context Effects and Biases in Sensory Judgment

the dumping effect. Careful p-testing and discussion phases in descriptive training may help. It is obviously important to be inclusive and exhaustive, but also not to waste the panelists’ time with irrelevant attributes.

9.6 Classical Psychological Errors and Other Biases A number of psychological errors in judgment have been described in the literature and are commonly listed in references in sensory evaluation (e.g., Amerine et al., 1965; Meilgaard et al., 2006). They are only briefly listed here, as they serve primarily as empirical descriptions of behavior, without much reference to cause or any theoretical bases. It is important to distinguish between description and explanation, and not to confuse naming something with trying to explain why it occurred in terms of mechanism or larger theory. The sensory evaluation practitioner needs to be aware of these errors and the conditions under which they may occur.

9.6.1 Errors in Structured Sequences: Anticipation and Habituation Two errors may be seen when a non-random sequence of products is psented for evaluation, and the observer is aware that a sequence or a particular order of items is going to be psented. The error of anticipation is said to occur when the subject shifts responses in the sequence before the sensory information would indicate that it is appropriate to do so (Mattes and Lawless, 1985). An example is found in the method of limits for thresholds, where an ascending sequence is psented and the observer expects a sensation to occur at some point and “jumps the gun.” The opposite effect is said to be the error of habituation, in which the panelist stays put too long with one pvious response, when the sensory information would indicate that a change is overdue. Obviously, the psentation of samples in random order will help to undo the expectations involved in causing the error of anticipation. Perseveration is a little bit harder to understand but may have to do with lack of attention or motivation

9.7 Antidotes

on the part of the observer or having an unusually strict criterion for changing responses. Attention and motivation can be addressed by sufficient incentives and keeping the test session from being too long.

219

The stimulus error is another classical problem in sensory measurement. This occurs when the observer knows or psumes to know the identity of the stimulus and thus draws some inference about what it should taste, smell, or look like. The judgment is biased due to expectations about stimulus identity. In the old parlor game of trying to identify the origin and vintage of a wine, stimulus error is actually a big help. It is much easier to guess the wine if you know what the host is prone to drink or you have taken a peek at the bottles in the kitchen beforehand. In sensory evaluation, the principle of blind testing and the practice using random three-digit coding of samples mitigate against the stimulus error. However, panelists are not always completely in the dark about the origin or the identity of samples. Employee panels may have a fair amount of knowledge about what is being tested and they may make inferences, correctly or incorrectly. For example, in small-plant quality assurance, workers may be aware of what types of products are being manufactured that day and these same workers may serve as sensory panelists. In the worst possible scenario, the persons drawing the samples from production are actually doing the tasting. As much as possible, these situations should be avoided. In quality control panels, insertion of blind control samples (both positive controls and flawed samples) will tend to minimize the guesswork by panelists.

the sequential effects may be counterbalanced or averaged out in the group data. The second approach is to consider the order effects of interest. In this case, different orders are analyzed as a purpose of the experiment and, if order effects are observed, they are duly noted and discussed. Whether order effects are of interest will depend upon the circumstances of the product evaluation and its goals. If counterbalanced orders or randomization cannot be worked into the experimental design, the experimenter must consider whether product differences are true sensory differences or are artifacts of stimulus order. Purposeful experimentation and analysis of order effects in at least some of the routine evaluations may give one an appciation for where and when these effects occur in the product category of interest. Another well-known order effect in acceptance testing is the reception of a higher score for the first sample in a series (Kofes et al., 2009). Counterbalancing orders is of course appropriate, but one can also give a “dummy” product first to absorb the first product’s s


【#2】Luật Chất Lượng, Sản Phẩm Hàng Hóa Năm 2007

chúng tôi độc quyền cung cấp bản dịch chính thống Công báo tiếng Anh của Thông Tấn Xã Việt Nam

Xem có chú thích thay đổi nội dung

Căn cứ Hiến pháp nước Cộng hoà xã hội chủ nghĩa Việt Nam năm 1992 đã được sửa đổi, bổ sung một số điều theo Nghị quyết số 51/2001/QH10;

Quốc hội ban hành Luật chất lượng sản phẩm, hàng hoá.

Chương I NHỮNG QUY ĐỊNH CHUNG

1. Sản phẩm là kết quả của quá trình sản xuất hoặc cung ứng dịch vụ nhằm mục đích kinh doanh hoặc tiêu dùng.

2. Hàng hoá là sản phẩm được đưa vào thị trường, tiêu dùng thông qua trao đổi, mua bán, tiếp thị.

3. Sản phẩm, hàng hoá không có khả năng gây mất an toàn (sau đây gọi là sản phẩm, hàng hoá nhóm 1) là sản phẩm, hàng hoá trong điều kiện vận chuyển, lưu giữ, bảo quản, sử dụng hợp lý và đúng mục đích, không gây hại cho người, động vật, thực vật, tài sản, môi trường.

4. Sản phẩm, hàng hoá có khả năng gây mất an toàn (sau đây gọi là sản phẩm, hàng hoá nhóm 2) là sản phẩm, hàng hoá trong điều kiện vận chuyển, lưu giữ, bảo quản, sử dụng hợp lý và đúng mục đích, vẫn tiềm ẩn khả năng gây hại cho người, động vật, thực vật, tài sản, môi trường.

5. Chất lượng sản phẩm, hàng hóa là mức độ của các đặc tính của sản phẩm, hàng hóa đáp ứng yêu cầu trong tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng.

6. Tổ chức, cá nhân sản xuất, kinh doanh là tổ chức, cá nhân tổ chức và thực hiện việc sản xuất (sau đây gọi là người sản xuất), nhập khẩu (sau đây gọi là người nhập khẩu), xuất khẩu (sau đây gọi là người xuất khẩu), bán hàng, cung cấp dịch vụ (sau đây gọi là người bán hàng).

8. Tổ chức đánh giá sự phù hợp là tổ chức tiến hành hoạt động thử nghiệm, giám định, kiểm định, chứng nhận sự phù hợp của sản phẩm, hàng hoá, quá trình sản xuất, cung ứng dịch vụ phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng.

9. Tổ chức đánh giá sự phù hợp được chỉ định là tổ chức đáp ứng các điều kiện theo quy định của pháp luật và được cơ quan quản lý nhà nước có thẩm quyền xem xét, quyết định công bố danh sách để tổ chức, cá nhân sản xuất, kinh doanh lựa chọn sử dụng dịch vụ đánh giá sự phù hợp phục vụ yêu cầu quản lý nhà nước.

10. Thử nghiệm là thao tác kỹ thuật nhằm xác định một hay nhiều đặc tính của sản phẩm, hàng hóa theo một quy trình nhất định.

11. Giám định là việc xem xét sự phù hợp của sản phẩm, hàng hóa so với hợp đồng hoặc tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng bằng cách quan trắc và đánh giá kết quả đo, thử nghiệm.

12. Chứng nhận là việc đánh giá và xác nhận sự phù hợp của sản phẩm, quá trình sản xuất, cung ứng dịch vụ với tiêu chuẩn công bố áp dụng (gọi là chứng nhận hợp chuẩn) hoặc với quy chuẩn kỹ thuật (gọi là chứng nhận hợp quy).

13. Kiểm định là hoạt động kỹ thuật theo một quy trình nhất định nhằm đánh giá và xác nhận sự phù hợp của sản phẩm, hàng hoá với yêu cầu quy định trong quy chuẩn kỹ thuật tương ứng.

14. Thừa nhận kết quả đánh giá sự phù hợp là việc cơ quan nhà nước có thẩm quyền, tổ chức, cá nhân hoạt động tại Việt Nam chấp nhận kết quả đánh giá sự phù hợp do tổ chức đánh giá sự phù hợp của quốc gia, vùng lãnh thổ khác thực hiện.

15. Kiểm tra nhà nước về chất lượng sản phẩm, hàng hoá (sau đây gọi là kiểm tra chất lượng sản phẩm, hàng hóa) là việc cơ quan nhà nước xem xét, đánh giá lại chất lượng sản phẩm, hàng hoá, quá trình sản xuất, cung ứng dịch vụ đã được đánh giá chất lượng bởi các tổ chức đánh giá sự phù hợp hoặc đã được áp dụng các biện pháp quản lý chất lượng khác của các tổ chức, cá nhân sản xuất, kinh doanh.

16. Cơ quan có trách nhiệm thực hiện việc kiểm tra nhà nước về chất lượng sản phẩm, hàng hoá (sau đây gọi là cơ quan kiểm tra chất lượng sản phẩm, hàng hoá) là cơ quan được phân công, phân cấp thực hiện nhiệm vụ quản lý nhà nước về chất lượng sản phẩm, hàng hoá thuộc Bộ quản lý ngành, lĩnh vực, cơ quan chuyên môn thuộc Uỷ ban nhân dân tỉnh, thành phố trực thuộc trung ương.

Điều 4. Áp dụng pháp luật

3. Trong trường hợp điều ước quốc tế mà Cộng hòa xã hội chủ nghĩa Việt Nam là thành viên có quy định khác với quy định của Luật này thì áp dụng quy định của điều ước quốc tế đó.

Điều 5. Nguyên tắc quản lý chất lượng sản phẩm, hàng hoá

1. Chất lượng sản phẩm, hàng hoá được quản lý trên cơ sở tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng. Căn cứ vào khả năng gây mất an toàn, sản phẩm, hàng hoá được quản lý như sau:

a) Sản phẩm, hàng hoá nhóm 1 được quản lý chất lượng trên cơ sở tiêu chuẩn do người sản xuất công bố áp dụng;

2. Quản lý chất lượng sản phẩm, hàng hoá là trách nhiệm của người sản xuất, kinh doanh nhằm bảo đảm an toàn cho người, động vật, thực vật, tài sản, môi trường; nâng cao năng suất, chất lượng và khả năng cạnh tranh của sản phẩm, hàng hoá Việt Nam.

1. Khuyến khích tổ chức, cá nhân xây dựng và áp dụng tiêu chuẩn tiên tiến cho sản phẩm, hàng hoá và công tác quản lý, điều hành sản xuất, kinh doanh.

2. Xây dựng chương trình quốc gia nâng cao năng suất, chất lượng và khả năng cạnh tranh của sản phẩm, hàng hóa.

3. Đầu tư, phát triển hệ thống thử nghiệm đáp ứng yêu cầu sản xuất, kinh doanh và quản lý nhà nước về chất lượng sản phẩm, hàng hoá.

4. Đẩy mạnh việc đào tạo, bồi dưỡng nguồn nhân lực phục vụ hoạt động quản lý chất lượng sản phẩm, hàng hoá.

5. Tuyên truyền, phổ biến chính sách, pháp luật về chất lượng sản phẩm, hàng hoá; xây dựng ý thức sản xuất, kinh doanh sản phẩm, hàng hoá có chất lượng, vì quyền lợi người tiêu dùng, tiết kiệm năng lượng, thân thiện môi trường; nâng cao nhận thức xã hội về tiêu dùng, xây dựng tập quán tiêu dùng văn minh.

6. Khuyến khích, tạo điều kiện cho tổ chức, cá nhân trong nước và tổ chức, cá nhân nước ngoài đầu tư, tham gia vào hoạt động thử nghiệm, giám định, kiểm định, chứng nhận chất lượng sản phẩm, hàng hóa.

Điều 7. Giải thưởng chất lượng sản phẩm, hàng hoá

1. Giải thưởng chất lượng sản phẩm, hàng hoá bao gồm Giải thưởng chất lượng quốc gia và giải thưởng của tổ chức, cá nhân.

2. Điều kiện, thủ tục xét tặng Giải thưởng chất lượng quốc gia do Chính phủ quy định.

3. Điều kiện, thủ tục xét tặng giải thưởng chất lượng sản phẩm, hàng hoá của tổ chức, cá nhân do Bộ Khoa học và Công nghệ quy định.

Điều 8. Những hành vi bị nghiêm cấm

1. Sản xuất sản phẩm, nhập khẩu, mua bán hàng hoá đã bị Nhà nước cấm lưu thông.

2. Sản xuất sản phẩm, xuất khẩu, nhập khẩu, mua bán hàng hóa, trao đổi, tiếp thị sản phẩm, hàng hoá không bảo đảm quy chuẩn kỹ thuật tương ứng.

3. Xuất khẩu, nhập khẩu, mua bán hàng hoá không có nguồn gốc rõ ràng.

4. Xuất khẩu, nhập khẩu, mua bán hàng hoá, trao đổi, tiếp thị sản phẩm, hàng hoá đã hết hạn sử dụng.

5. Dùng thực phẩm, dược phẩm không bảo đảm chất lượng hoặc đã hết hạn sử dụng làm từ thiện hoặc cho, tặng để sử dụng cho người.

6. Cố tình cung cấp sai hoặc giả mạo kết quả thử nghiệm, kiểm tra, giám định, kiểm định, chứng nhận chất lượng sản phẩm, hàng hóa.

7. Giả mạo hoặc sử dụng trái phép dấu hợp chuẩn, dấu hợp quy, các dấu hiệu khác về chất lượng sản phẩm, hàng hóa.

8. Thay thế, đánh tráo, thêm, bớt thành phần hoặc chất phụ gia, pha trộn tạp chất làm giảm chất lượng sản phẩm, hàng hóa so với tiêu chuẩn công bố áp dụng hoặc quy chuẩn kỹ thuật tương ứng.

10. Che giấu thông tin về khả năng gây mất an toàn của sản phẩm, hàng hoá đối với ng­ười, động vật, thực vật, tài sản, môi tr­ường.

11. Sản xuất, chế biến sản phẩm, hàng hoá bằng nguyên liệu, vật liệu cấm sử dụng để sản xuất, chế biến sản phẩm, hàng hoá đó.

12. Lợi dụng chức vụ, quyền hạn hoặc hoạt động quản lý chất lượng sản phẩm, hàng hóa để cản trở bất hợp pháp, gây phiền hà, sách nhiễu đối với hoạt động sản xuất, kinh doanh của tổ chức, cá nhân hoặc bao che hành vi vi phạm pháp luật về chất lượng sản phẩm, hàng hoá.

13. Lợi dụng hoạt động quản lý chất lượng sản phẩm, hàng hóa để gây phương hại cho lợi ích quốc gia, trật tự, an toàn xã hội.

Chương II QUYỀN VÀ NGHĨA VỤ CỦA TỔ CHỨC, CÁ NHÂN ĐỐI VỚI CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Mục 1 QUYỀN VÀ NGHĨA VỤ CỦA TỔ CHỨC, CÁ NHÂN SẢN XUẤT, KINH DOANH

Điều 9. Quyền của người sản xuất

1. Quyết định và công bố mức chất lượng sản phẩm do mình sản xuất, cung cấp.

2. Quyết định các biện pháp kiểm soát nội bộ để bảo đảm chất lượng sản phẩm.

3. Lựa chọn tổ chức đánh giá sự phù hợp để thử nghiệm, kiểm định, giám định, chứng nhận chất lượng sản phẩm, hàng hóa.

Trường hợp chứng nhận hợp quy, kiểm tra chất lượng sản phẩm theo yêu cầu quản lý nhà nước thì người sản xuất lựa chọn tổ chức đánh giá sự phù hợp được chỉ định.

4. Sử dụng dấu hợp chuẩn, dấu hợp quy và các dấu hiệu khác cho sản phẩm theo quy định của pháp luật.

5. Yêu cầu người bán hàng hợp tác trong việc thu hồi và xử lý hàng hóa không bảo đảm chất lượng.

6. Khiếu nại kết luận của đoàn kiểm tra, quyết định của cơ quan kiểm tra, cơ quan quản lý nhà nước có thẩm quyền.

Điều 10. Nghĩa vụ của người sản xuất

1. Tuân thủ các điều kiện bảo đảm chất lượng đối với sản phẩm trước khi đưa ra thị trường theo quy định tại Điều 28 của Luật này và chịu trách nhiệm về chất lượng sản phẩm do mình sản xuất.

2. Thể hiện các thông tin về chất lượng trên nhãn hàng hóa, bao bì, trong tài liệu kèm theo hàng hóa theo quy định của pháp luật về nhãn hàng hóa.

3. Thông tin trung thực về chất lượng sản phẩm, hàng hoá.

4. Cảnh báo về khả năng gây mất an toàn của sản phẩm và cách phòng ngừa cho người bán hàng và người tiêu dùng.

5. Thông báo yêu cầu về vận chuyển, lưu giữ, bảo quản, sử dụng sản phẩm, hàng hóa.

6. Cung cấp thông tin về việc bảo hành và thực hiện việc bảo hành sản phẩm, hàng hóa cho người mua, người tiêu dùng.

7. Sửa chữa, hoàn lại hoặc đổi hàng mới, nhận lại hàng có khuyết tật bị người bán hàng, người tiêu dùng trả lại.

9. Thu hồi, xử lý sản phẩm, hàng hóa không bảo đảm chất lượng. Trong trường hợp phải tiêu huỷ hàng hóa thì phải chịu toàn bộ chi phí cho việc tiêu huỷ hàng hoá và chịu trách nhiệm về hậu quả của việc tiêu huỷ hàng hoá theo quy định của pháp luật.

11. Tuân thủ các quy định, quyết định về thanh tra, kiểm tra của cơ quan nhà nước có thẩm quyền.

12. Trả chi phí thử nghiệm, chứng nhận hợp chuẩn, chứng nhận hợp quy theo quy định tại Điều 31; chi phí lấy mẫu, thử nghiệm theo quy định tại Điều 41; chi phí lấy mẫu, thử nghiệm, giám định theo quy định tại Điều 58 của Luật này.

13. Chứng minh kết quả sai và lỗi của tổ chức đánh giá sự phù hợp theo quy định tại khoản 2 Điều 63 của Luật này.

Điều 11. Quyền của người nhập khẩu

1. Quyết định lựa chọn mức chất lượng của hàng hoá do mình nhập khẩu.

2. Yêu cầu người xuất khẩu cung cấp hàng hoá đúng chất lượng đã thoả thuận theo hợp đồng.

3. Lựa chọn tổ chức giám định để giám định chất lượng hàng hoá do mình nhập khẩu.

4. Sử dụng dấu hợp chuẩn, dấu hợp quy và các dấu hiệu khác cho hàng hoá nhập khẩu theo quy định.

5. Quyết định các biện pháp kiểm soát nội bộ để duy trì chất lượng sản phẩm, hàng hoá do mình nhập khẩu.

6. Yêu cầu người bán hàng hợp tác trong việc thu hồi và xử lý hàng hóa không bảo đảm chất lượng.

7. Khiếu nại kết luận của kiểm soát viên chất lượng, đoàn kiểm tra, quyết định của cơ quan kiểm tra, cơ quan quản lý nhà nước có thẩm quyền.

Điều 12. Nghĩa vụ của người nhập khẩu

1. Tuân thủ các điều kiện bảo đảm chất lượng đối với hàng hoá nhập khẩu theo quy định tại Điều 34 của Luật này.

2. Chịu trách nhiệm về chất lượng và ghi nhãn hàng hoá theo quy định của pháp luật đối với hàng hóa do mình nhập khẩu.

3. Thông tin trung thực về chất lượng sản phẩm, hàng hoá.

4. Tổ chức và kiểm soát quá trình vận chuyển, lưu giữ, bảo quản để duy trì chất lượng hàng hóa.

5. Thông báo điều kiện phải thực hiện khi vận chuyển, lưu giữ, bảo quản hàng hoá theo quy định của pháp luật.

6. Cảnh báo về khả năng gây mất an toàn của hàng hóa và cách phòng ngừa cho người bán hàng và người tiêu dùng.

7. Cung cấp thông tin về việc bảo hành và thực hiện việc bảo hành hàng hóa cho người bán hàng, người tiêu dùng.

8. Sửa chữa, hoàn lại hoặc đổi hàng mới, nhận lại hàng có khuyết tật bị người bán hàng trả lại.

10. Tái xuất hàng hóa nhập khẩu không phù hợp với quy chuẩn kỹ thuật tương ứng.

11. Tiêu hủy hàng hóa nhập khẩu không phù hợp quy chuẩn kỹ thuật tương ứng nhưng không tái xuất được; chịu toàn bộ chi phí cho việc tiêu huỷ hàng hoá và chịu trách nhiệm về hậu quả của việc tiêu huỷ hàng hoá theo quy định của pháp luật.

12. Thu hồi, xử lý hàng hóa không bảo đảm chất lượng.

14. Tuân thủ các quy định, quyết định về thanh tra, kiểm tra của cơ quan nhà nước có thẩm quyền.

15. Trả chi phí, lệ phí phục vụ kiểm tra chất lượng hàng hoá nhập khẩu theo quy định tại Điều 37; chi phí lấy mẫu, thử nghiệm theo quy định tại Điều 41; chi phí lấy mẫu, thử nghiệm, giám định theo quy định tại Điều 58 của Luật này.

Điều 13. Quyền của người xuất khẩu

1. Quyết định lựa chọn mức chất lượng của hàng hoá xuất khẩu.

2. Lựa chọn tổ chức đánh giá sự phù hợp để thử nghiệm, giám định, chứng nhận chất lượng hàng hóa xuất khẩu.

3. Quyết định các biện pháp kiểm soát nội bộ để duy trì chất lượng hàng hoá cho đến thời điểm chuyển giao quyền sở hữu hàng hoá đó cho người nhập khẩu.

4. Sử dụng dấu hợp chuẩn, dấu hợp quy và các dấu hiệu khác cho hàng hoá xuất khẩu theo quy định.

5. Yêu cầu người nhập khẩu hàng hoá hợp tác trong việc thu hồi và xử lý hàng hóa không bảo đảm chất lượng theo thoả thuận.

6. Khiếu nại quyết định của cơ quan kiểm tra, cơ quan quản lý nhà nước có thẩm quyền.

Điều 14. Nghĩa vụ của người xuất khẩu

1. Tuân thủ các điều kiện bảo đảm chất lượng đối với hàng hoá xuất khẩu theo quy định tại Điều 32 của Luật này và chịu trách nhiệm về chất lượng hàng hoá.

2. Thực hiện các biện pháp xử lý hàng hoá xuất khẩu không phù hợp theo quy định tại Điều 33 của Luật này. Trong trường hợp phải tiêu huỷ hàng hóa thì phải chịu toàn bộ chi phí cho việc tiêu huỷ hàng hoá và chịu trách nhiệm về hậu quả của việc tiêu huỷ hàng hoá theo quy định của pháp luật.

3. Tuân thủ các quy định, quyết định về thanh tra, kiểm tra của cơ quan nhà nước có thẩm quyền.

4. Trả chi phí thử nghiệm, chứng nhận hợp chuẩn, chứng nhận hợp quy theo quy định tại Điều 31, chi phí lấy mẫu, thử nghiệm theo quy định tại Điều 41 và chi phí lấy mẫu, thử nghiệm, giám định theo quy định tại Điều 58 của Luật này.

Điều 15. Quyền của người bán hàng

1. Quyết định cách thức kiểm tra chất lượng hàng hoá.

2. Lựa chọn tổ chức đánh giá sự phù hợp để thử nghiệm, giám định hàng hoá.

3. Quyết định các biện pháp kiểm soát nội bộ để duy trì chất lượng hàng hoá.

4. Được giải quyết tranh chấp theo quy định tại Mục 1 Chương V của Luật này và yêu cầu người sản xuất, người nhập khẩu đã cung cấp hàng hoá bồi thường thiệt hại theo quy định tại khoản 1 Điều 61 của Luật này.

5. Khiếu nại kết luận của kiểm soát viên chất lượng, đoàn kiểm tra và quyết định của cơ quan kiểm tra, cơ quan quản lý nhà nước có thẩm quyền.

Điều 16. Nghĩa vụ của người bán hàng

1. Tuân thủ các điều kiện bảo đảm chất lượng đối với hàng hoá lưu thông trên thị trường theo quy định tại Điều 38 của Luật này và chịu trách nhiệm về chất lượng hàng hóa.

3. Thông tin trung thực về chất lượng sản phẩm, hàng hoá.

4. Áp dụng các biện pháp để duy trì chất lượng hàng hoá trong vận chuyển, lưu giữ, bảo quản.

5. Thông báo cho người mua điều kiện phải thực hiện khi vận chuyển, lưu giữ, bảo quản và sử dụng hàng hoá.

6. Cung cấp thông tin về việc bảo hành hàng hóa cho người mua.

7. Cung cấp tài liệu, thông tin về hàng hoá bị kiểm tra cho kiểm soát viên chất lượng, đoàn kiểm tra, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá.

8. Kịp thời cung cấp thông tin về nguy cơ gây mất an toàn của hàng hoá và cách phòng ngừa cho người mua khi nhận được thông tin cảnh báo từ người sản xuất, người nhập khẩu.

9. Kịp thời dừng bán hàng, thông tin cho người sản xuất, người nhập khẩu và người mua khi phát hiện hàng hoá gây mất an toàn hoặc hàng hoá không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng.

10. Hoàn lại tiền hoặc đổi hàng mới, nhận lại hàng có khuyết tật bị người mua trả lại.

11. Hợp tác với người sản xuất, người nhập khẩu thu hồi, xử lý hàng hóa không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng.

13. Tuân thủ các quy định, quyết định về thanh tra, kiểm tra của cơ quan nhà nước có thẩm quyền.

14. Trả chi phí lấy mẫu, thử nghiệm theo quy định tại Điều 41; chi phí lấy mẫu, thử nghiệm, giám định theo quy định tại Điều 58 của Luật này.

Mục 2 QUYỀN VÀ NGHĨA VỤ CỦA NGƯỜI TIÊU DÙNG

Điều 17. Quyền của người tiêu dùng

1. Được cung cấp thông tin trung thực về mức độ an toàn, chất lượng, hướng dẫn vận chuyển, lưu giữ, bảo quản, sử dụng sản phẩm, hàng hoá.

2. Được cung cấp thông tin về việc bảo hành hàng hóa, khả năng gây mất an toàn của hàng hoá và cách phòng ngừa khi nhận được thông tin cảnh báo từ người sản xuất, người nhập khẩu.

3. Yêu cầu người bán hàng sửa chữa, hoàn lại tiền hoặc đổi hàng mới, nhận lại hàng có khuyết tật.

5. Yêu cầu tổ chức, cá nhân sản xuất, kinh doanh hàng hóa thực hiện trách nhiệm về bảo vệ quyền lợi người tiêu dùng theo quy định của pháp luật về bảo vệ quyền lợi người tiêu dùng.

6. Yêu cầu tổ chức bảo vệ quyền lợi người tiêu dùng trợ giúp bảo vệ quyền lợi hợp pháp của mình theo quy định của pháp luật về bảo vệ quyền lợi người tiêu dùng.

Điều 18. Nghĩa vụ của người tiêu dùng

1. Tuân thủ các điều kiện bảo đảm chất lượng đối với hàng hoá trong quá trình sử dụng theo quy định tại Điều 42 của Luật này.

2. Tuân thủ quy định và hướng dẫn của người sản xuất, người nhập khẩu, người bán hàng về việc vận chuyển, lưu giữ, bảo quản, sử dụng sản phẩm, hàng hóa.

4. Tuân thủ quy định của pháp luật về bảo vệ môi trường trong quá trình sử dụng sản phẩm, hàng hoá.

Mục 3 QUYỀN VÀ NGHĨA VỤ CỦA TỔ CHỨC ĐÁNH GIÁ SỰ PHÙ HỢP, TỔ CHỨC NGHỀ NGHIỆP VÀ TỔ CHỨC BẢO VỆ QUYỀN LỢI NGƯỜI TIÊU DÙNG

Điều 19. Quyền của tổ chức đánh giá sự phù hợp

1. Tiến hành thử nghiệm, giám định, kiểm định, chứng nhận chất lượng sản phẩm, hàng hóa trên cơ sở hợp đồng ký kết với tổ chức, cá nhân đề nghị đánh giá sự phù hợp trong lĩnh vực đã đăng ký hoạt động hoặc được cơ quan nhà nước có thẩm quyền chỉ định.

2. Được thanh toán chi phí theo thoả thuận với các tổ chức, cá nhân sản xuất, kinh doanh sản phẩm, hàng hoá có nhu cầu đánh giá sự phù hợp hoặc theo yêu cầu của cơ quan nhà nước có thẩm quyền.

3. Cung cấp kết quả thử nghiệm cho đối tượng được đánh giá sự phù hợp tương ứng.

4. Cấp, cấp lại, mở rộng, thu hẹp phạm vi hoặc tạm đình chỉ, thu hồi giấy chứng nhận sự phù hợp, quyền sử dụng dấu hợp chuẩn, dấu hợp quy đã cấp cho các đối tượng được giám định hoặc chứng nhận tương ứng.

6. Được cơ quan nhà nước có thẩm quyền thừa nhận kết quả đánh giá sự phù hợp theo quy định của pháp luật.

7. Thu chi phí thử nghiệm, chứng nhận hợp chuẩn, chứng nhận hợp quy theo quy định tại Điều 31; thu chi phí, lệ phí phục vụ kiểm tra chất lượng hàng hoá nhập khẩu theo quy định tại Điều 37; thu chi phí thử nghiệm theo quy định tại Điều 41; thu chi phí thử nghiệm, giám định theo quy định tại Điều 58 của Luật này.

Điều 20. Nghĩa vụ của tổ chức đánh giá sự phù hợp

1. Đáp ứng điều kiện theo quy định tại khoản 5 Điều 25 của Luật này.

2. Không được từ chối cung cấp dịch vụ khi không có lý do chính đáng.

3. Bảo mật các thông tin, số liệu, kết quả quả đánh giá sự phù hợp của tổ chức được đánh giá sự phù hợp, trừ trường hợp được cơ quan nhà nước có thẩm quyền yêu cầu.

5. Bảo đảm trình tự, thủ tục đánh giá sự phù hợp theo quy định của pháp luật về tiêu chuẩn và quy chuẩn kỹ thuật.

6. Thông báo trên các phương tiện thông tin đại chúng về việc cấp, cấp lại, mở rộng, thu hẹp phạm vi hoặc tạm đình chỉ, thu hồi giấy chứng nhận sự phù hợp và quyền sử dụng dấu hợp chuẩn, dấu hợp quy.

7. Chịu sự kiểm tra, thanh tra của cơ quan nhà nước có thẩm quyền về hoạt động đánh giá sự phù hợp.

8. Chịu trách nhiệm trước pháp luật về kết quả đánh giá sự phù hợp.

9. Trả tiền phạt cho tổ chức, cá nhân có sản phẩm, hàng hóa được đánh giá trong trường hợp cung cấp sai kết quả đánh giá sự phù hợp. Mức phạt do các bên thỏa thuận, nhưng không vượt quá 10 lần chi phí đánh giá, trường hợp các bên không thoả thuận được thì mức phạt do trọng tài hoặc toà án quyết định, nhưng không vượt quá 10 lần chi phí đánh giá.

10. Bồi thường thiệt hại theo quy định tại khoản 1 Điều 63 của Luật này.

Điều 21. Quyền và nghĩa vụ của tổ chức nghề nghiệp

1. Tuyên truyền, phổ biến kiến thức cho các tổ chức, cá nhân sản xuất, kinh doanh trong việc áp dụng pháp luật về chất lượng sản phẩm, hàng hóa; xây dựng ý thức sản xuất, kinh doanh sản phẩm, hàng hoá có chất lượng, vì quyền lợi người tiêu dùng, tiết kiệm năng lượng, thân thiện môi trường; nâng cao nhận thức xã hội về tiêu dùng, xây dựng tập quán tiêu dùng văn minh.

2. Hỗ trợ, nâng cao nhận thức và vận động tổ chức, cá nhân sản xuất, kinh doanh áp dụng pháp luật về chất lượng sản phẩm, hàng hóa.

3. Đào tạo, bồi dưỡng về phương thức quản lý chất lượng sản phẩm, hàng hóa và phản biện xã hội trong hoạt động quản lý chất lượng sản phẩm, hàng hóa.

4. Góp ý xây dựng các văn bản quy phạm pháp luật về chất lượng sản phẩm, hàng hóa.

5. Khiếu nại, khởi kiện trong tranh chấp về chất lượng sản phẩm, hàng hoá gây thiệt hại cho thành viên, tổ chức nghề nghiệp.

Điều 22. Quyền và nghĩa vụ của các tổ chức bảo vệ quyền lợi người tiêu dùng

1. Đại diện cho người tiêu dùng bảo vệ quyền và lợi ích hợp pháp của họ khi nhận được khiếu nại, phản ánh về chất lượng hàng hóa không phù hợp với tiêu chuẩn đã công bố áp dụng, quy chuẩn kỹ thuật tương ứng, định lượng ghi trên nhãn hoặc không bảo đảm chất lượng theo hợp đồng.

3. Kiến nghị cơ quan kiểm tra, thanh tra, cơ quan nhà nước có thẩm quyền xử lý hoặc giải quyết các vi phạm của tổ chức, cá nhân sản xuất, kinh doanh về chất lượng sản phẩm, hàng hóa.

4. Khiếu nại, khởi kiện tranh chấp về chất lượng sản phẩm, hàng hoá gây thiệt hại cho người tiêu dùng.

Chương III QUẢN LÝ CHẤT LƯỢNG SẢN PHẨM, HÀNG HÓA TRONG SẢN XUẤT, XUẤT KHẨU, NHẬP KHẨU, LƯU THÔNG TRÊN THỊ TRƯỜNG VÀ TRONG QUÁ TRÌNH SỬ DỤNG

Mục 1 QUY ĐỊNH CHUNG VỀ BIỆN PHÁP QUẢN LÝ CHẤT LƯỢNG SẢN PHẨM, HÀNG HÓA

Điều 23. Công bố tiêu chuẩn áp dụng

1. Người sản xuất, người nhập khẩu tự công bố các đặc tính cơ bản, thông tin cảnh báo, số hiệu tiêu chuẩn trên hàng hoá hoặc một trong các phương tiện sau đây:

c) Tài liệu kèm theo sản phẩm, hàng hoá.

2. Nội dung của tiêu chuẩn công bố áp dụng không được trái với yêu cầu của quy chuẩn kỹ thuật do cơ quan quản lý nhà nước có thẩm quyền ban hành.

Điều 24. Công bố sự phù hợp

1. Người sản xuất thông báo sản phẩm của mình phù hợp với tiêu chuẩn (sau đây gọi là công bố hợp chuẩn) hoặc với quy chuẩn kỹ thuật (sau đây gọi là công bố hợp quy).

2. Việc công bố hợp chuẩn, công bố hợp quy được thực hiện theo quy định của pháp luật về tiêu chuẩn và quy chuẩn kỹ thuật.

Điều 25. Đánh giá sự phù hợp

1. Việc thử nghiệm được quy định như sau:

a) Thử nghiệm phục vụ hoạt động của tổ chức, cá nhân sản xuất, kinh doanh được thực hiện theo thoả thuận với tổ chức thử nghiệm;

b) Thử nghiệm phục vụ quản lý nhà nước được thực hiện tại phòng thử nghiệm được chỉ định.

2. Việc giám định được quy định như sau:

a) Giám định phục vụ mục đích thương mại do tổ chức giám định thực hiện theo thỏa thuận với tổ chức, cá nhân có nhu cầu giám định;

b) Việc giám định phục vụ quản lý nhà nước do tổ chức giám định được chỉ định thực hiện.

3. Việc chứng nhận được quy định như sau:

a) Chứng nhận hợp chuẩn được thực hiện theo thỏa thuận của tổ chức, cá nhân có nhu cầu chứng nhận với tổ chức chứng nhận;

b) Việc chứng nhận hợp quy do tổ chức chứng nhận được chỉ định thực hiện.

4. Việc kiểm định được quy định như sau:

a) Kiểm định bao gồm kiểm định định kỳ, kiểm định bất thường;

b) Việc kiểm định phải do tổ chức kiểm định được chỉ định thực hiện.

5. Tổ chức đánh giá sự phù hợp phải đáp ứng các điều kiện sau đây:

a) Có tổ chức và năng lực đáp ứng yêu cầu chung đối với tổ chức đánh giá sự phù hợp quy định trong tiêu chuẩn quốc gia, tiêu chuẩn quốc tế tương ứng;

b) Thiết lập và duy trì hệ thống quản lý phù hợp với yêu cầu quy định trong tiêu chuẩn quốc gia, tiêu chuẩn quốc tế tương ứng;

c) Đăng ký lĩnh vực hoạt động đánh giá sự phù hợp tại cơ quan quản lý nhà nước có thẩm quyền.

Điều 26. Thừa nhận kết quả đánh giá sự phù hợp

1. Việc thừa nhận kết quả đánh giá sự phù hợp giữa tổ chức, cá nhân tại Việt Nam với tổ chức, cá nhân nước ngoài, vùng lãnh thổ do các bên tự thoả thuận.

2. Việc thừa nhận kết quả đánh giá sự phù hợp phục vụ quản lý nhà nước được thực hiện theo điều ước quốc tế mà Cộng hoà xã hội chủ nghĩa Việt Nam là thành viên hoặc thoả thuận quốc tế mà cơ quan nhà nước có thẩm quyền của Cộng hoà xã hội chủ nghĩa Việt Nam ký kết.

Điều 27. Kiểm tra chất lượng sản phẩm, hàng hoá

1. Kiểm tra chất lượng sản phẩm trong sản xuất bao gồm các nội dung sau đây:

b) Kiểm tra kết quả đánh giá sự phù hợp, nhãn hàng hoá, dấu hợp chuẩn, dấu hợp quy và các tài liệu kèm theo sản phẩm cần kiểm tra;

c) Thử nghiệm mẫu theo tiêu chuẩn đã công bố áp dụng, quy chuẩn kỹ thuật tương ứng khi cần thiết.

2. Kiểm tra chất lượng hàng hoá trong nhập khẩu, lưu thông trên thị trường bao gồm các nội dung sau đây:

a) Kiểm tra kết quả đánh giá sự phù hợp, nhãn hàng hoá, dấu hợp chuẩn, dấu hợp quy và các tài liệu kèm theo sản phẩm, hàng hoá cần kiểm tra;

b) Thử nghiệm mẫu theo tiêu chuẩn đã công bố áp dụng, quy chuẩn kỹ thuật tương ứng khi cần thiết.

3. Việc kiểm tra chất lượng sản phẩm, hàng hóa do cơ quan kiểm tra chất lượng sản phẩm, hàng hoá quy định tại Điều 45 của Luật này tiến hành.

4. Việc miễn, giảm kiểm tra chất lượng đối với sản phẩm, hàng hoá đã được chứng nhận hợp chuẩn, chứng nhận hợp quy, công bố hợp chuẩn, công bố hợp quy, chứng nhận đã áp dụng các hệ thống quản lý tiên tiến theo tiêu chuẩn quốc tế, tiêu chuẩn khu vực được thực hiện theo quy định của Bộ quản lý ngành, lĩnh vực.

Mục 2 QUẢN LÝ CHẤT LƯỢNG SẢN PHẨM TRONG SẢN XUẤT

Điều 28. Điều kiện bảo đảm chất lượng sản phẩm trong sản xuất trước khi đưa ra thị trường

1. Người sản xuất phải thực hiện các yêu cầu về quản lý chất lượng sản phẩm trong sản xuất như sau:

a) Áp dụng hệ thống quản lý nhằm bảo đảm chất lượng sản phẩm do mình sản xuất phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng.

b) Công bố tiêu chuẩn áp dụng quy định tại Điều 23 của Luật này và ghi nhãn theo quy định của pháp luật về nhãn hàng hoá.

c) Lựa chọn chứng nhận hợp chuẩn, công bố hợp chuẩn đối với sản phẩm thuộc nhóm 1.

2. Việc bảo đảm chất lượng sản phẩm sản xuất, kinh doanh nhỏ lẻ trước khi đưa ra thị trường được thực hiện theo quy định của Bộ quản lý ngành, lĩnh vực.

Điều 29. Kiểm tra chất lượng sản phẩm trong sản xuất

1. Việc kiểm tra chất lượng sản phẩm trong sản xuất được tiến hành theo một trong các trường hợp sau đây:

a) Hàng hoá xuất khẩu không bảo đảm chất lượng quy định tại Điều 32 của Luật này;

b) Hàng hoá lưu thông trên thị trường không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng quy định tại khoản 3 Điều 40 của Luật này.

2. Việc kiểm tra chất lượng sản phẩm trong sản xuất do cơ quan kiểm tra chất lượng sản phẩm, hàng hoá được thực hiện dưới hình thức đoàn kiểm tra quy định tại Điều 48 của Luật này.

3. Trình tự, thủ tục kiểm tra chất lượng sản phẩm trong sản xuất được quy định như sau:

a) Xuất trình quyết định kiểm tra;

b) Tiến hành kiểm tra theo nội dung quy định tại khoản 1 Điều 27 của Luật này;

c) Lập biên bản kiểm tra;

d) Thông báo cho người sản xuất và báo cáo cho cơ quan kiểm tra chất lượng sản phẩm, hàng hoá về kết quả kiểm tra;

đ) Xử lý vi phạm theo quy định tại Điều 30 của Luật này.

Điều 30. Xử lý vi phạm trong quá trình kiểm tra chất lượng sản phẩm trong sản xuất

a) Đoàn kiểm tra yêu cầu người sản xuất thực hiện các biện pháp khắc phục, sửa chữa để bảo đảm chất lượng sản phẩm trước khi đưa ra thị trường;

b) Sau khi có yêu cầu của đoàn kiểm tra mà ngư­ời sản xuất vẫn tiếp tục vi phạm thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá trong thời hạn 7 ngày làm việc, kể từ ngày có kết luận về vi phạm của tổ chức, cá nhân sản xuất, kinh doanh, thông báo công khai trên ph­ương tiện thông tin đại chúng tên, địa chỉ của người sản xuất, tên sản phẩm không phù hợp và mức độ không phù hợp của sản phẩm;

c) Sau khi bị thông báo công khai trên phương tiện thông tin đại chúng mà người sản xuất vẫn tiếp tục vi phạm thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá kiến nghị cơ quan nhà nước có thẩm quyền xử lý theo quy định của pháp luật.

2. Trong quá trình kiểm tra chất lượng sản phẩm trong sản xuất, mà kết quả thử nghiệm khẳng định sản phẩm không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng đe dọa đến sự an toàn của người, động vật, thực vật, tài sản, môi trường thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá thông báo công khai trên phương tiện thông tin đại chúng, tạm đình chỉ sản xuất sản phẩm không phù hợp và kiến nghị cơ quan nhà nước có thẩm quyền xử lý theo quy định của pháp luật.

Điều 31. Chi phí thử nghiệm, chứng nhận hợp chuẩn, chứng nhận hợp quy

Người sản xuất phải trả chi phí thử nghiệm, chứng nhận hợp chuẩn, chứng nhận hợp quy theo thỏa thuận với tổ chức thử nghiệm, tổ chức chứng nhận hợp chuẩn, hợp quy.

Mục 3 QUẢN LÝ CHẤT LƯỢNG HÀNG HOÁ XUẤT KHẨU

Điều 32. Điều kiện bảo đảm chất lượng hàng hoá xuất khẩu

2. Áp dụng các quy chuẩn kỹ thuật tương ứng trong quá trình sản xuất hoặc tự xây dựng và áp dụng các hệ thống quản lý nhằm bảo đảm chất lượng sản phẩm do mình sản xuất.

Điều 33. Biện pháp xử lý hàng hoá xuất khẩu không bảo đảm điều kiện xuất khẩu

Hàng hoá không bảo đảm điều kiện xuất khẩu quy định tại khoản 1 Điều 32 của Luật này mà không xuất khẩu được hoặc bị trả lại thì tuỳ theo tính chất, mức độ vi phạm, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá áp dụng một hoặc các biện pháp xử lý sau đây:

1. Thực hiện biện pháp kiểm tra chất lượng sản phẩm trong sản xuất theo nội dung quy định tại khoản 1 Điều 27, trình tự thủ tục theo quy định tại khoản 3 Điều 29 của Luật này đối với hàng hoá xuất khẩu không bảo đảm chất lượng gây ảnh hưởng đến lợi ích và uy tín quốc gia.

2. Cho lưu thông trên thị trường nếu chất lượng hàng hoá phù hợp với quy chuẩn kỹ thuật tương ứng của Việt Nam.

3. Yêu cầu người sản xuất khắc phục, sửa chữa để hàng hoá được tiếp tục xuất khẩu hoặc được lưu thông trên thị trường Việt Nam sau khi đã đáp ứng theo quy chuẩn kỹ thuật tương ứng.

4. Kiến nghị cơ quan nhà nước có thẩm quyền ra quyết định tiêu huỷ.

Mục 4 QUẢN LÝ CHẤT LƯỢNG HÀNG HOÁ NHẬP KHẨU

Điều 34. Điều kiện bảo đảm chất lượng hàng hoá nhập khẩu

1. Hàng hóa nhập khẩu phải được công bố tiêu chuẩn áp dụng theo quy định tại Điều 23 của Luật này và ghi nhãn theo quy định của pháp luật về nhãn hàng hoá.

3. Hàng hóa nhập khẩu thuộc nhóm 2 không đáp ứng quy định tại khoản 2 Điều này khi nhập khẩu phải được tổ chức giám định được chỉ định hoặc được thừa nhận theo quy định tại Điều 26 của Luật này giám định tại cửa khẩu xuất hoặc cửa khẩu nhập.

4. Hàng hóa nhập khẩu thuộc nhóm 2 phải được kiểm tra chất lượng khi nhập khẩu theo nội dung quy định tại khoản 2 Điều 27, trình tự, thủ tục quy định tại Điều 35 của Luật này.

Điều 35. Trình tự, thủ tục kiểm tra chất lượng hàng hoá nhập khẩu

1. Cơ quan kiểm tra chất lượng sản phẩm, hàng hoá tiến hành kiểm tra theo trình tự, thủ tục sau đây:

b) Xem xét tính hợp lệ và đầy đủ của hồ sơ đăng ký kiểm tra ngay khi tiếp nhận hồ sơ đăng ký kiểm tra chất lượng của người nhập khẩu;

c) Tiến hành kiểm tra theo nội dung kiểm tra quy định tại khoản 2 Điều 27 của Luật này;

d) Thông báo kết quả kiểm tra cho người nhập khẩu, xác nhận hàng hoá đã đáp ứng yêu cầu chất lượng để được làm thủ tục nhập khẩu hàng hoá với cơ quan hải quan;

đ) Xử lý vi phạm trong quá trình kiểm tra theo quy định tại Điều 36 của Luật này.

2. Căn cứ quy định tại khoản 1 Điều này, Bộ quản lý ngành, lĩnh vực quy định chi tiết trình tự, thủ tục kiểm tra chất lượng hàng hoá nhập khẩu thuộc phạm vi được phân công quản lý quy định tại khoản 5 Điều 68, khoản 4 Điều 69 và khoản 2 Điều 70 của Luật này.

Điều 36. Xử lý vi phạm trong quá trình kiểm tra chất lượng hàng hóa nhập khẩu

1. Hàng hóa có giấy chứng nhận kết quả đánh giá sự phù hợp nhưng không đáp ứng yêu cầu về nhãn hàng hóa, dấu hợp chuẩn, dấu hợp quy thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá yêu cầu người nhập khẩu khắc phục trước khi xác nhận để làm thủ tục nhập khẩu với cơ quan hải quan.

2. Trường hợp hàng hoá đáp ứng yêu cầu về nhãn hàng hoá nhưng không có giấy chứng nhận kết quả đánh giá sự phù hợp thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá yêu cầu người nhập khẩu lựa chọn một trong số tổ chức giám định đã được chỉ định hoặc thừa nhận thực hiện việc đánh giá và cấp giấy chứng nhận nhập khẩu tại cửa khẩu nhập hoặc cửa khẩu xuất.

3. Trong trường hợp kết quả thử nghiệm, giám định chất lượng hàng hoá xác định hàng hóa không đáp ứng tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng của Việt Nam, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá báo cáo và đề xuất biện pháp xử lý với cơ quan nhà nước có thẩm quyền; tuỳ theo tính chất, mức độ vi phạm yêu cầu quản lý chất lượng, cơ quan nhà nước có thẩm quyền quyết định buộc tái xuất, tiêu hủy hoặc tái chế hàng hoá, đồng thời thông báo cho cơ quan hải quan để phối hợp xử lý và người nhập khẩu biết để thực hiện.

4. Hàng hoá nhập khẩu sau khi được thông quan được phép lưu thông trên thị trường và chịu sự kiểm tra chất lượng theo quy định tại Mục 5 Chương này.

Điều 37. Chi phí, lệ phí phục vụ kiểm tra chất lượng hàng hoá nhập khẩu

1. Người nhập khẩu trả chi phí thử nghiệm, giám định theo thoả thuận với tổ chức thử nghiệm, tổ chức giám định chất lượng.

2. Người nhập khẩu nộp lệ phí kiểm tra chất lượng hàng hoá nhập khẩu.

3. Bộ Tài chính quy định mức, việc thu và quản lý lệ phí kiểm tra chất lượng hàng hoá nhập khẩu.

Mục 5 QUẢN LÝ CHẤT LƯỢNG HÀNG HOÁ LƯU THÔNG TRÊN THỊ TRƯỜNG

Điều 38. Điều kiện bảo đảm chất lượng hàng hoá lưu thông trên thị trường

Hàng hoá lưu thông trên thị trường phải được người bán hàng thực hiện các yêu cầu về quản lý chất lượng sau đây:

1. Tuân thủ các quy chuẩn kỹ thuật tương ứng trong quá trình lưu thông hàng hoá hoặc tự áp dụng các biện pháp kiểm soát chất lượng nhằm duy trì chất lượng của hàng hoá do mình bán;

2. Chịu sự kiểm tra chất lượng hàng hoá theo nội dung kiểm tra quy định tại khoản 2 Điều 27; trình tự, thủ tục kiểm tra quy định tại Điều 39; xử lý vi phạm pháp luât quy định tại Điều 40 của Luật này.

Điều 39. Trình tự, thủ tục kiểm tra chất lượng hàng hoá lưu thông trên thị trường

1. Đoàn kiểm tra tiến hành kiểm tra theo trình tự, thủ tục như sau:

a) Xuất trình quyết định kiểm tra trước khi kiểm tra;

b) Tiến hành kiểm tra theo nội dung quy định tại khoản 2 Điều 27 của Luật này;

c) Lập biên bản kiểm tra;

d) Thông báo kết quả kiểm tra cho người bán hàng và báo cáo cơ quan kiểm tra chất lượng sản phẩm, hàng hoá;

đ) Xử lý vi phạm theo quy định tại Điều 40 của Luật này.

2. Kiểm soát viên chất l­ượng tiến hành kiểm tra độc lập theo trình tự, thủ tục như sau:

a) Xuất trình thẻ kiểm soát viên trước khi kiểm tra;

b) Tiến hành kiểm tra theo nội dung quy định tại khoản 2 Điều 27 của Luật này;

c) Lập biên bản kiểm tra;

d) Thông báo kết quả kiểm tra cho người bán hàng và báo cáo cơ quan kiểm tra chất lượng sản phẩm, hàng hoá;

đ) Xử lý vi phạm theo quy định tại Điều 40 của Luật này.

Điều 40. Xử lý vi phạm trong quá trình kiểm tra chất lượng hàng hoá lưu thông trên thị trường

a) Đoàn kiểm tra, kiểm soát viên chất lượng yêu cầu người bán hàng tạm dừng việc bán hàng hoá và trong thời hạn không quá 24 giờ phải báo cáo với cơ quan kiểm tra chất lượng sản phẩm, hàng hoá để xử lý theo thẩm quyền;

b) Yêu cầu người bán hàng liên hệ với người sản xuất, người nhập khẩu để thực hiện các biện pháp xử lý, khắc phục, sửa chữa;

c) Trường hợp người bán hàng vẫn tiếp tục vi phạm thì theo đề nghị của đoàn kiểm tra, kiểm soát viên chất lượng, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá trong thời hạn 7 ngày làm việc, kể từ ngày có kết luận về vi phạm của tổ chức, cá nhân sản xuất, kinh doanh, thông báo công khai trên phương tiện thông tin đại chúng tên người bán hàng, địa chỉ nơi bán hàng, tên hàng hoá và mức độ không phù hợp của hàng hoá;

d) Sau khi thông báo công khai, người bán hàng vẫn tiếp tục vi phạm thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá kiến nghị cơ quan nhà nước có thẩm quyền xử lý theo quy định của pháp luật.

2. Trong trường hợp kết quả thử nghiệm mẫu hàng hoá không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng thì tuỳ theo tính chất, mức độ vi phạm, đoàn kiểm tra, kiểm soát viên chất lượng áp dụng các biện pháp xử lý như sau:

a) Niêm phong hàng hóa, không cho người bán hàng được phép tiếp tục bán hàng hoá không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng và trong thời hạn không quá 24 giờ phải báo cáo với cơ quan kiểm tra chất lượng sản phẩm, hàng hoá để xử lý theo thẩm quyền;

b) Yêu cầu người bán hàng liên hệ với người sản xuất, người nhập khẩu để thực hiện các biện pháp xử lý, khắc phục, sửa chữa;

c) Trường hợp người bán hàng vẫn tiếp tục vi phạm hoặc hàng hóa không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng đe dọa sự an toàn của người, động vật, thực vật, tài sản, môi trường thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá thông báo công khai trên ph­ương tiện thông tin đại chúng tên, địa chỉ của tổ chức, cá nhân kinh doanh hàng hoá, tên hàng hoá không phù hợp và mức độ không phù hợp của hàng hóa;

d) Sau khi thông báo công khai mà người bán hàng vẫn tiếp tục vi phạm thì cơ quan kiểm tra chất lượng sản phẩm, hàng hoá kiến nghị cơ quan nhà nước có thẩm quyền xử lý theo quy định của pháp luật.

3. Trong trường hợp phát hiện hàng hoá lưu thông trên thị trường không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng thì tuỳ theo tính chất, mức độ vi phạm, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá tiến hành việc kiểm tra chất lượng sản phẩm theo nội dung quy định tại khoản 1 Điều 27 của Luật này.

Điều 41. Chi phí lấy mẫu và thử nghiệm phục vụ kiểm tra chất lượng và giải quyết khiếu nại, tố cáo về chất lượng sản phẩm, hàng hoá

1. Chi phí lấy mẫu và thử nghiệm để kiểm tra chất lượng sản phẩm trong sản xuất và hàng hoá trên thị trường do cơ quan kiểm tra chất lượng sản phẩm, hàng hoá quyết định việc lấy mẫu và thử nghiệm chất lượng sản phẩm, hàng hóa chi trả. Chi phí lấy mẫu và thử nghiệm được bố trí trong dự toán kinh phí hoạt động của cơ quan kiểm tra chất lượng sản phẩm, hàng hoá.

2. Căn cứ kết quả thử nghiệm, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá kết luận người sản xuất, người bán hàng vi phạm quy định về chất lượng sản phẩm, hàng hóa thì người sản xuất, người bán hàng phải trả chi phí lấy mẫu và thử nghiệm chất lượng sản phẩm, hàng hóa cho cơ quan kiểm tra chất lượng sản phẩm, hàng hoá.

3. Trong trường hợp sản phẩm, hàng hoá bị khiếu nại, tố cáo về chất lượng mà cơ quan kiểm tra chất lượng sản phẩm, hàng hoá kết luận việc khiếu nại, tố cáo về chất lượng sản phẩm, hàng hoá không đúng thì người khiếu nại, tố cáo phải trả chi phí lấy mẫu và thử nghiệm chất lượng sản phẩm, hàng hóa cho cơ quan kiểm tra chất lượng sản phẩm, hàng hoá.

Mục 6 QUẢN LÝ CHẤT LƯỢNG HÀNG HOÁ TRONG QUÁ TRÌNH SỬ DỤNG

Điều 42. Điều kiện bảo đảm chất lượng hàng hoá trong quá trình sử dụng

1. Hàng hoá phải được sử dụng, vận chuyển, lưu giữ, bảo quản, bảo trì, bảo dưỡng theo hướng dẫn của người sản xuất.

2. Hàng hoá phải được kiểm định theo quy định trong quy chuẩn kỹ thuật tương ứng do cơ quan quản lý nhà nước có thẩm quyền ban hành.

Điều 43. Xử lý kết quả kiểm định

1. Hàng hoá sau khi được kiểm định, đáp ứng quy chuẩn kỹ thuật tương ứng thì được phép tiếp tục sử dụng trong thời gian quy định tại quy chuẩn kỹ thuật đó.

2. Hàng hoá sau khi được kiểm định không đáp ứng quy chuẩn kỹ thuật tương ứng thì người sở hữu hàng hoá phải có biện pháp khắc phục; sau khi khắc phục mà kết quả kiểm định vẫn không đạt yêu cầu thì tổ chức kiểm định không cấp giấy chứng nhận kiểm định và hàng hoá đó không được phép tiếp tục sử dụng.

Điều 44. Lệ phí kiểm định hàng hoá trong quá trình sử dụng

1. Việc kiểm định hàng hoá trong quá trình sử dụng phải trả lệ phí kiểm định.

2. Bộ Tài chính quy định mức, việc thu và quản lý lệ phí kiểm định hàng hoá trong quá trình sử dụng.

Chương IV KIỂM TRA, THANH TRA VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Mục 1 KIỂM TRA CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Điều 45. Phân công trách nhiệm của cơ quan kiểm tra chất lượng sản phẩm, hàng hoá

1. Cơ quan kiểm tra chất lượng sản phẩm, hàng hoá thuộc Bộ quản lý ngành, lĩnh vực thực hiện việc kiểm tra chất lượng sản phẩm trong sản xuất thuộc phạm vi được phân công theo quy định tại khoản 1 Điều 70 Luật này và hàng hoá trong xuất khẩu, nhập khẩu, lưu thông trên thị trường, trong quá trình sử dụng thuộc phạm vi được phân công theo quy định tại khoản 5 Điều 68 và khoản 2 Điều 70 của Luật này.

2. Cơ quan kiểm tra chất lượng sản phẩm, hàng hoá thuộc Bộ Khoa học và Công nghệ thực hiện việc kiểm tra chất lượng sản phẩm trong sản xuất thuộc phạm vi được phân công theo quy định tại khoản 1 Điều 70 Luật này và hàng hoá trong nhập khẩu, xuất khẩu, lưu thông trên thị trường, trong quá trình sử dụng thuộc phạm vi được phân công theo quy định tại khoản 4 Điều 69 của Luật này.

3. Cơ quan kiểm tra chất lượng sản phẩm, hàng hoá thuộc Uỷ ban nhân dân tỉnh, thành phố trực thuộc trung ương thực hiện việc kiểm tra chất lượng sản phẩm, hàng hoá trong phạm vi của địa phương theo quy định của Bộ quản lý ngành, lĩnh vực.

Điều 46. Quyền hạn của cơ quan kiểm tra chất lượng sản phẩm, hàng hoá

Trong phạm vi nhiệm vụ, quyền hạn của mình, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá có các quyền sau đây:

1. Quyết định thành lập đoàn kiểm tra hoặc phân công kiểm soát viên chất lượng thực hiện công tác kiểm tra theo kế hoạch hoặc đột xuất;

2. Cảnh báo các nguy cơ không bảo đảm chất lượng của sản phẩm, hàng hoá;

3. Xử lý vi phạm trong quá trình kiểm tra theo quy định tại các điều 30, 36 và 40 của Luật này;

4. Giải quyết khiếu nại, tố cáo về quyết định của đoàn kiểm tra, kiểm soát viên chất lượng, hành vi của thành viên đoàn kiểm tra, kiểm soát viên chất lượng theo quy định của pháp luật về khiếu nại, tố cáo.

Điều 47. Nhiệm vụ của cơ quan kiểm tra chất lượng sản phẩm, hàng hoá

Trong phạm vi nhiệm vụ, quyền hạn của mình, cơ quan kiểm tra chất lượng sản phẩm, hàng hoá có nhiệm vụ sau đây:

1. Xác định chủng loại hàng hoá cụ thể để tiến hành kiểm tra chất lượng;

2. Xây dựng kế hoạch kiểm tra hằng năm trình cơ quan nhà nước có thẩm quyền quyết định;

3. Tiếp nhận hồ sơ đăng ký kiểm tra chất lượng của người nhập khẩu;

4. Xác nhận điều kiện bảo đảm chất lượng đối với hàng hoá nhập khẩu;

5. Lập và trình cơ quan nhà nước có thẩm quyền quyết định việc xây dựng đội ngũ kiểm soát viên chất lượng, trang bị phương tiện kỹ thuật đáp ứng yêu cầu kiểm tra chất lượng sản phẩm, hàng hoá;

6. Ra quyết định xử lý trong thời hạn 3 ngày làm việc, kể từ ngày nhận được báo cáo của đoàn kiểm tra hoặc kiểm soát viên chất lượng về việc tạm đình chỉ sản xuất, niêm phong, tạm dừng bán hàng;

7. Bảo đảm khách quan, chính xác, công khai, minh bạch và không phân biệt đối xử trong hoạt động kiểm tra chất lượng sản phẩm, hàng hoá;

1. Đoàn kiểm tra do thủ trưởng cơ quan kiểm tra chất lượng sản phẩm, hàng hoá quyết định thành lập trên cơ sở chương trình, kế hoạch kiểm tra đã được cơ quan nhà nước có thẩm quyền phê duyệt hoặc trong trường hợp có yêu cầu kiểm tra đột xuất.

2. Đoàn kiểm tra phải có ít nhất năm mươi phần trăm số thành viên là kiểm soát viên chất lượng.

Điều 49. Nhiệm vụ, quyền hạn của đoàn kiểm tra

Trong quá trình kiểm tra chất lượng sản phẩm, hàng hoá, đoàn kiểm tra có những nhiệm vụ, quyền hạn sau đây:

2. Lấy mẫu để thử nghiệm khi cần thiết;

3. Niêm phong hàng hóa, tạm dừng bán hàng hoá không phù hợp trong quá trình kiểm tra trên thị trường;

4. Yêu cầu tổ chức, cá nhân sản xuất, kinh doanh sản phẩm, hàng hoá không phù hợp với tiêu chuẩn đã công bố áp dụng, quy chuẩn kỹ thuật tương ứng có biện pháp khắc phục, sửa chữa;

5. Kiến nghị cơ quan kiểm tra chất lượng sản phẩm, hàng hoá xử lý theo thẩm quyền quy định tại Điều 46 của Luật này.

6. Bảo đảm nguyên tắc khách quan, chính xác và không phân biệt đối xử khi tiến hành kiểm tra;

8. Báo cáo chính xác và kịp thời kết quả kiểm tra cho cơ quan kiểm tra chất lượng sản phẩm, hàng hoá;

9. Chịu trách nhiệm trước pháp luật về kết quả kiểm tra, kết luận và xử lý vi phạm của mình.

Điều 50. Kiểm soát viên chất lượng

1. Kiểm soát viên chất lượng là công chức được bổ nhiệm vào ngạch kiểm soát viên chất lượng thuộc cơ quan kiểm tra chất lượng sản phẩm, hàng hoá.

2. Tiêu chuẩn, chế độ và việc bổ nhiệm kiểm soát viên chất lượng do Chính phủ quy định.

Điều 51. Nhiệm vụ, quyền hạn của kiểm soát viên chất lượng

Trong quá trình kiểm tra chất lượng sản phẩm, hàng hoá, kiểm soát viên chất lượng có những nhiệm vụ, quyền hạn sau đây:

2. Niêm phong, tạm dừng bán hàng hoá không phù hợp trong quá trình kiểm tra trên thị trường;

3. Yêu cầu tổ chức, cá nhân sản xuất, kinh doanh sản phẩm, hàng hoá không phù hợp với tiêu chuẩn đã công bố áp dụng, quy chuẩn kỹ thuật tương ứng có biện pháp khắc phục, sửa chữa;

4. Kiến nghị cơ quan kiểm tra chất lượng sản phẩm, hàng hoá xử lý theo thẩm quyền quy định tại Điều 46 của Luật này;

5. Bảo đảm nguyên tắc khách quan, chính xác và không phân biệt đối xử khi tiến hành kiểm tra;

7. Báo cáo chính xác và kịp thời kết quả kiểm tra cho cơ quan kiểm tra chất lượng sản phẩm, hàng hoá;

8. Chịu trách nhiệm trước pháp luật về kết quả kiểm tra, kết luận và xử lý vi phạm của mình.

Mục 2 THANH TRA CHUYÊN NGÀNH VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HÓA

Điều 52. Thanh tra về chất lượng sản phẩm, hàng hoá

1. Thanh tra về chất lượng sản phẩm, hàng hóa là thanh tra chuyên ngành.

2. Việc thanh tra được thực hiện theo quy định của pháp luật về thanh tra.

3. Chính phủ quy định cụ thể về tổ chức và hoạt động của thanh tra chuyên ngành về chất lượng sản phẩm, hàng hóa.

Điều 53. Nhiệm vụ và đối tượng thanh tra chuyên ngành về chất lượng sản phẩm, hàng hoá

2. Đối tượng của thanh tra chuyên ngành về chất lượng sản phẩm, hàng hoá là tổ chức, cá nhân sản xuất, kinh doanh, người tiêu dùng, tổ chức đánh giá sự phù hợp, tổ chức nghề nghiệp, tổ chức bảo vệ quyền lợi người tiêu dùng và cơ quan kiểm tra chất lượng sản phẩm, hàng hoá.

Chương V GIẢI QUYẾT TRANH CHẤP, BỒI THƯỜNG THIỆT HẠI, GIẢI QUYẾT KHIẾU NẠI, TỐ CÁO VÀ XỬ LÝ VI PHẠM PHÁP LUẬT VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Mục 1 GIẢI QUYẾT TRANH CHẤP VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Điều 54. Tranh chấp về chất lượng sản phẩm, hàng hoá

Tranh chấp về chất lượng sản phẩm, hàng hoá bao gồm:

1. Tranh chấp giữa người mua với người nhập khẩu, người bán hàng hoặc giữa các thương nhân với nhau do sản phẩm, hàng hoá không phù hợp với tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng hoặc thỏa thuận về chất lượng trong hợp đồng.

Điều 55. Hình thức giải quyết tranh chấp về chất lượng sản phẩm, hàng hoá

1. Thương lượng giữa các bên tranh chấp về chất lượng sản phẩm, hàng hóa.

2. Hoà giải giữa các bên do một cơ quan, tổ chức hoặc cá nhân được các bên thoả thuận chọn làm trung gian.

3. Giải quyết tại trọng tài hoặc toà án.

Thủ tục giải quyết tranh chấp về chất lượng sản phẩm, hàng hóa tại trọng tài hoặc toà án được tiến hành theo quy định của pháp luật về tố tụng trọng tài hoặc tố tụng dân sự.

Điều 56. Thời hiệu khiếu nại, khởi kiện tranh chấp về chất lượng sản phẩm, hàng hoá

1. Thời hiệu khởi kiện về chất lượng sản phẩm, hàng hoá giữa người mua với người bán hàng được thực hiện theo quy định của Bộ luật dân sự.

2. Thời hiệu khiếu nại, khởi kiện về chất lượng sản phẩm, hàng hoá giữa các tổ chức, cá nhân sản xuất, kinh doanh được thực hiện theo quy định của Luật thương mại.

3. Thời hiệu khiếu nại, khởi kiện đòi bồi thường do sản phẩm, hàng hoá không bảo đảm chất lượng gây thiệt hại cho người, động vật, thực vật, tài sản, môi trường là 2 năm, kể từ thời điểm các bên được thông báo về thiệt hại với điều kiện thiệt hại xảy ra trong thời hạn sử dụng của sản phẩm, hàng hoá có ghi hạn sử dụng và 5 năm kể từ ngày giao hàng đối với sản phẩm, hàng hoá không ghi hạn sử dụng.

Điều 57. Kiểm tra, thử nghiệm, giám định để giải quyết tranh chấp về chất lượng sản phẩm, hàng hóa

1. Cơ quan, tổ chức giải quyết tranh chấp chỉ định hoặc các bên đương sự thoả thuận đề nghị cơ quan, tổ chức có chuyên môn, nghiệp vụ thực hiện việc kiểm tra, thử nghiệm, giám định sản phẩm, hàng hoá tranh chấp về chất lượng sản phẩm, hàng hóa.

2. Căn cứ kiểm tra, thử nghiệm, giám định sản phẩm, hàng hoá tranh chấp bao gồm:

a) Thoả thuận về chất lượng sản phẩm, hàng hoá trong hợp đồng;

b) Tiêu chuẩn công bố áp dụng, quy chuẩn kỹ thuật tương ứng đối với sản phẩm, hàng hoá.

Điều 58. Chi phí lấy mẫu và thử nghiệm hoặc giám định trong giải quyết tranh chấp về chất lượng sản phẩm, hàng hoá

1. Ng­ười khiếu nại, khởi kiện phải trả chi phí lấy mẫu và thử nghiệm hoặc giám định chất l­ượng sản phẩm, hàng hóa tranh chấp.

2. Trong trường hợp kết quả thử nghiệm hoặc giám định khẳng định tổ chức, cá nhân sản xuất, kinh doanh sản phẩm, hàng hoá vi phạm quy định về chất l­ượng sản phẩm, hàng hóa thì tổ chức, cá nhân sản xuất, kinh doanh phải trả lại chi phí lấy mẫu và thử nghiệm hoặc giám định chất lượng sản phẩm, hàng hóa tranh chấp cho ng­ười khiếu nại, khởi kiện.

Mục 2 BỒI THƯỜNG THIỆT HẠI VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Điều 59. Nguyên tắc bồi thường thiệt hại

1. Thiệt hại do vi phạm quy định về chất lượng sản phẩm, hàng hóa phải được bồi thường toàn bộ và kịp thời.

2. Thiệt hại được bồi thường là thiệt hại được quy định tại Điều 60 của Luật này, trừ trường hợp các bên tranh chấp có thỏa thuận khác.

Điều 60. Các thiệt hại phải bồi thường do hàng hóa không bảo đảm chất lượng

1. Thiệt hại về giá trị hàng hóa, tài sản bị h­ư hỏng hoặc bị huỷ hoại.

2. Thiệt hại về tính mạng, sức khoẻ con người.

3. Thiệt hại về lợi ích gắn liền với việc sử dụng, khai thác hàng hóa, tài sản.

4. Chi phí hợp lý để ngăn chặn, hạn chế và khắc phục thiệt hại.

Điều 61. Trách nhiệm bồi thường thiệt hại

Điều 62. Các trường hợp không phải bồi thường thiệt hại

1. Người sản xuất, người nhập khẩu không phải bồi thường trong các trường hợp sau đây:

a) Người bán hàng bán hàng hoá đã hết hạn sử dụng; người tiêu dùng sử dụng hàng hóa đã hết hạn sử dụng;

b) Đã hết thời hiệu khiếu nại, khởi kiện;

c) Đã thông báo thu hồi hàng hoá có khuyết tật đến người bán hàng, người tiêu dùng trước thời điểm hàng hoá gây thiệt hại;

d) Sản phẩm, hàng hoá có khuyết tật do tuân thủ quy định bắt buộc của cơ quan nhà nước có thẩm quyền;

đ) Trình độ khoa học, công nghệ của thế giới chưa đủ để phát hiện khả năng gây mất an toàn của sản phẩm tính đến thời điểm hàng hoá gây thiệt hại;

e) Thiệt hại phát sinh do lỗi của người bán hàng;

g) Thiệt hại phát sinh do lỗi của người mua, người tiêu dùng.

2. Người bán hàng không phải bồi thường cho người mua, người tiêu dùng trong các trường hợp sau đây:

a) Người tiêu dùng sử dụng hàng hoá đã hết hạn sử dụng;

b) Đã hết thời hiệu khiếu nại, khởi kiện;

c) Đã thông báo hàng hoá có khuyết tật đến người mua, người tiêu dùng nhưng người mua, người tiêu dùng vẫn mua, sử dụng hàng hoá đó;

d) Hàng hoá có khuyết tật do người sản xuất, người nhập khẩu tuân thủ quy định bắt buộc của cơ quan nhà nước có thẩm quyền;

đ) Trình độ khoa học, công nghệ của thế giới chưa đủ để phát hiện khả năng gây mất an toàn của hàng hóa tính đến thời điểm hàng hoá gây thiệt hại;

e) Thiệt hại phát sinh do lỗi của người mua, người tiêu dùng.

Điều 63. Trách nhiệm bồi thường thiệt hại của tổ chức đánh giá sự phù hợp khi cung cấp kết quả sai

1. Tổ chức đánh giá sự phù hợp cung cấp kết quả sai thì phải bồi thường thiệt hại phát sinh cho tổ chức, cá nhân yêu cầu đánh giá sự phù hợp theo quy định của pháp luật về dân sự.

2. Tổ chức, cá nhân có sản phẩm, hàng hóa được đánh giá sự phù hợp có nghĩa vụ chứng minh kết quả sai và lỗi của tổ chức đánh giá sự phù hợp quy định tại khoản 1 Điều này.

Mục 3 GIẢI QUYẾT KHIẾU NẠI, TỐ CÁO

Điều 64. Khiếu nại, tố cáo hành vi vi phạm pháp luật về chất lượng sản phẩm, hàng hoá

1. Tổ chức, cá nhân có quyền khiếu nại với cơ quan nhà nước hoặc người có thẩm quyền về quyết định hành chính, hành vi hành chính của cơ quan nhà nước, người có thẩm quyền mà tổ chức, cá nhân cho là trái pháp luật hoặc về hành vi xâm phạm quyền và lợi ích hợp pháp của mình trong lĩnh vực chất lượng sản phẩm, hàng hóa.

2. Cá nhân có quyền tố cáo với cơ quan nhà nước hoặc người có thẩm quyền về hành vi vi phạm pháp luật về chất lượng sản phẩm, hàng hóa của cơ quan, tổ chức, cá nhân gây thiệt hại hoặc đe dọa gây thiệt hại đến lợi ích của Nhà nước, quyền và lợi ích hợp pháp của công dân.

3. Tổ chức, cá nhân chịu trách nhiệm về khiếu nại, tố cáo của mình.

Điều 65. Giải quyết khiếu nại, tố cáo hành vi vi phạm pháp luật về chất lượng sản phẩm, hàng hoá

Cơ quan nhà nước, người có thẩm quyền giải quyết khiếu nại, tố cáo có trách nhiệm xem xét, giải quyết khiếu nại, tố cáo hành vi vi phạm pháp luật về chất lượng sản phẩm, hàng hóa theo quy định của pháp luật về khiếu nại, tố cáo.

Mục 4 XỬ LÝ VI PHẠM PHÁP LUẬT VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Điều 66. Xử lý vi phạm pháp luật về chất lượng sản phẩm, hàng hoá

1. Cá nhân có hành vi vi phạm pháp luật về chất lượng sản phẩm, hàng hóa thì tuỳ theo tính chất, mức độ vi phạm mà bị xử lý kỷ luật, xử phạt hành chính hoặc bị truy cứu trách nhiệm hình sự; nếu gây thiệt hại thì phải bồi thường theo quy định của pháp luật.

2. Tổ chức vi phạm pháp luật về chất lượng sản phẩm, hàng hóa thì tuỳ theo tính chất, mức độ vi phạm mà bị xử phạt hành chính, nếu gây thiệt hại thì phải bồi thường theo quy định của pháp luật.

3. Mức phạt tiền trong xử phạt vi phạm hành chính quy định tại khoản 1 và khoản 2 Điều này được ấn định ít nhất bằng giá trị sản phẩm, hàng hoá vi phạm đã tiêu thụ và nhiều nhất không quá năm lần giá trị hàng hoá vi phạm đã tiêu thụ; tiền do vi phạm mà có sẽ bị tịch thu theo quy định của pháp luật.

Chính phủ quy định cụ thể về hành vi, hình thức và mức xử phạt các hành vi vi phạm hành chính trong lĩnh vực chất lượng sản phẩm, hàng hoá và cách xác định giá trị sản phẩm, hàng hoá vi phạm.

Chương VI TRÁCH NHIỆM QUẢN LÝ NHÀ NƯỚC VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Điều 68. Trách nhiệm quản lý nhà nước về chất lượng sản phẩm, hàng hoá

1. Chính phủ thống nhất quản lý nhà nước về chất lượng sản phẩm, hàng hoá trong phạm vi cả nước.

2. Bộ Khoa học và Công nghệ chịu trách nhiệm trước Chính phủ thực hiện thống nhất quản lý nhà nước về chất lượng sản phẩm, hàng hoá.

3. Các Bộ quản lý ngành, lĩnh vực trong phạm vi nhiệm vụ, quyền hạn của mình có trách nhiệm phối hợp với Bộ Khoa học và Công nghệ thực hiện nhiệm vụ quản lý nhà nước về chất lượng sản phẩm, hàng hoá.

4. Uỷ ban nhân dân các cấp thực hiện việc quản lý nhà nước về chất lượng sản phẩm, hàng hoá trong phạm vi địa phương theo phân cấp của Chính phủ.

5. Căn cứ vào tình hình phát triển kinh tế-xã hội và yêu cầu quản lý nhà nước về chất lượng sản phẩm, hàng hoá trong từng thời kỳ, Chính phủ quy định cụ thể trách nhiệm quản lý nhà nước của các Bộ quản lý ngành, lĩnh vực đối với sản phẩm, hàng hoá chưa được quy định tại khoản 2 Điều 70 của Luật này.

Điều 69. Trách nhiệm quản lý nhà nước của Bộ Khoa học và Công nghệ

1. Xây dựng, ban hành hoặc trình Chính phủ ban hành và tổ chức thực hiện chính sách, chiến lược, quy hoạch, kế hoạch và văn bản quy phạm pháp luật về chất lượng sản phẩm, hàng hóa.

2. Chủ trì, phối hợp với các Bộ quản lý ngành, lĩnh vực, Uỷ ban nhân dân các tỉnh, thành phố trực thuộc trung ương xây dựng, trình cấp có thẩm quyền phê duyệt và tổ chức thực hiện chương trình quốc gia nâng cao năng suất, chất lượng và khả năng cạnh tranh của sản phẩm, hàng hóa.

3. Thực hiện quản lý nhà nước về chất lượng sản phẩm trong sản xuất theo quy định tại khoản 1 Điều 70 của Luật này.

5. Chủ trì, phối hợp với các Bộ quản lý ngành, lĩnh vực xây dựng và tổ chức thực hiện các biện pháp quản lý nhà nước về chất lượng, quy chế quản lý các tổ chức đánh giá sự phù hợp, quy chế chỉ định các tổ chức đánh giá sự phù hợp đối với sản phẩm trong sản xuất và hàng hóa trong xuất khẩu, nhập khẩu và lưu thông trên thị trường; tổ chức hoạt động hợp tác quốc tế về chất lượng sản phẩm, hàng hoá.

6. Chủ trì tổ chức đánh giá, đề xuất các hình thức tôn vinh, khen thưởng cấp quốc gia đối với sản phẩm, hàng hoá của tổ chức, cá nhân có thành tích xuất sắc về hoạt động chất lượng sản phẩm, hàng hóa; quy định điều kiện, thủ tục xét tặng giải thưởng của tổ chức, cá nhân về chất lượng sản phẩm, hàng hoá.

7. Theo dõi, thống kê, tổng hợp tình hình quản lý chất lượng sản phẩm, hàng hóa trong cả nước; tuyên truyền, phổ biến pháp luật, đào tạo, phổ biến kiến thức, thông tin về chất lượng và quản lý chất lượng sản phẩm, hàng hóa.

8. Thanh tra việc chấp hành pháp luật về chất lượng sản phẩm, hàng hóa; giải quyết khiếu nại, tố cáo, xử lý các vi phạm pháp luật về chất lượng sản phẩm, hàng hóa trong lĩnh vực được phân công.

Điều 70. Trách nhiệm quản lý nhà nước của bộ quản lý ngành, lĩnh vực

1. Bộ quản lý ngành, lĩnh vực thực hiện quản lý nhà nước về chất lượng sản phẩm, hàng hoá theo lĩnh vực được phân công, có trách nhiệm sau đây:

a) Xây dựng, ban hành hoặc trình Chính phủ ban hành và tổ chức thực hiện các chính sách, chiến lược, quy hoạch, kế hoạch, chương trình phát triển, nâng cao chất lượng sản phẩm, hàng hoá;

b) Xây dựng, ban hành và tổ chức thực hiện văn bản quy phạm pháp luật về chất lượng sản phẩm, hàng hoá phù hợp với yêu cầu, nhiệm vụ cụ thể của Bộ, ngành;

c) Tổ chức và chỉ đạo hoạt động quản lý nhà nước về chất lượng sản phẩm trong sản xuất;

d) Chỉ định và quản lý hoạt động của tổ chức đánh giá sự phù hợp phục vụ yêu cầu quản lý nhà nước về chất lượng sản phẩm, hàng hoá;

đ) Thanh tra việc chấp hành pháp luật về chất lượng sản phẩm, hàng hóa; giải quyết khiếu nại, tố cáo và xử lý các vi phạm pháp luật về chất lượng sản phẩm, hàng hóa trong phạm vi nhiệm vụ, quyền hạn của mình;

e) Theo dõi, thống kê, tổng hợp tình hình quản lý chất lượng sản phẩm, hàng hóa; tuyên truyền, phổ biến và tổ chức hướng dẫn pháp luật; hỗ trợ tổ chức, cá nhân sản xuất, kinh doanh tìm hiểu thông tin về chất lượng sản phẩm, hàng hóa;

g) Phối hợp với Bộ Khoa học và Công nghệ tổ chức thực hiện điều ước quốc tế, thỏa thuận quốc tế về thừa nhận lẫn nhau đối với kết quả đánh giá sự phù hợp.

2. Trách nhiệm quản lý nhà nước về chất lượng sản phẩm, hàng hoá xuất khẩu, nhập khẩu, lưu thông trên thị trường, trong quá trình sử dụng có khả năng gây mất an toàn được quy định như sau:

a) Bộ Y tế chịu trách nhiệm đối với thực phẩm, dược phẩm, vắc xin, sinh phẩm y tế, mỹ phẩm, nguyên liệu sản xuất thuốc và thuốc cho người, hoá chất gia dụng, chế phẩm diệt côn trùng, diệt khuẩn, trang thiết bị y tế;

b) Bộ Nông nghiệp và Phát triển nông thôn chịu trách nhiệm đối với cây trồng, vật nuôi, phân bón, thức ăn chăn nuôi, thuốc bảo vệ thực vật, thuốc thú y, chế phẩm sinh học dùng trong nông nghiệp, lâm nghiệp, thuỷ sản, công trình thuỷ lợi, đê điều;

c) Bộ Giao thông vận tải chịu trách nhiệm đối với phương tiện giao thông vận tải, phương tiện, thiết bị xếp dỡ, thi công vận tải chuyên dùng, phương tiện, thiết bị thăm dò, khai thác trên biển, công trình hạ tầng giao thông;

d) Bộ Công Thương chịu trách nhiệm đối với thiết bị áp lực, thiết bị nâng đặc thù chuyên ngành công nghiệp, hoá chất, vật liệu nổ công nghiệp, trang thiết bị khai thác mỏ, dầu khí, trừ các thiết bị, phương tiện thăm dò, khai thác trên biển;

đ) Bộ Xây dựng chịu trách nhiệm đối với công trình dân dụng, công trình công nghiệp, công trình hạ tầng kỹ thuật;

e) Bộ Quốc phòng chịu trách nhiệm đối với phương tiện, trang thiết bị quân sự, vũ khí đạn dược, khí tài, sản phẩm phục vụ quốc phòng, công trình quốc phòng;

g) Bộ Công an chịu trách nhiệm đối với trang thiết bị phòng cháy, chữa cháy; trang thiết bị kỹ thuật, vũ khí đạn dược, khí tài, công cụ hỗ trợ, trừ trường hợp quy định tại điểm e khoản này.

Chương VII ĐIỀU KHOẢN THI HÀNH

Điều 71. Hiệu lực thi hành

Luật này có hiệu lực thi hành từ ngày 01 tháng 7 năm 2008.

Pháp lệnh chất lượng hàng hóa ngày 24 tháng 12 năm 1999 hết hiệu lực kể từ ngày Luật này có hiệu lực.

Điều 72. Hướng dẫn thi hành

Chính phủ quy định chi tiết và hướng dẫn thi hành Luật này.

Luật này đã được Quốc hội nước Cộng hoà xã hội chủ nghĩa Việt Nam khoá XII, kỳ họp thứ 2 thông qua ngày 21 tháng 11 năm 2007.

CHỦ TỊCH QUỐC HỘI Nguyễn Phú Trọng

Luật chất lượng, sản phẩm hàng hoá số 05/2007/QH12 của Quốc hội

Cơ quan ban hành:

Quốc hội

Số công báo:

Đã biết

Số hiệu:

05/2007/QH12

Ngày đăng công báo:

Đã biết

Loại văn bản:

Luật

Người ký:

Nguyễn Phú Trọng

Ngày ban hành:

21/11/2007

Ngày hết hiệu lực:

Đang cập nhật

Áp dụng:

Đã biết

Tình trạng hiệu lực:

Đã biết

Lĩnh vực:

chúng tôi độc quyền cung cấp bản dịch chính thống Công báo tiếng Anh của Thông Tấn Xã Việt Nam

LUẬT

CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

CỦA QUỐC HỘI KHÓA XII, KỲ HỌP THỨ 2, SỐ 05/2007/QH12

NGÀY 21 THÁNG 11 NĂM 2007

Điều 2. Đối tượng áp dụng

Chương II. QUYỀN VÀ NGHĨA VỤ CỦA TỔ CHỨC, CÁ NHÂN ĐỐI VỚI CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Chương III. QUẢN LÝ CHẤT LƯỢNG SẢN PHẨM, HÀNG HÓA TRONG SẢN XUẤT, XUẤT KHẨU, NHẬP KHẨU, LƯU THÔNG TRÊN THỊ TRƯỜNG VÀ TRONG QUÁ TRÌNH SỬ DỤNG

Mục 1. QUY ĐỊNH CHUNG VỀ BIỆN PHÁP QUẢN LÝ CHẤT LƯỢNG SẢN PHẨM, HÀNG HÓA

Mục 2. QUẢN LÝ CHẤT LƯỢNG SẢN PHẨM TRONG SẢN XUẤT

3. Cơ quan kiểm tra chất lượng sản phẩm, hàng hoá thuộc Uỷ ban nhân dân tỉnh, thành phố trực thuộc trung ương thực hiện việc kiểm tra chất lượng sản phẩm, hàng hoá trong phạm vi của địa phương theo quy định của Bộ quản lý ngành, lĩnh vực.

Chương V. GIẢI QUYẾT TRANH CHẤP, BỒI THƯỜNG THIỆT HẠI, GIẢI QUYẾT KHIẾU NẠI, TỐ CÁO VÀ XỬ LÝ VI PHẠM PHÁP LUẬT VỀ CHẤT LƯỢNG SẢN PHẨM, HÀNG HOÁ

Điều 60. Các thiệt hại phải bồi thường do hàng hóa không bảo đảm chất lượng

Điều 62. Các trường hợp không phải bồi thường thiệt hại

Điều 66. Xử lý vi phạm pháp luật về chất lượng sản phẩm, hàng hoá

Điều 71. Hướng dẫn thi hành

Pháp lệnh Chất lượng hàng hóa ngày 24 tháng 12 năm 1999 hết hiệu lực kể từ ngày Luật này có hiệu lực.

Luật này đã được Quốc hội nước Cộng hoà xã hội chủ nghĩa Việt Nam khoá XII, kỳ họp thứ 2 thông qua ngày 21 tháng 11 năm 2007.

CHỦ TỊCH QUỐC HỘI

Nguyễn Phú Trọng

chúng tôi độc quyền cung cấp bản dịch chính thống Công báo tiếng Anh của Thông Tấn Xã Việt Nam

SOCIALIST REPUBLIC OF VIET NAM

Independence – Freedom – Happiness

ON PRODUCT AND GOODS QUALITY

(No. 05/2007/QH12)

Pursuant to the 1992 Constitution of the Socialist Republic of Vietnam with a number of articles already amended and supplemented under Resolution No. 51/2001/QH10;

The National Assembly promulgates the Law on Product and Goods Quality.

This Law provides for the rights and obligations of organizations and inpiduals producing or trading in products or goods as well as organizations and inpiduals conducting activities related to product and goods quality; and the management of product and goods quality.

This Law applies to organizations and inpiduals producing or trading in products or goods and organizations and inpiduals conducting activities related to product and goods quality in Vietnam.

In this Law, the terms below are construed as follows:

1. Product means the output of a production or service provision process for a commercial or consumption purpose.

2. Goods means products put on the market or for consumption through exchange, sale or marketing.

3. Products and goods incapable of causing unsafe (below referred to as group-1 products and goods) mean those products and goods which, under rational conditions of transportation, storage, pservation and use for proper purposes, cause no harms to humans, animals, plants, assets or the environment.

4. Products and goods capable of causing unsafe (below referred to as group-2 products and goods) mean those products and goods which, under rational conditions of transportation, storage, pservation and use for proper purposes, can latently cause harms to humans, animals, plants, assets or the environment.

5. Product and goods quality means the product and goods properties extent of satisfaction of the requirements under announced applicable standards or relevant technical regulations.

6. Production and business organizations and inpiduals mean organizations and inpiduals that organize and conduct production (below referred to as producers), import (below referred to as importers), export (below referred to as exporters), sale or service provision activities (below referred to as sellers).

7. Organizations and inpiduals involved in product and goods quality-related activities mean consumers, conformity evaluation organizations, professional organizations, consumer interest-protecting organizations, supervision agencies and state management agencies in charge of product and goods quality.

8. Conformity evaluation organization means an organization which tests, inspects, verifies and certifies the conformity of products, goods, production or service provision processes with announced applicable standards or relevant technical regulations.

9. Designated conformity evaluation organization means an organization which meets the conditions set in Clause 5, Article 25 of this Law and is considered and put on a list promulgated by a competent state agency for production and business organizations and inpiduals to select and use its conformity evaluation services to meet state management requirements.

10. Test means to conduct technical manipulations to identify one or properties of products and goods according to a certain process.

11. Inspect means to examine the conformity of products and goods with contracts or announced applicable standards or relevant technical regulations through observation and appraisal of measurement and test results.

12. Certify means to evaluate and attest the conformity of products, production or service provision processes with announced applicable standards (referred to as standard conformity certification) or with technical regulations (referred to as regulation conformity certification).

13. Verify means to conduct technical operations in a certain process aimed at assessing and attesting the products or goods satisfaction of the requirements of relevant technical regulations.

14. Recognition of conformity evaluation results means that a competent state agency, organization or inpidual operating in Vietnam recognizes conformity evaluation results given by a conformity evaluation organization of another nation or territory.

15. State examination of product and goods quality (below referred to as product and goods quality examination) means that a state agency reconsiders and reevaluates the quality of products, goods or production or service provision processes which have been evaluated in terms of quality by conformity evaluation organizations or to which other quality control measures have been applied by production and business organizations or inpiduals.

16. Agency in charge of state examination of product quality and goods (below referred to as product and goods quality examination agency) means an agency assigned or decentralized to perform the state control of product and goods quality under a line ministry or a specialized agency of the provincial/municipal Peoples Committee.

1. Production and business activities as well as product and goods quality-related activities must comply with this Law and relevant provisions of law.

2. Production and business activities as well as activities related to the quality of products and goods being construction works, services or used goods not subject to verification; products and goods used exclusively for defense or security and other particular ones must comply with general principles of this Law and be specifically governed by other legal documents.

3. When a treaty to which the Socialist Republic of Vietnam is a contracting party contains provisions different from those of this Law, the provisions of that treaty pvail.

1. Product and goods quality shall be controlled on the basis of announced applicable standards and relevant technical regulations. Depending on their capability of causing unsafe, products and goods are controlled as follows:

a/ The quality of group-1 products and goods shall be controlled on the basis of applicable standards announced by producers;

b/ The quality of group-2 products and goods shall be controlled on the basis of relevant technical regulations promulgated by competent state management agencies and applicable standards announced by producers.

The Government shall specify the promulgation of a list of group-2 products and goods.

2. Product and goods quality control is the responsibility of producers and traders, aimed to ensure safety for humans, animals, plants, assets and the environment and raise the productivity, quality and competitiveness of Vietnams products and goods.

3. State control of product and goods quality is the responsibility of competent state management agency, aimed to enforce legal provisions on product and goods quality.

Activities of state control of product and goods quality must ensure transparency, objectivity, no discrimination based on goods origin and organizations and inpiduals involved in product and goods quality-related activities, conformity with international practice, and protection of lawful rights and interests of production and business organizations and inpiduals as well as consumers.

2. To formulate national programs on raising the productivity, quality and competitiveness of products and goods.

3. To invest in and develop the testing system to meet the requirements of production, business and state control of product and goods quality.

4. To enhance training and retraining of human resources for product and goods quality control.

5. To propagate and disseminate policies and laws on product and goods quality; to form a sense of producing and trading in quality products and goods for consumers interest, which save energy and are environment-friendly; to raise social awareness about consumption and build civilized consumption habits.

6. To encourage and create conditions for domestic and foreign organizations and inpiduals to invest and participate in testing, inspecting, verifying and certifying product and goods quality.

7. To expand cooperation with other nations, territories, international organizations, regional organizations as well as foreign organizations and inpiduals in activities related to product and goods quality; to enhance the conclusion of treaties and agreements between Vietnam and other countries, territories, international or regional organizations on mutual recognition of conformity evaluation results; to encourage Vietnamese conformity evaluation organizations to sign agreements on recognition of conformity evaluation results given by corresponding organizations of other countries or territories so as to facilitate the development of trade between Vietnam and those countries or territories.

1. Product and goods quality prizes include national prizes and prizes awarded by organizations or inpiduals.

2. Conditions and procedures for consideration and award of national quality prizes are provided for by the Government.

3. Conditions and procedures for consideration and award of product and goods quality prizes of organizations and inpiduals are provided for by the Ministry of Science and Technology.

1. Manufacturing products, importing, selling or purchasing goods banned from circulation by the State.

2. Manufacturing products, importing, exporting, selling, purchasing, exchanging or marketing products or goods which fail to comply with relevant technical regulations.

3. Importing, exporting, selling or purchasing goods of unclear origin.

4. Importing, exporting, selling, purchasing goods, exchanging or marketing products or goods of which the use duration has expired.

5. Using foods or pharmaceuticals of poor quality or with expired use duration for charity purposes or donation.

6. Deliberately supplying untrue or forged results of testing, examination, inspection, verification and certification of product and goods quality.

7. Forging or illegally using standard- or regulation-conformity stamps or other signs of product and goods quality.

8. Replacing, fraudulently exchanging, adding or reducing ingredients or additives of, or mixing impurities with products and goods, thus reducing their quality compared with announced applicable standards or relevant technical regulations.

10. Concealing information on the products and goods capability of causing unsafe to humans, animals, plants, assets or the environment.

11. Manufacturing or processing products and goods with raw materials or materials banned from use for the production or processing of those products and goods.

12. Abusing ones positions or powers or the state control of product and goods quality to illegally obstruct, trouble or harass production and business activities of organizations and inpiduals or cover up acts of violation of the law on product and goods quality.

13. Abusing product and goods quality control activities to cause harms to national interests, social order and safety.

1. To decide on and announce the quality of products they manufacture or supply.

2. To decide on internal control measures in order to ensure product quality.

3. To select appropriate evaluation organizations for testing, verifying, inspecting and certifying product and goods quality.

In case of certification of regulation conformity, examination of product quality according to state management requirements, producers shall select designated conformity evaluation organizations.

4. To use standard- or regulation- conformity stamps and other signs for products according to law.

5. To request sellers to cooperate in the withdrawal and handling of goods of poor quality.

6. To complain about conclusions of examination teams or decisions of examination agencies or competent state management agencies.

7. To be entitled to compensation under the provisions of Section 2, Chapter V of this Law and relevant provisions of law.

1. To observe the conditions for ensuring the quality of products before they are put on the market according to Article 28 of this Law and take responsibility for products they manufacture.

2. To display quality information on goods labels, packing or in documents accompanying goods in accordance with legal provisions on goods labeling.

3. To supply truthful information on product and goods quality.

4. To warn about the products capability of causing unsafe and notify pventive methods to sellers and consumers.

5. To announce the requirements on transportation, storage, pservation and use of products and goods.

6. To supply purchasers and consumers with information on warranty and warranty services for products and goods.

7. To repair, refund money or exchange for new goods or receive back defective ones which are returned by sellers or consumers.

8. To promptly stop production, notify such to, concerned parties and take remedies when detecting that products or goods cause unsafe or fail to conform to announced applicable standards or relevant technical regulations.

9. To withdraw and handle products and goods of poor quality. If the goods must be destroyed, to bear all costs of goods destruction and be liable for its consequences in accordance with law.

10. To pay compensation under the provisions of Section 2, Chapter V of this Law and relevant provisions of law.

11. To observe competent state agencies regulations and decisions on inspection and supervision.

12. To pay costs of testing, standard- or regulation- conformity certification according to Article 31; sample-taking and testing costs according to Article 41; sample-taking, testing and inspection costs according to Article 58 of this Law.

13. To prove wrong results and faults of conformity evaluation organizations according to Clause 2. Article 63 of this Law.

1. To decide to select the level of quality of goods they import.

2. To request exporters to supply goods of proper quality as agreed upon in contracts.

3. To select inspection organizations to inspect the quality of goods they import.

4. To use standard- or regulation-conformity stamps and other signs for imported goods according to regulations.

5. To decide on internal control measures in order to maintain the quality of products and goods they import.

6. To request sellers to cooperate in withdrawing and handling goods of inferior quality.

7. To complain about conclusions of quality controllers or examination teams, or decisions of examination agencies or competent state management agencies.

8. To be entitled to compensation under the provisions of Section 2, Chapter V of this Law and relevant provisions of law.

1. To observe the conditions for ensuring the quality of imported goods according to Article 34 of this Law.

2. To take responsibility for the quality and labeling of goods they import in accordance with law.

3. To supply truthful information on product and goods quality.

4. To organize and control the process of transportation, storage and pservation in order to maintain goods quality.

5. To announce conditions which must be complied with during the transportation, storage and pservation of goods in accordance with law.

6. To warn sellers and consumers about the goods capability of causing unsafe and notify them of pventive measures.

7. To supply information on warranty and provide warranty service to sellers and consumers.

8. To repair, refund money or exchange for new goods or receive back defective ones returned by sellers or consumers.

9. To promptly stop the import, notify such to concerned parties and take remedies when detecting that the goods cause unsafe or fail to conform to announced applicable standards or relevant technical regulations.

10. To re-export imported goods which fail to conform to relevant technical regulations.

11. To destroy imported goods which fail to conform with relevant regulations but cannot be re-exported; to bear all costs of goods destruction and be liable for its consequences in accordance with law.

12. To withdraw and handle goods of poor quality.

13. To pay compensation under the provisions of Section 2, Chapter V of this Law and relevant provisions of law.

14. To observe competent state agencies regulations and decisions on inspection and supervision.

15. To pay goods quality examination costs and fees according to Article 37; sample-taking and testing costs according to Article 41; sample-taking, testing and expertise costs according to Article 58 of this Law.

1. To decide to select the level of quality of goods they export.

2. To select appropriate conformity evaluation organizations in order to test, inspect and certify the quality of export goods.

3. To decide on internal control measures in order to maintain the quality of goods till the time of transfer of ovnership of those goods to importers.

4. To use standard- or regulation-conformity stamps and other signs for exported goods according to regulations.

5. To request importers to cooperate in withdrawing and handling goods which fail to ensure the quality as agreed upon.

6. To complain about decisions of examination agencies or competent state management agencies.

7. To be entitled to compensation under the provisions of Section 2, Chapter V of this Law and relevant provisions of law.

1. To observe the conditions for ensuring the quality of exported goods according to Article 32 of this Law and take responsibility for goods quality.

2. To take measures for handling exported goods which fail to comply with Article 33 of this Law. If the goods must be destroyed, to bear all costs of goods destruction and take responsibility for its consequences in accordance with law.

3. To comply with competent state agencies regulations and decisions on inspection and supervision.

4. To pay costs of testing and standard- or regulation-conformity certification according to Article 31; sample-taking and testing costs according to Article 41 and sample-taking, testing and inspection costs according Article 58 of this Law.

1. To decide on modes of examination of goods quality.

2. To select appropriate evaluation organizations for testing and inspecting goods.

3. To decide on internal control measures in order to maintain goods quality.

4. To settle disputes under the provisions of Section 1, Chapter V of this Law and request producers or importers that have supplied goods to pay compensation for damage according to Clause 1, Article 61 of this Law.

5. To complain about conclusions of quality controllers, examination teams and decisions of examination agencies or competent state management agencies.

6. To be entitled to compensations under the provisions of Section 2, Chapter V of this Law and relevant provisions of. law.

1. To observe the conditions for ensuring the quality of goods circulated on the market according to Article 38 of this Law and take responsibility for goods quality.

2. To check the origin of goods, goods labels, standard- or regulation-conformity stamps and documents related to goods quality.

3. To supply truthful information on product and goods quality.

4. To apply measures to maintain the quality of goods during their transportation, storage and pservation.

5. To notify purchasers of conditions to be ensured during the transportation, storage, pservation and use of goods.

6. To supply purchasers with information on goods warranty.

7. To supply documents and information on goods subject to examination to quality controllers, and product and goods quality examination teams and agencies.

8. To promptly supply purchasers with information on the goods capability of causing unsafe and pventive measures upon receiving warnings from producers or importers.

9. To promptly stop selling goods and notifying producers, importers and purchasers when detecting unsafe goods or goods which fail to conform to announced applicable standards or relevant technical regulations.

10. To refund money or exchange goods for new ones or receive back defective goods returned by purchasers.

11. To cooperate with producers and importers in withdrawing and handling goods which fail to conform to announced applicable standards or relevant technical regulations.

12. To pay compensation under the provisions of Section 2, Chapter V of this Law and relevant provisions of law.

13. To comply with competent state agencies regulations and decisions on inspection and supervision.

14. To pay sample-taking and testing costs according to Article 41, and sample-taking and inspection costs according Article 58 of this Law.

1. To be supplied with truthful information on the safety and quality of products and goods and provided with instructions on their transportation, storage, pservation and use.

2. To be supplied with information on warranty for goods, their capability of causing unsafe and pventive measures upon receiving warnings from producers or importers.

3. To request sellers to repair, refund money for. exchange or receive back defective goods.

4. To be entitled to compensations under the provisions of Section 2, Chapter V of this Law and relevant provisions of law.

5. To request organizations and inpiduals producing and/or trading in goods to fulfill their responsibility for the protection of consumer interests according to the law on protection of consumer interests.

6. To request consumer interest-protecting organizations to protect their lawful interests according to the law on protection of consumer interests.

1. To observe the conditions for assurance of good quality in the course of using goods according to Article 42 of this Law.

2. To comply with producers, importers and sellers regulations and instructions on the transportation, storage, pservation and use of products and goods.

3. To comply with regulations on verification of the quality of products and goods quality in the process of using those on the lists issued by line ministries.

4. To observe legal provisions on environmental protection in the process of using products and goods.

1. To test, inspect, verify and certify product and goods quality on the basis of contracts signed with organizations and inpiduals requesting conformity evaluation in their registered operation domain or under designation by competent state agencies.

2. To get paid according to agreements with organizations or inpiduals producing and/or trading in products and goods that request conformity evaluation or at the request of competent state agencies.

3. To supply testing results to corresponding subjects of conformity evaluation.

4. To grant, re-grant, expand, narrow the scope or suspend and withdraw conformity evaluation certificates, the right to use standard – or regulation-conformity stamps already granted to corresponding subjects of verification or certification.

5. To refuse to supply information related to goods-quality test, inspection, verification and certification results to third parties, unless it is requested by a competent state agency.

6. To have their conformity evaluation results recognized by competence state agencies in accordance with law.

7. To collect costs of test and certification of standard or regulation conformity according to Article 31; charges and fees for examination of the quality of imported goods according to Article 37; testing costs according to Article 41; and testing and inspection costs according to Article 58 of this Law.

1. To meet the conditions set in Clause 5. Article 25 of this Law.

2. Not to refuse to supply services unless they can give plausible reasons.

3. To keep secret conformity evaluation information, data and results of organizations subject to conformity evaluation, unless they are requested by competent state agencies.

4. To ensure publicity, transparency, independence, objectivity, accuracy and no discrimination based on goods origin and organizations and inpiduals conducting product and goods quality-related activities.

5. To ensure the conformity evaluation order and procedures according to the law on standards and technical regulations.

6. To announce on the mass media the grant, re-grant, expansion, narrowing down of the scope, suspension or revocation of conformity certificates and the right to use standard – or regulation-conformity stamps.

7. To be subject to supervision or inspection of their conformity evaluation activities by competent state agencies.

8. To take responsibility before law for conformity evaluation results.

9. To pay fines to organizations and inpiduals with evaluated products and goods in case of supplying wrong conformity evaluation results. Fine levels shall be agreed upon by concerned parties but must not exceed 10 times the evaluation cost; if concerned parties cannot reach agreement, fine levels shall be decided by arbitration or a court but must not exceed 10 times the evaluation cost.

10. To pay compensation according to Clause 1, Article 63 of this Law.

1. To propagate and disseminate to production and business organizations and inpiduals knowledge about the application of the law on product and goods quality: to form a sense of responsibility for producing and trading in quality-products and goods for consumer interests, which save energy and are environment-friendly; to raise social awareness about consumption, form civilized consumption habits.

2. To support, and raise the awareness of, production and business organizations and inpiduals about and encourage them to apply the law on product and goods quality.

3. To organize training and retraining in the modes of product and goods quality control and social debate in the control of product and goods quality.

5. To lodge complaints or initiate lawsuits concerning disputes over product and goods quality which cause harms to their or their members interests.

1. To repsent consumers in protecting their lawful rights and interests when receiving complaints or reports on the quality of goods which fail to conform to announced applicable standards, relevant technical regulations, the quantity indicated on goods labels or the quality pscribed in contracts.

2. To receive information relating to organizations and inpiduals producing unconformable products or trading in unconformable goods, the degree of the goods non-conformity with announced applicable standards or corresponding technical regulations, and supply this information to mass media agencies, and concurrently take responsibility for their supplied information.

3. To propose supervision and inspection agencies or competent state agencies to handle production and business organizations for their violations related to product and goods quality.

4. To lodge complaints or initiate lawsuits about the quality of products or goods which cause harms to consumers.

1. Producers and importers shall announce fundamental properties, warning information and standard codes on goods on one of the following:

c/ Documents accompanying products or goods.

2. Contents of announced applicable standards must not be contrary to the requirements under technical regulations promulgated by competent state agencies.

* Lưu ý: Để đọc được văn bản tải trên chúng tôi bạn cần cài phần mềm đọc file DOC, DOCX và phần mềm đọc file PDF.


【#3】International Water Conference Technical Papers

Paper IWC 13-76 IWC 13-75 & D IWC 13-74 & D IWC 13-73 & D IWC 13-72 & D IWC 13-71 & D IWC 13-70 & D IWC 13-69 & D IWC 13-68 & D IWC 13-67 & D IWC 13-66 & D IWC 13-65 & D IWC 13-64 & D IWC 13-63 & D IWC 13-62 & D IWC 13-61 & D IWC 13-60 & D IWC 13-59 & D IWC 13-58 & D IWC 13-57 IWC 13-56 IWC 13-55 IWC 13-54 IWC 13-53 IWC 13-52 IWC 13-51 & D IWC 13-50 & D IWC 13-49 & D IWC 13-48 & D IWC 13-47 & D IWC 13-46 & D IWC 13-45 & D IWC 13-44 & D IWC 13-43 IWC 13-42 IWC 13-41 & D IWC 13-40 & D IWC 13-39 & D IWC 13-38 & D IWC 13-37 & D IWC 13-36 & D IWC 13-35 & D IWC 13-34 & D IWC 13-33 & D IWC 13-32 & D IWC 13-31 & D IWC 13-30 & D IWC 13-29 & D IWC 13-28 & D IWC 13-27 & D

Buy Now

Presentation Type

Buy Now and Download Paper Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Buy Now and Download Paper Buy Now and Download Paper Buy Now and Download Paper Buy Now and Download Paper Buy Now and Download Paper Buy Now and Download Paper Buy Now and Download Paper & Discussion Buy Now and Download Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion

Published? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC 13-26 & D IWC 13-25 & D IWC 13-24 & D IWC 13-23 & D IWC 13-22 & D IWC 13-21 & D IWC 13-19 IWC 13-18 & D IWC 13-17 & D IWC 13-16 & D IWC 13-15 & D IWC 13-14 & D IWC 13-13 & D IWC 13-12 & D IWC 13-11 & D IWC 13-10 & D IWC 13-09 & D IWC 13-08 & D IWC 13-07 & D IWC 13-06 & D IWC 13-05 & D IWC 13-04 & D IWC 13-03 IWC 13-02 & D IWC 13-01 IWC 12-76D IWC 12-76 IWC 12-75 IWC 12-74D IWC 12-74 IWC 12-73D IWC 12-73 IWC 12-72 IWC 12-71 IWC 12-69 IWC 12-68 IWC 12-67D IWC 12-67 IWC 12-66 IWC 12-65D IWC 12-65 IWC 12-64D IWC 12-64 IWC 12-63D IWC 12-63 IWC 12-62D IWC 12-62 IWC 12-61 IWC 12-60 IWC 12-59D IWC 12-59

Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper & Discussion Paper Paper & Discussion Paper Prepared Discussion Buy Now & DownloadPaper Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Buy Now & DownloadPaper Paper Buy Now & DownloadPaper Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Buy Now & DownloadPaper Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC 12-58 IWC 12-57D IWC 12-57 IWC 12-56 IWC 12-55D IWC 12-55 IWC 12-54D IWC 12-54 IWC 12-53 IWC 12-52 IWC 12-51 IWC 12-50D IWC 12-50 IWC 12-49 IWC 12-48D IWC 12-48 IWC 12-47D IWC 12-47 IWC 12-46D IWC 12-46 IWC 12-45D IWC 12-45 IWC 12-44D IWC 12-44 IWC 12-43D IWC 12-43 IWC 12-42D IWC 12-42 IWC 12-41D IWC 12-41 IWC 12-40 IWC 12-39 IWC 12-38 IWC 12-37 IWC 12-36D IWC 12-36 IWC 12-35D IWC 12-35 IWC 12-34 IWC 12-33D IWC 12-33 IWC 12-32 IWC 12-31 IWC 12-30D IWC 12-30 IWC 12-29D IWC 12-29 IWC 12-28 IWC 12-27D IWC 12-27 IWC 12-26D

Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Buy Now & DownloadPaper Prepared Discussion Buy Now & DownloadPaper Prepared Discussion Paper Paper Paper Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Paper Paper Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Paper Paper Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC 12-26 IWC 12-25 IWC 12-24D IWC 12-24 IWC 12-23D IWC 12-23 IWC 12-22D IWC 12-22 IWC 12-21D IWC 12-21 IWC 12-20D IWC 12-20 IWC 12-19D IWC 12-19 IWC 12-18D IWC 12-18 IWC 12-17D IWC 12-17 IWC 12-16D IWC 12-16 IWC 12-15 IWC 12-14D IWC 12-14 IWC 12-13D IWC 12-13 IWC 12-12 IWC 12-11 IWC 12-10D IWC 12-10 IWC 12-09D IWC 12-09 IWC 12-08D IWC 12-08 IWC 12-07D IWC 12-07 IWC 12-06 IWC 12-05 IWC 12-04 IWC 12-03D IWC 12-03 IWC 12-02 IWC 12-01D IWC 12-01 IWC 11-78 IWC 11-77D IWC 11-77 IWC 11-76D IWC 11-76 IWC 11-75D IWC 11-75 IWC 11-73D

Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Paper Prepared Discussion Paper Paper Prepared Discussion Buy Now & DownloadPaper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC 11-73 IWC 11-72D IWC 11-72 IWC 11-71D IWC 11-71 IWC 11-69D IWC 11-69 IWC 11-68D IWC 11-68 IWC 11-67D IWC 11-67 IWC 11-66D IWC 11-66 IWC 11-65D IWC 11-65 IWC 11-64D IWC 11-64 IWC 11-63D IWC 11-63 IWC 11-62D IWC 11-62 IWC 11-61D IWC 11-61 IWC 11-60D IWC 11-60 IWC 11-59D IWC 11-59 IWC 11-58D IWC 11-58 IWC 11-57 IWC 11-56 IWC 11-55 IWC 11-54 IWC 11-53 IWC 11-52D IWC 11-52 IWC 11-51D IWC 11-51 IWC 11-50D IWC 11-50 IWC 11-49D IWC 11-49 IWC 11-48D IWC 11-48 IWC 11-47D IWC 11-47 IWC 11-46D IWC 11-46 IWC 11-45D IWC 11-45 IWC 11-44D

Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Paper Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes No Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No

IWC 11-44 IWC 11-43D IWC 11-43 IWC 11-42D IWC 11-42 IWC 11-41D IWC 11-41 IWC 11-40D IWC 11-40 IWC 11-39D IWC 11-39 IWC 11-38D IWC 11-38 IWC 11-37D IWC 11-37 IWC 11-36D IWC 11-36 IWC 11-35D IWC 11-35 IWC 11-34D IWC 11-34 IWC 11-33D IWC 11-33 IWC 11-32D IWC 11-32 IWC 11-31D IWC 11-31 IWC 11-29D IWC 11-29 IWC 11-28D IWC 11-28 IWC 11-27D IWC 11-27 IWC 11-26 IWC 11-25D IWC 11-25 IWC 11-24D IWC 11-24 IWC 11-23D IWC 11-23 IWC 11-22 IWC 11-21D IWC 11-21 IWC 11-20D IWC 11-20 IWC 11-19 IWC 11-18 IWC 11-17D IWC 11-17 IWC 11-16D IWC 11-16

Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Prepared Discussion Paper Prepared Discussion Paper

Yes No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes No Yes No Yes Yes Yes Yes No Yes No Yes No Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC 11-15D IWC 11-15 IWC 11-14D IWC 11-14 IWC 11-13D IWC 11-13 IWC 11-12D IWC 11-12 IWC 11-11D IWC 11-11 IWC 11-10D IWC 11-10 IWC 11-09D IWC 11-09 IWC 11-08D IWC 11-08 IWC 11-07D IWC 11-07 IWC 11-06D IWC 11-06 IWC 11-05D IWC 11-05 IWC 11-04D IWC 11-04 IWC 11-03D IWC 11-03 IWC 11-02D IWC 11-02 IWC 11-01D IWC 11-01 IWC-10-71D IWC-10-71 IWC-10-70D IWC-10-70 IWC-10-69 IWC-10-68 IWC-10-67 IWC-10-66 IWC-10-65D IWC-10-65 IWC-10-64D IWC-10-64 IWC-10-63D IWC-10-63 IWC-10-62D IWC-10-62 IWC-10-61D IWC-10-61 IWC-10-60D IWC-10-60 IWC-10-59D

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes No Yes No Yes No Yes No Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-10-59 IWC-10-58 IWC-10-57 IWC-10-56D IWC-10-56 IWC-10-55 IWC-10-54D IWC-10-54 IWC-10-53D IWC-10-53 IWC-10-52D IWC-10-52 IWC-10-51D IWC-10-51 IWC-10-50D IWC-10-50 IWC-10-49D IWC-10-49 IWC-10-48D IWC-10-48 IWC-10-47D IWC-10-47 IWC-10-46D IWC-10-46 IWC-10-45D IWC-10-45 IWC-10-44D IWC-10-44 IWC-10-43D IWC-10-43 IWC-10-42D IWC-10-42 IWC-10-41D IWC-10-41 IWC-10-40D IWC-10-40 IWC-10-39D IWC-10-39 IWC-10-38D IWC-10-38 IWC-10-37D IWC-10-37 IWC-10-36D IWC-10-36 IWC-10-35 IWC-10-34D IWC-10-34 IWC-10-33D IWC-10-33 IWC-10-32D IWC-10-32

Paper Paper Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-10-31D IWC-10-31 IWC-10-30D IWC-10-30 IWC-10-29D IWC-10-29 IWC-10-28D IWC-10-28 IWC-10-27 IWC-10-25D IWC-10-25 IWC-10-24 IWC-10-23D IWC-10-23 IWC-10-21 IWC-10-20D IWC-10-20 IWC-10-19D IWC-10-19 IWC-10-18D IWC-10-18 IWC-10-17D IWC-10-17 IWC-10-16D IWC-10-16 IWC-10-15D IWC-10-15 IWC-10-14D IWC-10-14 IWC-10-13D IWC-10-13 IWC-10-12D IWC-10-12 IWC-10-11D IWC-10-11 IWC-10-10D IWC-10-10 IWC-10-09D IWC-10-09 IWC-10-08D IWC-10-08 IWC-10-07D IWC-10-07 IWC-10-06D IWC-10-06 IWC-10-04D IWC-10-04 IWC-10-03 IWC-10-02 IWC-10-01 IWC-09-S13PD

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Paper Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Paper Panel Discussion Transcript

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-09-S09PD IWC-09-70D IWC-09-70AC IWC-09-70 IWC-09-69D IWC-09-69 IWC-09-68 IWC-09-67D IWC-09-67 IWC-09-66 IWC-09-65D IWC-09-65 IWC-09-64D IWC-09-64 IWC-09-63D IWC-09-63 IWC-09-62D IWC-09-62 IWC-09-61D IWC-09-61AC IWC-09-61 IWC-09-60D IWC-09-60 IWC-09-59D IWC-09-59 IWC-09-58D IWC-09-58 IWC-09-57D IWC-09-57AC IWC-09-57 IWC-09-56 IWC-09-55 IWC-09-54D IWC-09-54 IWC-09-53D IWC-09-53 IWC-09-52D IWC-09-52 IWC-09-51 IWC-09-50D IWC-09-50 IWC-09-49D IWC-09-49 IWC-09-48D IWC-09-48 IWC-09-47D IWC-09-47 IWC-09-46 IWC-09-45 IWC-09-44 IWC-09-43

Panel Discussion Transcript Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Panel Report Panel Report Panel Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-09-42 IWC-09-41D IWC-09-41 IWC-09-40D IWC-09-40 IWC-09-39D IWC-09-39 IWC-09-38D IWC-09-38 IWC-09-37D IWC-09-37AC IWC-09-37 IWC-09-36D IWC-09-36 IWC-09-35D IWC-09-35 IWC-09-34D IWC-09-34 IWC-09-33D IWC-09-33 IWC-09-32D IWC-09-32 IWC-09-31 IWC-09-30 IWC-09-29 IWC-09-28 IWC-09-27D IWC-09-27AC IWC-09-27 IWC-09-26D IWC-09-26AC IWC-09-26 IWC-09-25D IWC-09-25 IWC-09-24D IWC-09-24 IWC-09-23D IWC-09-23AC IWC-09-23 IWC-09-22 IWC-09-21 IWC-09-20 IWC-09-19 IWC-09-18 IWC-09-17D IWC-09-17 IWC-09-16D IWC-09-16AC IWC-09-16 IWC-09-15 IWC-09-14

Panel Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-09-13D IWC-09-13 IWC-09-12D IWC-09-12 IWC-09-11D IWC-09-11 IWC-09-10D IWC-09-10 IWC-09-09D IWC-09-09 IWC-09-08D IWC-09-08 IWC-09-07D IWC-09-07 IWC-09-06D IWC-09-06 IWC-09-05D IWC-09-05 IWC-09-03D IWC-09-03AC IWC-09-03 IWC-09-02D IWC-09-02 IWC-09-01D IWC-09-01AC IWC-09-01 IWC-09~00 IWC-08-S08PD IWC-08-76D IWC-08-76 IWC-08-75D IWC-08-75 IWC-08-74D IWC-08-74 IWC-08-73D IWC-08-73 IWC-08-72D IWC-08-72 IWC-08-71D IWC-08-71 IWC-08-70D IWC-08-70 IWC-08-69D IWC-08-69AC IWC-08-69 IWC-08-68D IWC-08-68 IWC-08-67D IWC-08-67 IWC-08-66D IWC-08-66AC

Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Keynote Address No Panel Discussion Transcript Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Powerpoint Yes Paper Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Authors Closure Yes

IWC-08-66 IWC-08-65 IWC-08-64D IWC-08-64 IWC-08-63 IWC-08-61D3 IWC-08-61D2 IWC-08-61D1 IWC-08-61AC IWC-08-61 IWC-08-60 IWC-08-59 IWC-08-58 IWC-08-57 IWC-08-56D IWC-08-56 IWC-08-55D IWC-08-55 IWC-08-54 IWC-08-53D IWC-08-53 IWC-08-52D IWC-08-52 IWC-08-51D IWC-08-51 IWC-08-50D IWC-08-50 IWC-08-49D IWC-08-49 IWC-08-48D IWC-08-48 IWC-08-47D IWC-08-47AC IWC-08-47 IWC-08-46D IWC-08-46 IWC-08-45D IWC-08-45 IWC-08-44D IWC-08-44 IWC-08-43D IWC-08-43AC IWC-08-43 IWC-08-42D IWC-08-42 IWC-08-41D IWC-08-41 IWC-08-40D IWC-08-40 IWC-08-39D IWC-08-39AC

Paper Yes Report Yes Prepared Discussion Yes Paper Yes Report Yes Prepared Discussion Yes Prepared Discussion Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Report Yes Report Yes Report Yes Paper No Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Report Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Powerpoint Yes Authors Closure Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Authors Closure Yes

IWC-08-39 IWC-08-38D IWC-08-38 IWC-08-37 IWC-08-36D IWC-08-36 IWC-08-35D IWC-08-35 IWC-08-34 IWC-08-33D IWC-08-33 IWC-08-32D IWC-08-32 IWC-08-31D IWC-08-31 IWC-08-30 IWC-08-29 IWC-08-28 IWC-08-27 IWC-08-26 IWC-08-25D IWC-08-25 IWC-08-24D IWC-08-24AC IWC-08-24 IWC-08-23 IWC-08-22 IWC-08-21D IWC-08-21AC IWC-08-21 IWC-08-20 IWC-08-19 IWC-08-18D IWC-08-18 IWC-08-17 IWC-08-16D IWC-08-16 IWC-08-14 IWC-08-13 IWC-08-12 IWC-08-11 IWC-08-10D IWC-08-10 IWC-08-09 IWC-08-08D IWC-08-08 IWC-08-07D IWC-08-07 IWC-08-06 IWC-08-05D IWC-08-05

Paper Prepared Discussion Report Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Paper Report Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-08-04 IWC-08-03D IWC-08-03 IWC-08-02D IWC-08-02 IWC-08~S17PD IWC-08~S04PD IWC-08~00 IWC-07-78 IWC-07-77 IWC-07-75 IWC-07-71 IWC-07-70 IWC-07-69 IWC-07-68 IWC-07-67 IWC-07-66 IWC-07-65 IWC-07-64D IWC-07-64 IWC-07-63 IWC-07-62 IWC-07-61D IWC-07-61AC IWC-07-61 IWC-07-60D IWC-07-60 IWC-07-59D IWC-07-59 IWC-07-58D IWC-07-58 IWC-07-57D IWC-07-57AC IWC-07-57 IWC-07-56D IWC-07-56AC IWC-07-56 IWC-07-55D IWC-07-55AC IWC-07-55 IWC-07-54D IWC-07-54 IWC-07-53D IWC-07-53 IWC-07-52D IWC-07-52 IWC-07-51D IWC-07-51 IWC-07-50D IWC-07-50 IWC-07-49D

Report Prepared Discussion Paper Prepared Discussion Paper Panel Discussion Transcript Panel Discussion Transcript PowerPoint Report Poster Poster Report Report Paper Report Report Report Report Prepared Discussion Paper Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Paper Prepared Discussion Paper Prepared Discussion Authors Closure Prepared Discussion Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-07-49 IWC-07-48 IWC-07-47 IWC-07-46 IWC-07-45 IWC-07-44 IWC-07-43D IWC-07-43 IWC-07-42D IWC-07-42 IWC-07-41 IWC-07-40D IWC-07-40 IWC-07-39D IWC-07-39 IWC-07-38D IWC-07-38 IWC-07-37 IWC-07-36D IWC-07-36AC IWC-07-36 IWC-07-35D IWC-07-35 IWC-07-34D IWC-07-34 IWC-07-33 IWC-07-32 IWC-07-31 IWC-07-30 IWC-07-29 IWC-07-28 IWC-07-27 IWC-07-26D IWC-07-26 IWC-07-25D IWC-07-25 IWC-07-24D IWC-07-24AC IWC-07-24 IWC-07-23 IWC-07-22 IWC-07-21 IWC-07-20 IWC-07-19 IWC-07-18D IWC-07-18 IWC-07-17D IWC-07-17 IWC-07-16D IWC-07-16 IWC-07-15D

Paper Report Report Report Report Report Report Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Panel Report Panel Report Panel Report Panel Report Prepared Discussion Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-07-15 IWC-07-14D IWC-07-14 IWC-07-13D IWC-07-13 IWC-07-12D IWC-07-12 IWC-07-11 IWC-07-10D IWC-07-10 IWC-07-09D IWC-07-09 IWC-07-08D IWC-07-08 IWC-07-07D IWC-07-07 IWC-07-06 IWC-07-05D IWC-07-05 IWC-07-04 IWC-07-03D IWC-07-03 IWC-07-02 IWC-07~S08PD IWC-07~S06PD IWC-07~00 IWC-06-58 IWC-06-57 IWC-06-56 IWC-06-55 IWC-06-54 IWC-06-53D IWC-06-53 IWC-06-52D IWC-06-52 IWC-06-51D IWC-06-51 IWC-06-50D IWC-06-50AC IWC-06-50 IWC-06-49 IWC-06-48 IWC-06-47 IWC-06-46 IWC-06-45D IWC-06-45 IWC-06-44 IWC-06-43D IWC-06-43 IWC-06-42 IWC-06-41

Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Report Panel Discussion Transcript Panel Discussion Transcript Keynote Address Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-06-40D IWC-06-40AC IWC-06-40 IWC-06-39 IWC-06-38 IWC-06-37 IWC-06-36 IWC-06-35D IWC-06-35 IWC-06-34D IWC-06-34 IWC-06-33D IWC-06-33AC IWC-06-33 IWC-06-32 IWC-06-31 IWC-06-30 IWC-06-29 IWC-06-28 IWC-06-27 IWC-06-26 IWC-06-25 IWC-06-24 IWC-06-23 IWC-06-22 IWC-06-21 IWC-06-20D IWC-06-20 IWC-06-19 IWC-06-18D IWC-06-18 IWC-06-17D IWC-06-17 IWC-06-16 IWC-06-15 IWC-06-14D IWC-06-14 IWC-06-13 IWC-06-12 IWC-06-11 IWC-06-10 IWC-06-09D IWC-06-09AC IWC-06-09 IWC-06-08D IWC-06-08 IWC-06-07 IWC-06-06D IWC-06-06 IWC-06-05D IWC-06-05

Prepared Discussion Authors Closure Paper Panel Report Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Panel Report Panel Report Panel Report Panel Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Report Report Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-06-04 IWC-06-03 IWC-06-02D IWC-06-02AC IWC-06-02 IWC-06-01D IWC-06-01 IWC-06~S11PD IWC-06~S09PD IWC-06~00 IWC-05-81 IWC-05-80 IWC-05-79D IWC-05-79 IWC-05-78 IWC-05-77 IWC-05-76D IWC-05-76 IWC-05-75D IWC-05-75AC IWC-05-75 IWC-05-74 IWC-05-73D IWC-05-73 IWC-05-72D IWC-05-72 IWC-05-71 IWC-05-70D IWC-05-70 IWC-05-69 IWC-05-68D IWC-05-68 IWC-05-67 IWC-05-66D IWC-05-66 IWC-05-64D IWC-05-64 IWC-05-63D IWC-05-63 IWC-05-62D IWC-05-62 IWC-05-61D IWC-05-61 IWC-05-60 IWC-05-59 IWC-05-58 IWC-05-57 IWC-05-56 IWC-05-55D IWC-05-55 IWC-05-54D

Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Panel Discussion Transcript Panel Discussion Transcript Keynote Address Report Report Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Report Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Report Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-05-54 IWC-05-53 IWC-05-52 IWC-05-51 IWC-05-50D IWC-05-50AC IWC-05-50 IWC-05-49D IWC-05-49 IWC-05-48D IWC-05-48 IWC-05-47 IWC-05-46 IWC-05-45D IWC-05-45 IWC-05-44D IWC-05-44 IWC-05-43 IWC-05-41 IWC-05-40D IWC-05-40 IWC-05-39 IWC-05-38 IWC-05-37D IWC-05-37 IWC-05-36 IWC-05-35 IWC-05-34 IWC-05-33D IWC-05-33 IWC-05-32D IWC-05-32 IWC-05-31D IWC-05-31 IWC-05-30D IWC-05-30 IWC-05-29D IWC-05-29AC IWC-05-29 IWC-05-28 IWC-05-27 IWC-05-26D IWC-05-26 IWC-05-25D IWC-05-25 IWC-05-24D IWC-05-24AC IWC-05-24 IWC-05-23D IWC-05-23 IWC-05-22

Paper Report Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Report Report Prepared Discussion Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-05-21 IWC-05-20 IWC-05-19 IWC-05-18 IWC-05-17D IWC-05-17 IWC-05-16D IWC-05-16 IWC-05-15D IWC-05-15AC IWC-05-15 IWC-05-14 IWC-05-13 IWC-05-12 IWC-05-11 IWC-05-10 IWC-05-09 IWC-05-08D IWC-05-08 IWC-05-07D IWC-05-07 IWC-05-06D IWC-05-06 IWC-05-05D IWC-05-05 IWC-05-04D IWC-05-04 IWC-05-03 IWC-05-02D IWC-05-02AC IWC-05-02 IWC-05-01D IWC-05-01 IWC-05~S15PD IWC-05~00 IWC-04-52 IWC-04-51D IWC-04-51AC IWC-04-51 IWC-04-48D IWC-04-48 IWC-04-47 IWC-04-46 IWC-04-44 IWC-04-43 IWC-04-42 IWC-04-41D IWC-04-41 IWC-04-40D IWC-04-40 IWC-04-39D

Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Report Report Prepared Discussion Authors Closure Paper Report Paper Panel Discussion Transcript Keynote Address Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Paper Report Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-04-39 IWC-04-38 IWC-04-37 IWC-04-36 IWC-04-35 IWC-04-34 IWC-04-33 IWC-04-32 IWC-04-31D IWC-04-31 IWC-04-30 IWC-04-29 IWC-04-28 IWC-04-27 IWC-04-26 IWC-04-23D IWC-04-23 IWC-04-22D IWC-04-22 IWC-04-21D IWC-04-21 IWC-04-20D IWC-04-20 IWC-04-19 IWC-04-16 IWC-04-15 IWC-04-13 IWC-04-12 IWC-04-11D IWC-04-11AC IWC-04-11 IWC-04-10D IWC-04-10 IWC-04-09D IWC-04-09 IWC-04-08D IWC-04-08 IWC-04-07D IWC-04-07 IWC-04-06D IWC-04-06 IWC-04-05D IWC-04-05 IWC-04-04D IWC-04-04 IWC-04-02D IWC-04-02 IWC-04-01D IWC-04-01 IWC-04~S10PD IWC-04~S08PD

Paper Panel Report Panel Report Panel Report Panel Report Report Report Report Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Paper Panel Discussion Transcript Panel Discussion Transcript

Yes Yes No Yes Yes Yes Yes No Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-04~00 IWC-03-48 IWC-03-47D IWC-03-47 IWC-03-46D IWC-03-46 IWC-03-45D IWC-03-45 IWC-03-44D IWC-03-44 IWC-03-43D IWC-03-43 IWC-03-42 IWC-03-41 IWC-03-40D IWC-03-40 IWC-03-39D IWC-03-39AC IWC-03-39 IWC-03-38D IWC-03-38 IWC-03-37 IWC-03-36D IWC-03-36 IWC-03-35 IWC-03-34D IWC-03-34 IWC-03-33D IWC-03-33 IWC-03-32D IWC-03-32 IWC-03-31 IWC-03-30 IWC-03-29D IWC-03-29 IWC-03-28D IWC-03-28 IWC-03-27 IWC-03-26 IWC-03-25 IWC-03-24 IWC-03-23 IWC-03-22D IWC-03-22 IWC-03-21D IWC-03-21AC IWC-03-21 IWC-03-20 IWC-03-19 IWC-03-18D IWC-03-18

Keynote Address Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Paper Paper Paper Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Paper

No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes

IWC-03-17D IWC-03-17 IWC-03-15D IWC-03-15 IWC-03-14D IWC-03-14 IWC-03-13 IWC-03-12 IWC-03-11 IWC-03-10 IWC-03-09 IWC-03-08 IWC-03-07D IWC-03-07 IWC-03-06D IWC-03-06 IWC-03-05 IWC-03-04D IWC-03-04 IWC-03-03 IWC-03-02D IWC-03-02 IWC-03-01D IWC-03-01 IWC-03~S05PD IWC-03~00 IWC-02-66D IWC-02-66AC IWC-02-66 IWC-02-65 IWC-02-64D IWC-02-64 IWC-02-63 IWC-02-62 IWC-02-61 IWC-02-60 IWC-02-59 IWC-02-56 IWC-02-55 IWC-02-54 IWC-02-53 IWC-02-52 IWC-02-50D IWC-02-50 IWC-02-49 IWC-02-48 IWC-02-46D IWC-02-46 IWC-02-45 IWC-02-44D IWC-02-44

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Panel Discussion Transcript Keynote Address Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Report Report Report Report Report Panel Report Panel Report Panel Report Panel Report Report Prepared Discussion Paper Report Report Prepared Discussion Paper Report Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-02-42 IWC-02-41 IWC-02-40 IWC-02-39 IWC-02-38 IWC-02-37 IWC-02-36 IWC-02-35 IWC-02-34 IWC-02-33 IWC-02-32D IWC-02-32 IWC-02-31 IWC-02-30D IWC-02-30 IWC-02-29 IWC-02-28 IWC-02-27 IWC-02-25 IWC-02-23D IWC-02-23 IWC-02-22 IWC-02-21 IWC-02-20 IWC-02-19 IWC-02-18 IWC-02-17D IWC-02-17 IWC-02-16D IWC-02-16AC IWC-02-16 IWC-02-15D IWC-02-15AC IWC-02-15 IWC-02-14D IWC-02-14 IWC-02-13D IWC-02-13 IWC-02-12D IWC-02-12AC IWC-02-12 IWC-02-11 IWC-02-10 IWC-02-09 IWC-02-08 IWC-02-07 IWC-02-06D IWC-02-06 IWC-02-05D IWC-02-05 IWC-02-04

Report Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Panel Report Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Paper Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes

IWC-02-02D IWC-02-02 IWC-02-01 IWC-02~S14PD IWC-02~S06PD IWC-02~S03PD IWC-01-55D IWC-01-55 IWC-01-54 IWC-01-53 IWC-01-52 IWC-01-51 IWC-01-50D IWC-01-50 IWC-01-49 IWC-01-48D IWC-01-48 IWC-01-46D IWC-01-46AC IWC-01-46 IWC-01-45 IWC-01-44 IWC-01-43 IWC-01-42D IWC-01-42AC IWC-01-42 IWC-01-40D IWC-01-40 IWC-01-39 IWC-01-38D IWC-01-38AC IWC-01-38 IWC-01-37D IWC-01-37 IWC-01-36 IWC-01-35D IWC-01-35 IWC-01-34 IWC-01-33D IWC-01-33 IWC-01-32D IWC-01-32 IWC-01-31 IWC-01-30AC IWC-01-30 IWC-01-29 IWC-01-28 IWC-01-27D IWC-01-27 IWC-01-26D IWC-01-26

Prepared Discussion Yes Paper Yes Report Yes Panel Discussion Transcript Yes Panel Discussion Transcript Yes Panel Discussion Transcript Yes Prepared Discussion Yes Paper Yes Report Yes Report Yes Report No Report Yes Report Yes Paper Yes Report No Prepared Discussion Yes Paper Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Report Yes Report Yes Report Yes Prepared Discussion No Authors Closure Yes Paper Yes Prepared Discussion Yes Paper Yes Report Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Prepared Discussion Yes Paper Yes Report Yes Prepared Discussion Yes Paper Yes Report Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Panel Report No Panel Report & Discussion TranscYes Panel Report No Panel Report No Report Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes

IWC-01-25D IWC-01-25 IWC-01-24D IWC-01-24 IWC-01-23 IWC-01-22 IWC-01-21D IWC-01-21 IWC-01-20 IWC-01-19 IWC-01-18 IWC-01-17 IWC-01-16 IWC-01-15D IWC-01-15 IWC-01-14D IWC-01-14 IWC-01-13 IWC-01-12 IWC-01-11 IWC-01-10 IWC-01-09 IWC-01-08 IWC-01-07 IWC-01-06 IWC-01-05D IWC-01-05 IWC-01-04D IWC-01-04 IWC-01-03 IWC-01-02D IWC-01-02 IWC-01-01D IWC-01-01 IWC-01~S09PD IWC-01~S06PD IWC-01~00 IWC-00-60 IWC-00-59 IWC-00-58 IWC-00-57 IWC-00-56 IWC-00-55 IWC-00-54 IWC-00-53 IWC-00-52 IWC-00-50D IWC-00-50 IWC-00-49 IWC-00-48D IWC-00-48

Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Report Report Prepared Discussion Paper Report Paper Report Prepared Discussion Paper Prepared Discussion Paper Panel Discussion Transcript Panel Discussion Transcript Keynote Address Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-00-47 IWC-00-46 IWC-00-45D IWC-00-45 IWC-00-44 IWC-00-43 IWC-00-42 IWC-00-41 IWC-00-40 IWC-00-39 IWC-00-38 IWC-00-37D IWC-00-37 IWC-00-35 IWC-00-34D IWC-00-34 IWC-00-33 IWC-00-32 IWC-00-31 IWC-00-30 IWC-00-29 IWC-00-28 IWC-00-27 IWC-00-26D IWC-00-26 IWC-00-25 IWC-00-24D IWC-00-24 IWC-00-23D IWC-00-23 IWC-00-22 IWC-00-21D IWC-00-21 IWC-00-20 IWC-00-19 IWC-00-18D IWC-00-18 IWC-00-17D IWC-00-17AC IWC-00-17 IWC-00-16D IWC-00-16 IWC-00-15D IWC-00-15 IWC-00-14 IWC-00-13 IWC-00-12D IWC-00-12 IWC-00-11D IWC-00-11 IWC-00-10

Report Report Prepared Discussion Paper Paper Report Report Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Paper Prepared Discussion Paper Paper Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-00-08D IWC-00-08 IWC-00-07D IWC-00-07 IWC-00-06D IWC-00-06 IWC-00-05D IWC-00-05 IWC-00-04D IWC-00-04 IWC-00-03 IWC-00-02D IWC-00-02 IWC-00-01 IWC-00~S09PD IWC-00~00 IWC-99-81 IWC-99-80 IWC-99-79D IWC-99-79 IWC-99-77D IWC-99-77 IWC-99-76 IWC-99-75 IWC-99-74 IWC-99-73D IWC-99-73AC IWC-99-73 IWC-99-72 IWC-99-71D IWC-99-71AC IWC-99-71 IWC-99-70 IWC-99-69D IWC-99-69 IWC-99-68D IWC-99-68 IWC-99-67D IWC-99-67 IWC-99-66D IWC-99-66 IWC-99-65 IWC-99-64D IWC-99-64 IWC-99-63D IWC-99-63 IWC-99-62 IWC-99-61D IWC-99-61 IWC-99-60 IWC-99-58

Report Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Panel Discussion Transcript Keynote Address Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Authors Closure Paper Report Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes No Yes No Yes Yes Yes Yes

IWC-99-57D IWC-99-57 IWC-99-56D IWC-99-56 IWC-99-55D IWC-99-55 IWC-99-54D IWC-99-54 IWC-99-53 IWC-99-52 IWC-99-51 IWC-99-50D IWC-99-50 IWC-99-49 IWC-99-48 IWC-99-47D IWC-99-47AC IWC-99-47 IWC-99-46 IWC-99-45D IWC-99-45 IWC-99-44D IWC-99-44 IWC-99-43 IWC-99-42 IWC-99-41D IWC-99-41 IWC-99-40D IWC-99-40 IWC-99-39 IWC-99-38D IWC-99-38AC IWC-99-38 IWC-99-37D IWC-99-37 IWC-99-36D IWC-99-36 IWC-99-35D IWC-99-35AC IWC-99-35 IWC-99-34 IWC-99-33D IWC-99-33 IWC-99-32D IWC-99-32 IWC-99-31D IWC-99-31AC IWC-99-31 IWC-99-30 IWC-99-29 IWC-99-28D

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Report Report Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Prepared Discussion

No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes

IWC-99-28 IWC-99-27D IWC-99-27 IWC-99-26 IWC-99-25 IWC-99-24 IWC-99-23D IWC-99-23 IWC-99-22D IWC-99-22 IWC-99-21D IWC-99-21 IWC-99-20D IWC-99-20 IWC-99-19 IWC-99-18 IWC-99-17 IWC-99-16 IWC-99-15 IWC-99-14 IWC-99-13 IWC-99-12 IWC-99-11D IWC-99-11AC IWC-99-11 IWC-99-10D IWC-99-10 IWC-99-09D IWC-99-09 IWC-99-08D IWC-99-08 IWC-99-07D IWC-99-07 IWC-99-06 IWC-99-05 IWC-99-04 IWC-99-03D IWC-99-03 IWC-99-02D IWC-99-02 IWC-99-01 IWC-99~S08PD IWC-99~S06PD IWC-99~S05PD IWC-99~00 IWC-98-80D IWC-98-80 IWC-98-78 IWC-98-76D IWC-98-76 IWC-98-75D

Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Panel Discussion Transcript Panel Discussion Transcript Panel Discussion Transcript Keynote Address Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion

Yes No Yes No Yes Yes Yes Yes No Yes No Yes No Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes No Yes No Yes No Yes Yes Yes Yes No Yes No Yes No Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes

IWC-98-75 IWC-98-74 IWC-98-71 IWC-98-70 IWC-98-69 IWC-98-68 IWC-98-67D IWC-98-67 IWC-98-66 IWC-98-65 IWC-98-64D IWC-98-64 IWC-98-63D IWC-98-63 IWC-98-62D IWC-98-62 IWC-98-61D IWC-98-61 IWC-98-60D IWC-98-60AC IWC-98-60 IWC-98-59 IWC-98-57D IWC-98-57AC IWC-98-57 IWC-98-56D IWC-98-56 IWC-98-55 IWC-98-54D IWC-98-54 IWC-98-53D IWC-98-53 IWC-98-50D IWC-98-50 IWC-98-49 IWC-98-48 IWC-98-47D IWC-98-47 IWC-98-46 IWC-98-44D IWC-98-44 IWC-98-43 IWC-98-42D IWC-98-42 IWC-98-40 IWC-98-39 IWC-98-38D IWC-98-38 IWC-98-37D IWC-98-37 IWC-98-36

Paper Report Paper Report Report Report Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report

Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-98-33 IWC-98-32 IWC-98-31 IWC-98-30 IWC-98-29 IWC-98-28D IWC-98-28 IWC-98-27D IWC-98-27 IWC-98-26D IWC-98-26 IWC-98-25 IWC-98-24 IWC-98-23 IWC-98-22 IWC-98-21 IWC-98-20 IWC-98-19 IWC-98-18 IWC-98-17D IWC-98-17 IWC-98-16 IWC-98-15 IWC-98-14 IWC-98-13 IWC-98-12 IWC-98-11 IWC-98-09D IWC-98-09 IWC-98-08D IWC-98-08 IWC-98-07D IWC-98-07 IWC-98-05 IWC-98-04D IWC-98-04 IWC-98-03 IWC-98-02 IWC-98-01 IWC-98~S04PD IWC-98~00 IWC-97-84D IWC-97-84 IWC-97-83 IWC-97-81 IWC-97-80 IWC-97-79 IWC-97-78 IWC-97-77D IWC-97-77 IWC-97-76D

Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Paper Report Report Report Report Report Report Report Report Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Panel Discussion Transcript Keynote Address Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Paper Prepared Discussion

No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes No Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes

IWC-97-76 IWC-97-75D IWC-97-75 IWC-97-74D IWC-97-74 IWC-97-73D IWC-97-73 IWC-97-72D IWC-97-72 IWC-97-71 IWC-97-70D IWC-97-70AC IWC-97-70 IWC-97-69D IWC-97-69 IWC-97-68 IWC-97-67D IWC-97-67 IWC-97-66D IWC-97-66 IWC-97-65D IWC-97-65 IWC-97-64 IWC-97-63D IWC-97-63 IWC-97-61D IWC-97-61 IWC-97-60D IWC-97-60 IWC-97-59 IWC-97-58D IWC-97-58 IWC-97-57 IWC-97-56 IWC-97-55 IWC-97-53D IWC-97-53 IWC-97-52 IWC-97-51D IWC-97-51 IWC-97-50 IWC-97-49 IWC-97-48 IWC-97-47 IWC-97-46 IWC-97-45D2 IWC-97-45D1 IWC-97-45 IWC-97-43 IWC-97-42D IWC-97-42

Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Prepared Discussion Paper Report Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-97-41D IWC-97-41 IWC-97-40D IWC-97-40 IWC-97-39D IWC-97-39 IWC-97-38D IWC-97-38 IWC-97-37 IWC-97-36 IWC-97-35 IWC-97-34 IWC-97-32 IWC-97-31D IWC-97-31 IWC-97-30 IWC-97-29D IWC-97-29 IWC-97-28D IWC-97-28 IWC-97-27D IWC-97-27 IWC-97-26 IWC-97-25D IWC-97-25AC IWC-97-25 IWC-97-24D IWC-97-24 IWC-97-23 IWC-97-21D IWC-97-21 IWC-97-20 IWC-97-19 IWC-97-18 IWC-97-17D IWC-97-17AC IWC-97-17 IWC-97-16D IWC-97-16 IWC-97-15D IWC-97-15 IWC-97-14D IWC-97-14 IWC-97-13D IWC-97-13 IWC-97-12D IWC-97-12 IWC-97-11 IWC-97-10 IWC-97-09 IWC-97-08

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-97-07 IWC-97-06D IWC-97-06 IWC-97-05D IWC-97-05 IWC-97-04D IWC-97-04 IWC-97-03D IWC-97-03 IWC-97-01 IWC-97~S15FD IWC-97~S06PD IWC-97~S06FD IWC-97~S02FD IWC-96-79D IWC-96-79 IWC-96-78D IWC-96-78 IWC-96-77D IWC-96-77 IWC-96-76D IWC-96-76 IWC-96-75D IWC-96-75 IWC-96-74D IWC-96-74 IWC-96-73D IWC-96-73 IWC-96-72D IWC-96-72 IWC-96-71 IWC-96-70D IWC-96-70 IWC-96-69 IWC-96-68D IWC-96-68 IWC-96-67D IWC-96-67 IWC-96-65D IWC-96-65 IWC-96-64 IWC-96-63D IWC-96-63AC IWC-96-63 IWC-96-62 IWC-96-61D IWC-96-61 IWC-96-59 IWC-96-58D IWC-96-58 IWC-96-57D

Report Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Report Yes Open Floor Discussion TranscriptYes Panel Discussion Transcript Yes Moderated Open Floor DiscussionYes Floor Discussion Transcript Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion No Paper Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Report No Paper Yes Prepared Discussion Yes Report Yes Report Yes Prepared Discussion Yes Paper Yes Report Yes Prepared Discussion Yes Paper Yes Prepared Discussion Yes Paper Yes Report No Paper Yes Report Yes Prepared Discussion Yes Authors Closure Yes Paper Yes Report Yes Report Yes Paper Yes Report Yes Prepared Discussion Yes Paper Yes Prepared Discussion No

IWC-96-57 IWC-96-56 IWC-96-55 IWC-96-54D IWC-96-54 IWC-96-53D IWC-96-53 IWC-96-52D IWC-96-52 IWC-96-51 IWC-96-50D IWC-96-50 IWC-96-49D IWC-96-49 IWC-96-48D IWC-96-48 IWC-96-47 IWC-96-46 IWC-96-45 IWC-96-44D IWC-96-44 IWC-96-43D IWC-96-43 IWC-96-42D IWC-96-42 IWC-96-41D IWC-96-41 IWC-96-40D IWC-96-40 IWC-96-39D IWC-96-39 IWC-96-38D IWC-96-38 IWC-96-37D IWC-96-37 IWC-96-36D IWC-96-36 IWC-96-35D IWC-96-35 IWC-96-34 IWC-96-33 IWC-96-32 IWC-96-30D IWC-96-30 IWC-96-29D IWC-96-29 IWC-96-28D IWC-96-28 IWC-96-27 IWC-96-26D IWC-96-26AC

Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Report Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes No Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-96-26 IWC-96-25D IWC-96-25 IWC-96-24D IWC-96-24 IWC-96-23D IWC-96-23 IWC-96-22D IWC-96-22 IWC-96-21D IWC-96-21 IWC-96-20D IWC-96-20 IWC-96-19 IWC-96-18 IWC-96-17 IWC-96-16 IWC-96-15D IWC-96-15 IWC-96-14D IWC-96-14 IWC-96-13D IWC-96-13 IWC-96-12 IWC-96-11 IWC-96-10 IWC-96-09 IWC-96-08D IWC-96-08AC IWC-96-08 IWC-96-07 IWC-96-06D IWC-96-06AC IWC-96-06 IWC-96-05 IWC-96-04 IWC-96-03D IWC-96-03 IWC-96-02 IWC-96-01D IWC-96-01 IWC-96~S06FD IWC-96~S02FD IWC-95-71 IWC-95-70 IWC-95-69 IWC-95-68D IWC-95-68 IWC-95-67 IWC-95-66 IWC-95-62

Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Authors Closure Paper Report Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Paper Report Prepared Discussion Paper Floor Discussion Transcript Floor Discussion Transcript Poster Poster Poster Prepared Discussion Paper Report Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes

IWC-95-61 IWC-95-60 IWC-95-59D IWC-95-59 IWC-95-58D IWC-95-58 IWC-95-57 IWC-95-56 IWC-95-55D IWC-95-55 IWC-95-54D IWC-95-54 IWC-95-53D IWC-95-53 IWC-95-52D IWC-95-52 IWC-95-51D IWC-95-51 IWC-95-50D IWC-95-50 IWC-95-49D IWC-95-49 IWC-95-48D IWC-95-48 IWC-95-47D IWC-95-47 IWC-95-46D IWC-95-46 IWC-95-45D IWC-95-45 IWC-95-44 IWC-95-43 IWC-95-42D IWC-95-42 IWC-95-41D IWC-95-41 IWC-95-40D IWC-95-40AC IWC-95-40 IWC-95-39 IWC-95-38 IWC-95-37D IWC-95-37AC IWC-95-37 IWC-95-36 IWC-95-34 IWC-95-33 IWC-95-32 IWC-95-31 IWC-95-30 IWC-95-29

Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Paper Report Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Authors Closure Paper Report Report Report Report Report Report Report

Yes Yes Yes Yes Yes Yes Yes No Yes Yes No Yes Yes Yes Yes Yes Yes Yes No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes

IWC-95-28D IWC-95-28 IWC-95-27D IWC-95-27 IWC-95-26D IWC-95-26AC IWC-95-26 IWC-95-25 IWC-95-24 IWC-95-23D IWC-95-23 IWC-95-22D IWC-95-22 IWC-95-21 IWC-95-20 IWC-95-19D IWC-95-19 IWC-95-17D IWC-95-17 IWC-95-16 IWC-95-15 IWC-95-14D IWC-95-14AC IWC-95-14 IWC-95-12D IWC-95-12 IWC-95-11 IWC-95-10D IWC-95-10 IWC-95-09D IWC-95-09 IWC-95-08 IWC-95-07 IWC-95-06D IWC-95-06 IWC-95-05D IWC-95-05 IWC-95-04 IWC-95-03 IWC-95-02 IWC-95-01D IWC-95-01 IWC-94-76D IWC-94-76 IWC-94-75D IWC-94-75 IWC-94-74 IWC-94-73 IWC-94-69 IWC-94-68 IWC-94-67D

Prepared Discussion Paper Prepared Discussion Paper Report Authors Closure Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Report Paper Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion

Yes Yes No No Yes Yes Yes Yes No Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-94-67 IWC-94-66D IWC-94-66 IWC-94-65 IWC-94-63 IWC-94-62 IWC-94-61D IWC-94-61 IWC-94-60D IWC-94-60 IWC-94-58D IWC-94-58 IWC-94-57D IWC-94-57 IWC-94-56D IWC-94-56 IWC-94-55D IWC-94-55 IWC-94-54 IWC-94-53 IWC-94-52D IWC-94-52 IWC-94-51D IWC-94-51 IWC-94-50D IWC-94-50 IWC-94-48D IWC-94-48 IWC-94-47D IWC-94-47AC IWC-94-47 IWC-94-46D IWC-94-46 IWC-94-45 IWC-94-44D IWC-94-44 IWC-94-42D IWC-94-42 IWC-94-41 IWC-94-40D IWC-94-40 IWC-94-39 IWC-94-38D IWC-94-38 IWC-94-37D IWC-94-37 IWC-94-36 IWC-94-35 IWC-94-34D IWC-94-34 IWC-94-33D

Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes No Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-94-33 IWC-94-32D IWC-94-32 IWC-94-31 IWC-94-30 IWC-94-29D IWC-94-29 IWC-94-28D IWC-94-28 IWC-94-27 IWC-94-26 IWC-94-25 IWC-94-24D IWC-94-24 IWC-94-23D IWC-94-23 IWC-94-22 IWC-94-21 IWC-94-20D IWC-94-20 IWC-94-19D IWC-94-19 IWC-94-18D IWC-94-18 IWC-94-17 IWC-94-16 IWC-94-15 IWC-94-14 IWC-94-13 IWC-94-12 IWC-94-11 IWC-94-10D IWC-94-10 IWC-94-09D IWC-94-09 IWC-94-08 IWC-94-07 IWC-94-06D IWC-94-06 IWC-94-05D IWC-94-05 IWC-94-04 IWC-94-03 IWC-94-02D IWC-94-02 IWC-93-65D IWC-93-65 IWC-93-63 IWC-93-62 IWC-93-60 IWC-93-59

Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report

Yes No Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes

IWC-93-58 IWC-93-57 IWC-93-56 IWC-93-55 IWC-93-54D IWC-93-54 IWC-93-53 IWC-93-52 IWC-93-51 IWC-93-50D IWC-93-50 IWC-93-49D IWC-93-49 IWC-93-48D IWC-93-48 IWC-93-47 IWC-93-46D IWC-93-46 IWC-93-45 IWC-93-44 IWC-93-43 IWC-93-42 IWC-93-41 IWC-93-40 IWC-93-39D IWC-93-39AC IWC-93-39 IWC-93-38D IWC-93-38 IWC-93-37D IWC-93-37 IWC-93-36D IWC-93-36 IWC-93-35AC IWC-93-35 IWC-93-34D IWC-93-34 IWC-93-33D IWC-93-33 IWC-93-32D IWC-93-32 IWC-93-31D IWC-93-31 IWC-93-28 IWC-93-27D IWC-93-27AC IWC-93-27 IWC-93-26 IWC-93-25 IWC-93-24D IWC-93-24

Report Report Report Report Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Paper Report Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Authors Closure Paper Prepared Discussion Paper Report Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Paper

Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes No Yes No No Yes Yes Yes Yes Yes No Yes Yes

IWC-93-22D IWC-93-22 IWC-93-21 IWC-93-20D IWC-93-20 IWC-93-19 IWC-93-18 IWC-93-17 IWC-93-15D IWC-93-15 IWC-93-14D IWC-93-14 IWC-93-13D IWC-93-13AC IWC-93-13 IWC-93-12D IWC-93-12 IWC-93-11 IWC-93-10 IWC-93-09 IWC-93-08 IWC-93-07 IWC-93-06D IWC-93-06 IWC-93-05D IWC-93-05 IWC-93-04D IWC-93-04 IWC-93-03D IWC-93-03 IWC-93-02D IWC-93-02 IWC-93-01D IWC-93-01AC IWC-93-01 IWC-93~00 IWC-92-58 IWC-92-57D IWC-92-57AC IWC-92-57 IWC-92-56 IWC-92-55D IWC-92-55 IWC-92-54D IWC-92-54 IWC-92-53 IWC-92-52 IWC-92-51D IWC-92-51 IWC-92-50D IWC-92-50AC

Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Keynote Address Report Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Authors Closure

Yes Yes Yes No Yes Yes Yes Yes No No No No Yes Yes Yes Yes Yes Yes Yes No Yes Yes No No No No Yes Yes No No No Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes

IWC-92-50 IWC-92-49 IWC-92-48 IWC-92-47 IWC-92-46 IWC-92-45 IWC-92-44 IWC-92-43 IWC-92-42 IWC-92-41 IWC-92-40D IWC-92-40AC IWC-92-40 IWC-92-39 IWC-92-38 IWC-92-37 IWC-92-36 IWC-92-35 IWC-92-34 IWC-92-33 IWC-92-32 IWC-92-31 IWC-92-30D IWC-92-30 IWC-92-29D IWC-92-29 IWC-92-28 IWC-92-27 IWC-92-26D IWC-92-26 IWC-92-25D IWC-92-25AC IWC-92-25 IWC-92-24D IWC-92-24AC IWC-92-24 IWC-92-23D IWC-92-23AC IWC-92-23 IWC-92-22D IWC-92-22 IWC-92-21D IWC-92-21 IWC-92-20 IWC-92-19D IWC-92-19AC IWC-92-19 IWC-92-18D IWC-92-18 IWC-92-17D IWC-92-17

Paper Report Report Report Report Report Report Report Report Report Prepared Discussion Authors Closure Paper Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper

Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes No Yes Yes Yes No Yes Yes

IWC-92-16D IWC-92-16AC IWC-92-16 IWC-92-15D IWC-92-15 IWC-92-14D IWC-92-14AC IWC-92-14 IWC-92-13 IWC-92-12D IWC-92-12AC IWC-92-12 IWC-92-11D IWC-92-11 IWC-92-10 IWC-92-09D IWC-92-09 IWC-92-08 IWC-92-07D IWC-92-07AC IWC-92-07 IWC-92-06 IWC-92-05 IWC-92-04D IWC-92-04 IWC-92-03 IWC-92-02 IWC-92-01D IWC-92-01 IWC-92~00 IWC-91-57 IWC-91-56D IWC-91-56AC IWC-91-56 IWC-91-55D IWC-91-55 IWC-91-54D IWC-91-54 IWC-91-53D IWC-91-53 IWC-91-52 IWC-91-51 IWC-91-50 IWC-91-49D IWC-91-49 IWC-91-48D IWC-91-48 IWC-91-47 IWC-91-46 IWC-91-45 IWC-91-44

Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Report Report Prepared Discussion Paper Report Report Prepared Discussion Paper Keynote Address Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-91-43 IWC-91-42 IWC-91-41D IWC-91-41AC IWC-91-41 IWC-91-40 IWC-91-38 IWC-91-37 IWC-91-36 IWC-91-35D IWC-91-35 IWC-91-34D IWC-91-34AC IWC-91-34 IWC-91-33 IWC-91-32D IWC-91-32AC IWC-91-32 IWC-91-31D IWC-91-31 IWC-91-30D IWC-91-30AC IWC-91-30 IWC-91-28 IWC-91-27 IWC-91-26 IWC-91-25D IWC-91-25 IWC-91-24D IWC-91-24AC IWC-91-24 IWC-91-23 IWC-91-22D IWC-91-22 IWC-91-21D IWC-91-21AC IWC-91-21 IWC-91-20 IWC-91-19D IWC-91-19AC IWC-91-19 IWC-91-18D2 IWC-91-18D1 IWC-91-18 IWC-91-17D IWC-91-17AC IWC-91-17 IWC-91-16D IWC-91-16 IWC-91-15D IWC-91-15AC

Report Report Prepared Discussion Authors Closure Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Report Paper Paper Prepared Discussion Authors Closure Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure

Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-91-15 IWC-91-14 IWC-91-13D IWC-91-13 IWC-91-12 IWC-91-11D IWC-91-11 IWC-91-10 IWC-91-08 IWC-91-07 IWC-91-06 IWC-91-05 IWC-91-04 IWC-91-02D IWC-91-02 IWC-91-01D IWC-91-01 IWC-91~00 IWC-90-63 IWC-90-62 IWC-90-61 IWC-90-60 IWC-90-59 IWC-90-58D IWC-90-58 IWC-90-57 IWC-90-56 IWC-90-54 IWC-90-53 IWC-90-51D IWC-90-51 IWC-90-50D IWC-90-50AC IWC-90-50 IWC-90-49D IWC-90-49 IWC-90-48D IWC-90-48AC IWC-90-48 IWC-90-47D IWC-90-47 IWC-90-46D IWC-90-46 IWC-90-45D IWC-90-45AC IWC-90-45 IWC-90-44D IWC-90-44 IWC-90-43D IWC-90-43AC IWC-90-43

Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Keynote Address Report Report Report Report Report Report Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Prepared Discussion Paper Prepared Discussion Paper Report Paper Prepared Discussion Authors Closure Paper Report Paper Prepared Discussion Authors Closure Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes

IWC-90-42D IWC-90-42AC IWC-90-42 IWC-90-41D IWC-90-41AC IWC-90-41 IWC-90-39 IWC-90-38D IWC-90-38 IWC-90-37D IWC-90-37 IWC-90-36D IWC-90-36 IWC-90-35 IWC-90-34 IWC-90-32 IWC-90-31 IWC-90-30 IWC-90-29 IWC-90-28 IWC-90-27 IWC-90-26 IWC-90-25 IWC-90-24D IWC-90-24 IWC-90-23D IWC-90-23AC IWC-90-23 IWC-90-22D IWC-90-22AC IWC-90-22 IWC-90-21D IWC-90-21 IWC-90-19D IWC-90-19 IWC-90-18D IWC-90-18AC IWC-90-18 IWC-90-17D IWC-90-17AC IWC-90-17 IWC-90-16D IWC-90-16 IWC-90-15D IWC-90-15 IWC-90-14D IWC-90-14 IWC-90-13D IWC-90-13 IWC-90-12D IWC-90-12AC

Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-90-12 IWC-90-11D IWC-90-11AC IWC-90-11 IWC-90-10D IWC-90-10AC IWC-90-10 IWC-90-09D IWC-90-09AC IWC-90-09 IWC-90-08D IWC-90-08 IWC-90-07D IWC-90-07 IWC-90-06D IWC-90-06AC IWC-90-06 IWC-90-05D IWC-90-05AC IWC-90-05 IWC-90-04D IWC-90-04AC IWC-90-04 IWC-90-03D IWC-90-03AC IWC-90-03 IWC-90-02D IWC-90-02 IWC-90-01D IWC-90-01 IWC-90~00 IWC-89-61D IWC-89-61 IWC-89-60D IWC-89-60AC IWC-89-60 IWC-89-59D IWC-89-59 IWC-89-58D IWC-89-58AC IWC-89-58 IWC-89-57D IWC-89-57AC IWC-89-57 IWC-89-56D IWC-89-56 IWC-89-55D IWC-89-55AC IWC-89-55 IWC-89-52 IWC-89-51

Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Report Prepared Discussion Paper Prepared Discussion Paper Keynote Address Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes

IWC-89-50 IWC-89-49 IWC-89-48D IWC-89-48AC IWC-89-48 IWC-89-47D IWC-89-47AC IWC-89-47 IWC-89-46D IWC-89-46AC IWC-89-46 IWC-89-45 IWC-89-42D IWC-89-42 IWC-89-41D IWC-89-41 IWC-89-40D IWC-89-40 IWC-89-39D IWC-89-39AC IWC-89-39 IWC-89-38D IWC-89-38AC IWC-89-38 IWC-89-37D IWC-89-37AC IWC-89-37 IWC-89-36D IWC-89-36 IWC-89-35D IWC-89-35AC IWC-89-35 IWC-89-34D IWC-89-34 IWC-89-33D IWC-89-33 IWC-89-32D IWC-89-32 IWC-89-31D IWC-89-31AC IWC-89-31 IWC-89-30D IWC-89-30AC IWC-89-30 IWC-89-29D IWC-89-29AC IWC-89-29 IWC-89-28D IWC-89-28AC IWC-89-28 IWC-89-27D

Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Report Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-89-27AC IWC-89-27 IWC-89-26D IWC-89-26AC IWC-89-26 IWC-89-25 IWC-89-24D IWC-89-24AC IWC-89-24 IWC-89-23D IWC-89-23 IWC-89-22D IWC-89-22 IWC-89-21D IWC-89-21AC IWC-89-21 IWC-89-20D IWC-89-20AC IWC-89-20 IWC-89-19D IWC-89-19 IWC-89-18D IWC-89-18 IWC-89-17 IWC-89-16D IWC-89-16AC IWC-89-16 IWC-89-15D IWC-89-15 IWC-89-14D IWC-89-14 IWC-89-13D IWC-89-13 IWC-89-12D IWC-89-12 IWC-89-11D IWC-89-11AC IWC-89-11 IWC-89-10D IWC-89-10AC IWC-89-10 IWC-89-09D IWC-89-09 IWC-89-08D IWC-89-08 IWC-89-07D IWC-89-07 IWC-89-06D IWC-89-06AC IWC-89-06 IWC-89-05

Authors Closure Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes Yes Yes

IWC-89-04D IWC-89-04AC IWC-89-04 IWC-89-03D IWC-89-03AC IWC-89-03 IWC-89-02D IWC-89-02 IWC-89-01 IWC-89~00 IWC-88-65 IWC-88-63 IWC-88-62 IWC-88-61 IWC-88-60 IWC-88-59 IWC-88-58 IWC-88-57 IWC-88-55 IWC-88-54 IWC-88-53 IWC-88-52 IWC-88-50 IWC-88-49 IWC-88-48 IWC-88-47 IWC-88-45 IWC-88-44D IWC-88-44 IWC-88-43D IWC-88-43 IWC-88-42D IWC-88-42AC IWC-88-42 IWC-88-41D IWC-88-41AC IWC-88-41 IWC-88-40D IWC-88-40AC IWC-88-40 IWC-88-39D IWC-88-39AC IWC-88-39 IWC-88-38D IWC-88-38AC IWC-88-38 IWC-88-37D IWC-88-37 IWC-88-36D IWC-88-36AC IWC-88-36

Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Keynote Address Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper

Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-88-35D IWC-88-35AC IWC-88-35 IWC-88-34D IWC-88-34 IWC-88-33D IWC-88-33AC IWC-88-33 IWC-88-32D IWC-88-32AC IWC-88-32 IWC-88-31D IWC-88-31AC IWC-88-31 IWC-88-30D IWC-88-30AC IWC-88-30 IWC-88-28D IWC-88-28 IWC-88-27 IWC-88-26 IWC-88-25 IWC-88-24 IWC-88-21D IWC-88-21 IWC-88-20D IWC-88-20AC IWC-88-20 IWC-88-19D IWC-88-19AC IWC-88-19 IWC-88-18 IWC-88-17D IWC-88-17 IWC-88-16D IWC-88-16 IWC-88-15D IWC-88-15AC IWC-88-15 IWC-88-13D IWC-88-13 IWC-88-12D IWC-88-12AC IWC-88-12 IWC-88-11D IWC-88-11 IWC-88-10D IWC-88-10 IWC-88-09D IWC-88-09AC IWC-88-09

Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes

IWC-88-08D IWC-88-08AC IWC-88-08 IWC-88-07D IWC-88-07 IWC-88-06D IWC-88-06AC IWC-88-06 IWC-88-05D IWC-88-05AC IWC-88-05 IWC-88-04D IWC-88-04AC IWC-88-04 IWC-88-03D IWC-88-03AC IWC-88-03 IWC-88-02D IWC-88-02AC IWC-88-02 IWC-88-01D IWC-88-01AC IWC-88-01 IWC-87-58 IWC-87-57 IWC-87-56 IWC-87-55 IWC-87-54D IWC-87-54 IWC-87-53D IWC-87-53 IWC-87-52D IWC-87-52 IWC-87-51D IWC-87-51 IWC-87-50D IWC-87-50 IWC-87-49D IWC-87-49 IWC-87-48 IWC-87-47 IWC-87-46 IWC-87-45 IWC-87-44 IWC-87-43 IWC-87-42D IWC-87-42 IWC-87-41D IWC-87-41 IWC-87-40D IWC-87-40

Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Report Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes

IWC-87-39D IWC-87-39 IWC-87-38 IWC-87-37 IWC-87-36 IWC-87-35 IWC-87-34 IWC-87-33 IWC-87-32 IWC-87-31 IWC-87-30D IWC-87-30 IWC-87-29D IWC-87-29 IWC-87-28D IWC-87-28 IWC-87-27D IWC-87-27 IWC-87-26D IWC-87-26 IWC-87-25D IWC-87-25 IWC-87-24D IWC-87-24 IWC-87-23 IWC-87-22D IWC-87-22 IWC-87-21D IWC-87-21 IWC-87-20D IWC-87-20 IWC-87-19 IWC-87-18D IWC-87-18 IWC-87-17D IWC-87-17 IWC-87-16 IWC-87-15 IWC-87-14 IWC-87-13 IWC-87-12 IWC-87-11D IWC-87-11 IWC-87-10D IWC-87-10 IWC-87-09D IWC-87-09 IWC-87-08D IWC-87-08 IWC-87-07 IWC-87-06

Prepared Discussion Paper Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-87-05 IWC-87-04 IWC-87-03D IWC-87-03 IWC-87-02D IWC-87-02 IWC-87-01D IWC-87-01 IWC-86-57 IWC-86-56 IWC-86-55 IWC-86-54 IWC-86-53 IWC-86-52 IWC-86-51 IWC-86-50 IWC-86-49D IWC-86-49 IWC-86-48D IWC-86-48 IWC-86-47D IWC-86-47 IWC-86-45D IWC-86-45 IWC-86-43D IWC-86-43 IWC-86-42D IWC-86-42 IWC-86-41 IWC-86-40 IWC-86-39 IWC-86-38 IWC-86-37D IWC-86-37 IWC-86-36 IWC-86-35D IWC-86-35 IWC-86-34D IWC-86-34 IWC-86-33 IWC-86-32D IWC-86-32 IWC-86-31D IWC-86-31 IWC-86-30D IWC-86-30 IWC-86-29D IWC-86-29 IWC-86-28D IWC-86-28 IWC-86-27D

Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-86-27 IWC-86-26D IWC-86-26 IWC-86-25D IWC-86-25 IWC-86-24D IWC-86-24 IWC-86-23D IWC-86-23 IWC-86-22 IWC-86-21D IWC-86-21 IWC-86-20 IWC-86-19 IWC-86-18 IWC-86-17 IWC-86-16 IWC-86-15 IWC-86-14D IWC-86-14 IWC-86-13D IWC-86-13 IWC-86-12D IWC-86-12 IWC-86-11D IWC-86-11 IWC-86-10D IWC-86-10 IWC-86-09D IWC-86-09 IWC-86-08D IWC-86-08 IWC-86-07D IWC-86-07 IWC-86-06D IWC-86-06 IWC-86-05D IWC-86-05 IWC-86-04D IWC-86-04 IWC-86-03 IWC-86-01D IWC-86-01 IWC-85-58D IWC-85-58 IWC-85-57 IWC-85-56D IWC-85-56 IWC-85-55 IWC-85-54D IWC-85-54

Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-85-53D IWC-85-53 IWC-85-52D IWC-85-52 IWC-85-51D IWC-85-51 IWC-85-50D IWC-85-50 IWC-85-49 IWC-85-48 IWC-85-47 IWC-85-46 IWC-85-45D IWC-85-45 IWC-85-43D IWC-85-43 IWC-85-42D IWC-85-42 IWC-85-41 IWC-85-40 IWC-85-39 IWC-85-38 IWC-85-37 IWC-85-36D IWC-85-36 IWC-85-35 IWC-85-34D IWC-85-34 IWC-85-33D IWC-85-33 IWC-85-32D IWC-85-32 IWC-85-31D IWC-85-31 IWC-85-30D IWC-85-30 IWC-85-29 IWC-85-28 IWC-85-27 IWC-85-26 IWC-85-25 IWC-85-24 IWC-85-23 IWC-85-22 IWC-85-20 IWC-85-19D IWC-85-19 IWC-85-18D IWC-85-18 IWC-85-17D IWC-85-17

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-85-16D IWC-85-16 IWC-85-15D IWC-85-15 IWC-85-13D IWC-85-13 IWC-85-12D IWC-85-12 IWC-85-11D IWC-85-11 IWC-85-10D IWC-85-10 IWC-85-08D IWC-85-08 IWC-85-07D IWC-85-07 IWC-85-06D IWC-85-06 IWC-85-05D IWC-85-05 IWC-85-04 IWC-85-03 IWC-85-02D IWC-85-02 IWC-85-01D IWC-85-01 IWC-84-96D IWC-84-96 IWC-84-93D IWC-84-93 IWC-84-91 IWC-84-86D IWC-84-86 IWC-84-81D IWC-84-81 IWC-84-80D IWC-84-80AC IWC-84-80 IWC-84-76D IWC-84-76 IWC-84-73D IWC-84-73 IWC-84-71 IWC-84-70 IWC-84-60D IWC-84-60 IWC-84-56 IWC-84-54 IWC-84-53D IWC-84-53 IWC-84-50

Report Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Prepared Discussion Paper Report Report Prepared Discussion Paper Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-84-49D IWC-84-49 IWC-84-44 IWC-84-43D IWC-84-43 IWC-84-42 IWC-84-39D IWC-84-39 IWC-84-35D IWC-84-35 IWC-84-33 IWC-84-26D IWC-84-26 IWC-84-23 IWC-84-21 IWC-84-20 IWC-84-18D IWC-84-18 IWC-84-15D IWC-84-15 IWC-84-13 IWC-84-116 IWC-84-115 IWC-84-114 IWC-84-113 IWC-84-112D IWC-84-112 IWC-84-109D IWC-84-109 IWC-84-108D IWC-84-108 IWC-84-105D IWC-84-105 IWC-84-104 IWC-84-103D IWC-84-103 IWC-84-10 IWC-84-09 IWC-84-06D IWC-84-06 IWC-84-02D IWC-84-02 IWC-83-49 IWC-83-48 IWC-83-47 IWC-83-46 IWC-83-45 IWC-83-44 IWC-83-43 IWC-83-42 IWC-83-41

Prepared Discussion Paper Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Report Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Report Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-83-40 IWC-83-39 IWC-83-38 IWC-83-37 IWC-83-36 IWC-83-35 IWC-83-34 IWC-83-33 IWC-83-32 IWC-83-31 IWC-83-30D IWC-83-30 IWC-83-29D IWC-83-29 IWC-83-28D IWC-83-28 IWC-83-27D IWC-83-27 IWC-83-26D IWC-83-26 IWC-83-25D IWC-83-25 IWC-83-24D IWC-83-24 IWC-83-23D IWC-83-23 IWC-83-22D IWC-83-22 IWC-83-21D IWC-83-21 IWC-83-20D IWC-83-20 IWC-83-19D IWC-83-19 IWC-83-18D IWC-83-18 IWC-83-17D IWC-83-17 IWC-83-16D IWC-83-16 IWC-83-15D IWC-83-15 IWC-83-14D IWC-83-14 IWC-83-13 IWC-83-12D IWC-83-12 IWC-83-11D IWC-83-11 IWC-83-10D IWC-83-10

Report Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Abstract

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-83-09D IWC-83-09 IWC-83-08D IWC-83-08 IWC-83-07D IWC-83-07 IWC-83-06D IWC-83-06 IWC-83-05D IWC-83-05 IWC-83-04D IWC-83-04 IWC-83-03D IWC-83-03 IWC-83-02 IWC-83-01D IWC-83-01 IWC-82-52 IWC-82-51 IWC-82-50 IWC-82-49 IWC-82-48 IWC-82-47 IWC-82-46 IWC-82-45 IWC-82-44 IWC-82-42 IWC-82-41 IWC-82-39 IWC-82-38D IWC-82-38 IWC-82-37D IWC-82-37 IWC-82-36D IWC-82-36AC IWC-82-36 IWC-82-35D IWC-82-35 IWC-82-34PD7 IWC-82-34PD6 IWC-82-34PD5 IWC-82-34PD4 IWC-82-34PD3 IWC-82-34PD2 IWC-82-34PD1 IWC-82-33D IWC-82-33 IWC-82-32D IWC-82-32 IWC-82-31D IWC-82-31

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Paper Report Prepared Discussion Paper Poster Poster Poster Poster Poster Poster Poster Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-82-30D IWC-82-30 IWC-82-29 IWC-82-28 IWC-82-27 IWC-82-26 IWC-82-25 IWC-82-24 IWC-82-23 IWC-82-22D IWC-82-22 IWC-82-21D IWC-82-21 IWC-82-20D IWC-82-20 IWC-82-19D IWC-82-19 IWC-82-18D IWC-82-18 IWC-82-17D IWC-82-17 IWC-82-16D IWC-82-16 IWC-82-15D IWC-82-15 IWC-82-14 IWC-82-13 IWC-82-12 IWC-82-11 IWC-82-10D IWC-82-10 IWC-82-09D IWC-82-09 IWC-82-08D IWC-82-08AC IWC-82-08 IWC-82-07D IWC-82-07 IWC-82-06D IWC-82-06 IWC-82-05 IWC-82-04D IWC-82-04 IWC-82-03D IWC-82-03 IWC-82-02D IWC-82-02 IWC-82-01D IWC-82-01 IWC-81-47 IWC-81-46

Prepared Discussion Paper Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-81-45 IWC-81-44 IWC-81-43 IWC-81-42 IWC-81-41 IWC-81-40D IWC-81-40AC IWC-81-40 IWC-81-39D IWC-81-39 IWC-81-38D IWC-81-38 IWC-81-35 IWC-81-34 IWC-81-33 IWC-81-32 IWC-81-31 IWC-81-30 IWC-81-28D IWC-81-28 IWC-81-27D IWC-81-27 IWC-81-26D IWC-81-26 IWC-81-25D IWC-81-25 IWC-81-24D IWC-81-24 IWC-81-23D IWC-81-23 IWC-81-22D IWC-81-22 IWC-81-21D IWC-81-21 IWC-81-20D IWC-81-20 IWC-81-19D IWC-81-19 IWC-81-18D IWC-81-18 IWC-81-17 IWC-81-16D IWC-81-16AC IWC-81-16 IWC-81-15D IWC-81-15 IWC-81-14D IWC-81-14 IWC-81-13D IWC-81-13 IWC-81-12D

Report Report Report Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Panel Discussion Transcript Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-81-12 IWC-81-11D IWC-81-11 IWC-81-10D IWC-81-10 IWC-81-09D IWC-81-09 IWC-81-08D IWC-81-08 IWC-81-07D IWC-81-07 IWC-81-06D IWC-81-06 IWC-81-05 IWC-81-04D IWC-81-04 IWC-81-03D IWC-81-03 IWC-81-02D IWC-81-02 IWC-81-01D IWC-81-01 IWC-80-52D IWC-80-52AC IWC-80-52 IWC-80-51 IWC-80-50 IWC-80-49 IWC-80-48 IWC-80-47 IWC-80-45 IWC-80-44 IWC-80-43 IWC-80-42 IWC-80-41 IWC-80-39 IWC-80-38 IWC-80-37 IWC-80-36 IWC-80-35 IWC-80-34 IWC-80-33 IWC-80-32 IWC-80-31 IWC-80-30D IWC-80-30 IWC-80-29D IWC-80-29 IWC-80-28D IWC-80-28AC IWC-80-28

Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper

Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-80-27D IWC-80-27AC IWC-80-27 IWC-80-26 IWC-80-25 IWC-80-24 IWC-80-22D IWC-80-22 IWC-80-21 IWC-80-20D IWC-80-20 IWC-80-19D IWC-80-19 IWC-80-18D IWC-80-18 IWC-80-17D IWC-80-17 IWC-80-16D IWC-80-16 IWC-80-15D IWC-80-15AC IWC-80-15 IWC-80-14D IWC-80-14 IWC-80-13D IWC-80-13 IWC-80-12D IWC-80-12 IWC-80-11D IWC-80-11 IWC-80-10D IWC-80-10 IWC-80-09D IWC-80-09 IWC-80-08D IWC-80-08 IWC-80-07D IWC-80-07 IWC-80-06D IWC-80-06 IWC-80-05D IWC-80-05 IWC-80-04 IWC-80-02D IWC-80-02 IWC-80-01D IWC-80-01 IWC-79~50 IWC-79~49 IWC-79~48 IWC-79~47

Prepared Discussion Authors Closure Paper Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Prepared Discussion Paper Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-79~46 IWC-79~45 IWC-79~44 IWC-79~43 IWC-79~42 IWC-79~41 IWC-79~40 IWC-79~39 IWC-79~38 IWC-79~37PD IWC-79~36PD IWC-79~35PD IWC-79~34PD IWC-79~33PD IWC-79~32PD IWC-79~31PD IWC-79~30PD IWC-79~29PD IWC-79~28 IWC-79~27D IWC-79~27 IWC-79~26D IWC-79~26AC IWC-79~26 IWC-79~25 IWC-79~24 IWC-79~23 IWC-79~22 IWC-79~21 IWC-79~20D IWC-79~20 IWC-79~19 IWC-79~18D IWC-79~18 IWC-79~17D IWC-79~17 IWC-79~16D IWC-79~16 IWC-79~15D IWC-79~15 IWC-79~14D IWC-79~14 IWC-79~13D IWC-79~13 IWC-79~12D IWC-79~12 IWC-79~11D3 IWC-79~11D2 IWC-79~11D1 IWC-79~11AC IWC-79~11

Report Report Report Report Report Report Report Report Report Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Panel Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Report Report Report Report Report Prepared Discussion Paper Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Prepared Discussion Prepared Discussion Paper Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-79~10D IWC-79~10 IWC-79~09D IWC-79~09 IWC-79~08D IWC-79~08 IWC-79~07D IWC-79~07 IWC-79~06D IWC-79~06 IWC-79~05 IWC-79~04 IWC-79~03 IWC-79~02D IWC-79~02 IWC-79~01D2 IWC-79~01D1 IWC-79~01AC IWC-79~01 IWC-78~42 IWC-78~41 IWC-78~40 IWC-78~39 IWC-78~38 IWC-78~37 IWC-78~36 IWC-78~35 IWC-78~34 IWC-78~33 IWC-78~32 IWC-78~31 IWC-78~30D IWC-78~30AC IWC-78~30 IWC-78~29D IWC-78~29 IWC-78~28D IWC-78~28 IWC-78~27D IWC-78~27 IWC-78~26D IWC-78~26 IWC-78~25D IWC-78~25 IWC-78~24D IWC-78~24AC IWC-78~24 IWC-78~23D IWC-78~23 IWC-78~22D IWC-78~22

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Prepared Discussion Paper Prepared Discussion Prepared Discussion Authors Closure Paper Report Report Report Report Report Report Report Report Report Report Report Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-78~21D IWC-78~21 IWC-78~20D IWC-78~20 IWC-78~19D IWC-78~19 IWC-78~18D IWC-78~18 IWC-78~17D IWC-78~17 IWC-78~16D IWC-78~16 IWC-78~15D IWC-78~15AC IWC-78~15 IWC-78~14D IWC-78~14 IWC-78~13 IWC-78~12 IWC-78~11 IWC-78~10 IWC-78~09D IWC-78~09 IWC-78~08D IWC-78~08 IWC-78~07D IWC-78~07 IWC-78~06D IWC-78~06 IWC-78~05D IWC-78~05 IWC-78~04D IWC-78~04 IWC-78~03D IWC-78~03AC IWC-78~03 IWC-78~02D IWC-78~02 IWC-78~01D IWC-78~01 IWC-77~34 IWC-77~33 IWC-77~32 IWC-77~31 IWC-77~30 IWC-77~29 IWC-77~28D IWC-77~28 IWC-77~27D IWC-77~27 IWC-77~26D

Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-77~26 IWC-77~25D IWC-77~25 IWC-77~24D IWC-77~24AC IWC-77~24 IWC-77~23D IWC-77~23 IWC-77~22D2 IWC-77~22D1 IWC-77~22AC IWC-77~22 IWC-77~21D2 IWC-77~21D1 IWC-77~21 IWC-77~20PDT IWC-77~19D IWC-77~19 IWC-77~18D IWC-77~18 IWC-77~17D IWC-77~17 IWC-77~16D IWC-77~16AC IWC-77~16 IWC-77~15D IWC-77~15AC IWC-77~15 IWC-77~14D IWC-77~14 IWC-77~13D IWC-77~13AC IWC-77~13 IWC-77~12D IWC-77~12 IWC-77~11D IWC-77~11 IWC-77~10D IWC-77~10 IWC-77~09 IWC-77~08 IWC-77~07 IWC-77~06 IWC-77~05D IWC-77~05 IWC-77~04D IWC-77~04 IWC-77~03D IWC-77~03 IWC-77~02D IWC-77~02AC

Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Prepared Discussion Authors Closure Paper Prepared Discussion Prepared Discussion Paper Panel Discussion Transcript Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-77~02 IWC-77~01D IWC-77~01 IWC-76~23 IWC-76~22 IWC-76~21 IWC-76~20 IWC-76~19 IWC-76~18 IWC-76~17D IWC-76~17 IWC-76~16D IWC-76~16AC IWC-76~16 IWC-76~15D IWC-76~15 IWC-76~14D IWC-76~14 IWC-76~13D IWC-76~13 IWC-76~12D IWC-76~12 IWC-76~11D IWC-76~11 IWC-76~10D IWC-76~10 IWC-76~09D IWC-76~09 IWC-76~08 IWC-76~07D IWC-76~07AC IWC-76~07 IWC-76~06D IWC-76~06AC IWC-76~06 IWC-76~05D IWC-76~05 IWC-76~04D IWC-76~04 IWC-76~03D IWC-76~03 IWC-76~02D IWC-76~02 IWC-76~01D IWC-76~01 IWC-75~22 IWC-75~21 IWC-75~20 IWC-75~19 IWC-75~18 IWC-75~17

Paper Prepared Discussion Paper Report Report Report Report Report Report Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Prepared Discussion Authors Closure Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Report Report Report Report Report Report

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-75~16D IWC-75~16AC IWC-75~16 IWC-75~15 IWC-75~14D IWC-75~14 IWC-75~13D2 IWC-75~13D1 IWC-75~13 IWC-75~12D IWC-75~12 IWC-75~11D IWC-75~11 IWC-75~10D IWC-75~10AC IWC-75~10 IWC-75~09D IWC-75~09 IWC-75~08D IWC-75~08 IWC-75~07D IWC-75~07 IWC-75~06D IWC-75~06AC IWC-75~06 IWC-75~05D IWC-75~05 IWC-75~04D IWC-75~04AC IWC-75~04 IWC-75~03D IWC-75~03 IWC-75~02D IWC-75~02 IWC-75~01D IWC-75~01 IWC-74~9 IWC-74~8 IWC-74~7 IWC-74~6 IWC-74~5 IWC-74~4 IWC-74~3 IWC-74~21 IWC-74~20 IWC-74~2 IWC-74~19 IWC-74~18 IWC-74~17 IWC-74~16 IWC-74~15

Prepared Discussion Authors Closure Paper Report Prepared Discussion Paper Prepared Discussion Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Authors Closure Paper Prepared Discussion Report Prepared Discussion Authors Closure Paper Prepared Discussion Paper Prepared Discussion Paper Prepared Discussion Paper

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-74~14 IWC-74~13 IWC-74~12 IWC-74~11 IWC-74~10 IWC-74~1 IWC-73~9 IWC-73~8 IWC-73~7 IWC-73~6 IWC-73~5 IWC-73~4 IWC-73~3 IWC-73~24 IWC-73~23 IWC-73~22 IWC-73~20 IWC-73~2 IWC-73~19 IWC-73~18 IWC-73~17 IWC-73~16 IWC-73~15 IWC-73~14 IWC-73~13 IWC-73~12 IWC-73~11 IWC-73~10 IWC-73~1 IWC-72~9 IWC-72~8 IWC-72~7 IWC-72~6 IWC-72~5 IWC-72~4 IWC-72~3 IWC-72~23 IWC-72~22 IWC-72~21 IWC-72~21 IWC-72~20 IWC-72~2 IWC-72~19 IWC-72~18 IWC-72~17 IWC-72~16 IWC-72~15 IWC-72~14 IWC-72~13 IWC-72~12 IWC-72~11

Yes Yes Yes Yes yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-72~10 IWC-72~1 IWC-71~9 IWC-71~8 IWC-71~7 IWC-71~6 IWC-71~5 IWC-71~4 IWC-71~3 IWC-71~26 IWC-71~25 IWC-71~24 IWC-71~23 IWC-71~22 IWC-71~21 IWC-71~20 IWC-71~2 IWC-71~19 IWC-71~19 IWC-71~18 IWC-71~17 IWC-71~16 IWC-71~15 IWC-71~14 IWC-71~13 IWC-71~12 IWC-71~11 IWC-71~10 IWC-71~1 IWC-70~9 IWC-70~8 IWC-70~7 IWC-70~6 IWC-70~5 IWC-70~4 IWC-70~3 IWC-70~28 IWC-70~27 IWC-70~26 IWC-70~25 IWC-70~24 IWC-70~23 IWC-70~22 IWC-70~21 IWC-70~20 IWC-70~2 IWC-70~18 IWC-70~17 IWC-70~16 IWC-70~15 IWC-70~14

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-70~13 IWC-70~12 IWC-70~11 IWC-70~10 IWC-70~1 IWC-69~9 IWC-69~25 IWC-69~24 IWC-69~23 IWC-69~22 IWC-69~21 IWC-69~20 IWC-69~19 IWC-69~18 IWC-69~17 IWC-69~16 IWC-69~15 IWC-69~14 IWC-69~13 IWC-69~12 IWC-69~11 IWC-69~10 IWC-68~9 IWC-68~8 IWC-68~8 IWC-68~7 IWC-68~7 IWC-68~6 IWC-68~6 IWC-68~5 IWC-68~5 IWC-68~4 IWC-68~4 IWC-68~3 IWC-68~3 IWC-68~22 IWC-68~21 IWC-68~20 IWC-68~2 IWC-68~2 IWC-68~19 IWC-68~18 IWC-68~17 IWC-68~16 IWC-68~15 IWC-68~14 IWC-68~13 IWC-68~12 IWC-68~11 IWC-68~10 IWC-68~1

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-68~1 IWC-67~9 IWC-67~8 IWC-67~7 IWC-67~6 IWC-67~5 IWC-67~4 IWC-67~3 IWC-67~22 IWC-67~21 IWC-67~20 IWC-67~2 IWC-67~19 IWC-67~18 IWC-67~17 IWC-67~16 IWC-67~15 IWC-67~14 IWC-67~13 IWC-67~12 IWC-67~11 IWC-67~10 IWC-66~9 IWC-66~8 IWC-66~7 IWC-66~6 IWC-66~5 IWC-66~4 IWC-66~3 IWC-66~21 IWC-66~20 IWC-66~2 IWC-66~19 IWC-66~18 IWC-66~17 IWC-66~16 IWC-66~15 IWC-66~14 IWC-66~13 IWC-66~12 IWC-66~11 IWC-66~10 IWC-66~1 IWC-66~1 IWC-65~9 IWC-65~8 IWC-65~7 IWC-65~6 IWC-65~5 IWC-65~4 IWC-65~3

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-65~23 IWC-65~22 IWC-65~21 IWC-65~20 IWC-65~2 IWC-65~19 IWC-65~18 IWC-65~17 IWC-65~16 IWC-65~15 IWC-65~14 IWC-65~13 IWC-65~12 IWC-65~11 IWC-65~10 IWC-65~1 IWC-64~9 IWC-64~8 IWC-64~7 IWC-64~6 IWC-64~5 IWC-64~4 IWC-64~3 IWC-64~22 IWC-64~21 IWC-64~20 IWC-64~2 IWC-64~19 IWC-64~18 IWC-64~17 IWC-64~16 IWC-64~15 IWC-64~14 IWC-64~13 IWC-64~12 IWC-64~11 IWC-64~10 IWC-64~1 IWC-63~9 IWC-63~8 IWC-63~7 IWC-63~6 IWC-63~5 IWC-63~4 IWC-63~3 IWC-63~2 IWC-63~14 IWC-63~13 IWC-63~12 IWC-63~11 IWC-63~10

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-63~1 IWC-62~9 IWC-62~8 IWC-62~7 IWC-62~6 IWC-62~5 IWC-62~4 IWC-62~3 IWC-62~2 IWC-62~14 IWC-62~12 IWC-62~11 IWC-62~10 IWC-62~1 IWC-61~9 IWC-61~8 IWC-61~7 IWC-61~6 IWC-61~5 IWC-61~4 IWC-61~3 IWC-61~2 IWC-61~14 IWC-61~13 IWC-61~12 IWC-61~11 IWC-61~10 IWC-61~1 IWC-60~9 IWC-60~8 IWC-60~7 IWC-60~6 IWC-60~5 IWC-60~4 IWC-60~3 IWC-60~2 IWC-60~18 IWC-60~17 IWC-60~16 IWC-60~15 IWC-60~14 IWC-60~13 IWC-60~12 IWC-60~11 IWC-60~10 IWC-60~1 IWC-59~9 IWC-59~8 IWC-59~7 IWC-59~6 IWC-59~5

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-59~4 IWC-59~3 IWC-59~2 IWC-59~19 IWC-59~18 IWC-59~17 IWC-59~16 IWC-59~15 IWC-59~14 IWC-59~13 IWC-59~12 IWC-59~11 IWC-59~10 IWC-59~1 IWC-58~9 IWC-58~8 IWC-58~7 IWC-58~6 IWC-58~5 IWC-58~4 IWC-58~3 IWC-58~2 IWC-58~14 IWC-58~13 IWC-58~12 IWC-58~11 IWC-58~10 IWC-58~1 IWC-57~9 IWC-57~8 IWC-57~7 IWC-57~6 IWC-57~5 IWC-57~4 IWC-57~3 IWC-57~25 IWC-57~24 IWC-57~23 IWC-57~22 IWC-57~21 IWC-57~20 IWC-57~2 IWC-57~19 IWC-57~18 IWC-57~17 IWC-57~16 IWC-57~15 IWC-57~14 IWC-57~13 IWC-57~12 IWC-57~11

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-57~10 IWC-57~1 IWC-56~9 IWC-56~8 IWC-56~7 IWC-56~6 IWC-56~5 IWC-56~4 IWC-56~3 IWC-56~2 IWC-56~12 IWC-56~11 IWC-56~10 IWC-56~1 IWC-55~9 IWC-55~8 IWC-55~7 IWC-55~6 IWC-55~5 IWC-55~4 IWC-55~3 IWC-55~2 IWC-55~11 IWC-55~10 IWC-55~1 IWC-54~9 IWC-54~8 IWC-54~7 IWC-54~6 IWC-54~5 IWC-54~4 IWC-54~3 IWC-54~22 IWC-54~21 IWC-54~20 IWC-54~2 IWC-54~19 IWC-54~18 IWC-54~17 IWC-54~16 IWC-54~15 IWC-54~14 IWC-54~13 IWC-54~12 IWC-54~11 IWC-54~10 IWC-54~1 IWC-53~9 IWC-53~8 IWC-53~7 IWC-53~6

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-53~5 IWC-53~4 IWC-53~3 IWC-53~2 IWC-53~17 IWC-53~16 IWC-53~15 IWC-53~14 IWC-53~13 IWC-53~12 IWC-53~11 IWC-53~10 IWC-53~1 IWC-52~9 IWC-52~8 IWC-52~7 IWC-52~6 IWC-52~5 IWC-52~4 IWC-52~3 IWC-52~2 IWC-52~13 IWC-52~12 IWC-52~11 IWC-52~10 IWC-52~1 IWC-51~9 IWC-51~8 IWC-51~7 IWC-51~6 IWC-51~5 IWC-51~4 IWC-51~3 IWC-51~20 IWC-51~2 IWC-51~19 IWC-51~18 IWC-51~17 IWC-51~16 IWC-51~15 IWC-51~14 IWC-51~13 IWC-51~12 IWC-51~11 IWC-51~10 IWC-51~1 IWC-50~9 IWC-50~8 IWC-50~7 IWC-50~6 IWC-50~5

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-50~4 IWC-50~3 IWC-50~2 IWC-50~11 IWC-50~10 IWC-50~1 IWC-49~9 IWC-49~8 IWC-49~7 IWC-49~6 IWC-49~5 IWC-49~4 IWC-49~3 IWC-49~2 IWC-49~13 IWC-49~12 IWC-49~11 IWC-49~10 IWC-49~1 IWC-48~9 IWC-48~8 IWC-48~7 IWC-48~6 IWC-48~5 IWC-48~4 IWC-48~3 IWC-48~2 IWC-48~12 IWC-48~11 IWC-48~10 IWC-48~1 IWC-47~9 IWC-47~8 IWC-47~7 IWC-47~6 IWC-47~5 IWC-47~4 IWC-47~3 IWC-47~2 IWC-47~16~D3 IWC-47~16~D2 IWC-47~16~D1 IWC-47~16 IWC-47~15~D3 IWC-47~15~D2 IWC-47~15~D1 IWC-47~15 IWC-47~14 IWC-47~13~D4 IWC-47~13~D3 IWC-47~13~D2

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No No No No No No No No No No No No No No No No No No No

IWC-47~13~D1 IWC-47~13 IWC-47~12~D5 IWC-47~12~D4 IWC-47~12~D3 IWC-47~12~D2 IWC-47~12~D1 IWC-47~12 IWC-47~11 IWC-47~10 IWC-47~1 IWC-47~08~D4 IWC-47~08~D3 IWC-47~08~D2 IWC-47~08~D1 IWC-47~07~D5 IWC-47~07~D4 IWC-47~07~D3 IWC-47~07~D2 IWC-47~07~D1 IWC-47~06~D4 IWC-47~06~D3 IWC-47~06~D2 IWC-47~06~D1 IWC-47~05~D4 IWC-47~05~D3 IWC-47~05~D2 IWC-47~05~D1 IWC-47~04~D3 IWC-47~04~D2 IWC-47~04~D1 IWC-47~03~D4 IWC-47~03~D3 IWC-47~03~D2 IWC-47~03~D1 IWC-47~02~D4 IWC-47~02~D3 IWC-47~02~D2 IWC-47~02~D1 IWC-47~01~D5 IWC-47~01~D4 IWC-47~01~D3 IWC-47~01~D2 IWC-47~01~D1 IWC-46~9 IWC-46~8 IWC-46~7 IWC-46~6 IWC-46~5 IWC-46~4 IWC-46~3

No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No Yes Yes Yes Yes Yes Yes Yes

IWC-46~2 IWC-46~13 IWC-46~12 IWC-46~11 IWC-46~10 IWC-46~1 IWC-45~9 IWC-45~8 IWC-45~7 IWC-45~6 IWC-45~5 IWC-45~4 IWC-45~3 IWC-45~2 IWC-45~11 IWC-45~10 IWC-45~1 IWC-44~9 IWC-44~8 IWC-44~7 IWC-44~6 IWC-44~5 IWC-44~4 IWC-44~3 IWC-44~2 IWC-44~11 IWC-44~10 IWC-44~1 IWC-43~9 IWC-43~8 IWC-43~7 IWC-43~6 IWC-43~5 IWC-43~4 IWC-43~3 IWC-43~2 IWC-43~10 IWC-43~1 IWC-42~9 IWC-42~8 IWC-42~7 IWC-42~6 IWC-42~5 IWC-42~4 IWC-42~3 IWC-42~2 IWC-42~1 IWC-41~9 IWC-41~8 IWC-41~7 IWC-41~6

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

IWC-41~5 IWC-41~4 IWC-41~3 IWC-41~2 IWC-41~14 IWC-41~13 IWC-41~12 IWC-41~11 IWC-41~10 IWC-41~1 IWC-40~9 IWC-40~8 IWC-40~7 IWC-40~6 IWC-40~5 IWC-40~4 IWC-40~3 IWC-40~2 IWC-40~15 IWC-40~14 IWC-40~13 IWC-40~12 IWC-40~11 IWC-40~10 IWC-40~1 IWC-40~04~D2 IWC-40~04~D1 IWC-40~03~D1

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No No No No No No No No No No No No No No No No No

Title

Author & Affiliation

Discusser & Affiliation

Outsourced WaterMichael Reyes, Veolia Water Solutions & Technologies, Vandalia, OH USA The Influence of David Moed, Delft University of TechnolDebbie Bloom, Nalco Company, Naperville, IL Monitoring Flow AKenneth Kuruc, Hach Company, LoveDaniel Sampson, Worley Parsons, Vallejo, CA The Impact of St Kenneth Chen, Fluor Enterprises, Inc., Josh Prusakiewicz, HDR Inc., Pittsburgh, PA Factors to Consid John Fair, Fair Canada Engineering LMike D’Ippolito, Devon Canada, Calgary, AB, Canada Enhancements in Greg E Mandigo, AquaChem ICD Aquatech Kevin Drake, P. Eng., Drake Consulting, Calgary, AB Canada Bench-Scale Evalu David Pernitsky, Suncor, Calgary, A Martin R. Godfrey Ph.D., Nalco, Calagary, AB Canada Key Parameters, D Andreas Ronnfors, chúng tôi M.E. Energy Ramesh Sharma, ConocoPhillips, Houston, TX Case Study of a FiReinhold Brenner, Von Roll BHU EnviroKatariina Majamaa, Dow Chemical China, Shanghai, China Pretreatment Syst Gregory Osen, AVANTech, Inc., ColuBrian Powers, HDR Engineering, Inc., Charlotte, NC Advanced OxidatioGlen Sundstrom, Siemens Industry IncRandy Cooper, Ph.D., Neotech Aqua Solutions, San Diego, CA Ultrapure Water ( Vyacheslav Libman, Ph.D., Air Liquid John Brittan, Siemens Water Technologies, LLC, Colorado Springs, C Real-time PerformDarcy Dauterive, Ashland Water TechnCraig Myers, Nalco, An Ecolab Co., Naperville, IL A Monitor for MeasCharles J. Reedy, Ph.D., Nalco, An EcLee Hollimon, Ashland Water Technologies, Springville, NY The Impact of pH R a obert J. Ferguson, French Creek Sof Prasad Kalakodimi, Chemtreat, Glen Allen, VA Novel Dual Active Linna Wang, GE Water and Process Te Sam Renfrow, Chemtreat, Columbus, MI Helping Industrie E. H. Kelle Zeiher, AkzoNobel, Chat James Beninati, HDR Engineering, Meeting the NeweNancy Sherwood, MAR Systems Inc, Rudy Labban, PE, Inflico Degremont, Ashland, VA Upcoming EffluentKatie A. Bland, Burns & McDonnell, Diane Martini, Sargent & Lundy, LLC, Chicago, IL New Heavy MetalsMelanie Solmos, Nalco, an Ecolab Com Michael Pudvay, GEA Process Engineering, Forensic Analysis Glenn Weagle, Ph.D., Nalco Champion, Sherwood Park, AB Canada Electrochemical Tzu-Yu (Liberty) Chen, Nalco Company, Naperville, IL On-line Silica An Ramesh Sharma, ConocoPhillips, Houston, TX Increasing OTSG Annie (Qian) Sun, Suncor Energy Inc., Calgary, Alberta Canada Treatment and Rec Brian Altland, P.E., Calgon Carbon CorWilliam Carlin, Dow, Spring House, PA Drinking Water NiRich Dennis, Severn Trent Services, IH.G. Sanjay, Ph.D, IONZ BlueWater Solutions, Inc., Royal Palm Bea Operation of the CFrancis DeSilva, ResinTech Inc, BerliDonald Thompson, CDM Smith, Jacksonville, FL Technical Evaluati William Schwartz, Envirogen Techno Ajit Dighe, Thermax Inc, Troubleshooting Kevin Boudreaux, Nalco Comapany, Daniel Sampson, Worley Parsons, Sacramento, CA The Thermal ZLD Patricia Scroggin, P.E., Burns & McD James Beninati, HDR Inc., Pittsburgh, PA Operating a ZLD, Vincent Como, Black & Veatch, OverChristian Haussmann, Water Systems Specialists, Inc., Seattle, WA ZLD Systems: VariSara Titus, P.E., Bechtel Power CorpoCraig Van Dyke, GE Water and Process Technologies, Brookfield, IL FGD Scrubber Was Kirk Ellison, Southern Company ServiKatie Bland, P.E., Burns & McDonnell, Kansas City, MO Meeting Part-per-Thomas Higgins, Ph.D., P.E., CH2M HIL Paul Chu, Electric Power Research Institue, Palo Alto, CA Purification of Fl Behrang Pakzadeh, Ph.D., Southern Res William Kennedy, P.E., Duke Energy, Belmont, NC How Operation of Bryan Hansen, P.E., Burns & McDonne Joseph Potts, Duke Energy, Belmont, NC Frac Water Treatme John Williams, Xchem-Terra Services,Doug McIlwaine, ChemTreat, Plasma Treatment Y.I. o Cho, Ph.D., Drexel University, PhJosh Prusakiewicz, HDR Inc., Pittsburgh, PA Pretreatment TargJames Silva, Ph.D., GE Global ReseaJerry Penland, Chester Engineers, Moon Twp., PA Recovering ValuabRussell Huffmyer, Nuverra Environmenta John Schubert, HDR, Inc., Removal of SeleniMissy Hayes, MAR Systems, Solon, Ivan Cooper, Civil & Environmental Consultants, Inc., Charolette, NC Ballasted Clarifica William Willersdorf, Veolia Water Sol Hollie Scott, Degremont, Salt Lake, UT Novel Process for Antonio Lau, Infilco Degremont, Inc., Frank Reightler, 4G Environmental Corp., Jim Thorpe, PA Removal of HardnDavid Kratochvil, Ph.D., BioteQ Envi Greg Behrens, P.E., URS Corp., Austin, TX Designing RO SysRobert Kimball, P.E., CDM Smith, Hel Brad Biagini, Veolia Water Solutions & Technologies, Moon Township Performance dataVenkat Jagannathan, Qua Group LLCSteven R. Gagnon, AVANTech Inc., Columbia, SC Novel PTFE Membra Vita Martez, Southern Alberta Institu Michael Snodgrass, SpiraSep, Goleta, CA Optimization of anStanley Karrs, Siemens Industry Inc. H.G. Sanjay, Ph.D., IONZ BlueWater Solutions, Inc., Royal Palm Bea The Importance ofBrad Buecker, Kiewit Power EngineersTed Beardwood, Ashland Chemical Company, Ajax, ON Canada Distinguishing Be Kevin Shields, Structural Integrity AssoMel J. Esmacher, P.E., GE Power & Water Process Technologies, Th

Hydrogen Damage Andrew Howell, Ph.D., Xcel Energy, Robert D. Bartholomew, Sheppard T. Powell Associates, LLC, Baltim Pressure Part DamAnton Banweg, Nalco, an Ecolab Compa Kevin Shields, Structural Integrity Associates, Inc., Annapolis, MD Diluent Aided Cle Jason Grundler, Statoil, Calgary, A Melonie Myszczyszyn, CNRL, Calgary, AB Canada Nanoflotation – waDavid Bromley, chúng tôi P.E., David Br Howard McCarthy, P.E., Tetra Tech Inc., Denver, CO Thermal RecoveryKarina Heitnes Hofstad, Statoil, Tro William A. Shaw, P.E., Veolia Water Solutions & Technologies, Pewau Peracetic Acid UsRobert Ryther, Ph.D., Nalco-ChampioGeorge A. Ganzer, BWA Water Additives US, LLC, Buckingham, PA Selenium Stability Donald Kirk, University of Toronto, T Kashi Banerjee, Veolia Water Solutions & Technologies, Moon Towns Case Study on a SMichael Soller, Bowen Engineering Corp Tom Rutkowski, Golder Associates, Lakewood, CO A Promising Path FKavithaa Loganathan, Ph.D., CanadiaMichael Dejak, Eco-Tec, Pickering, ON, Canada Ion Exchange Syste William Schwartz, P.E., Envirogen Sreekumar Janardhnan, Degremont -Anderson, Ancaster, ON Cana Selective RemovalKevin Slough, Filterboxx Water, Calg Claude Gauthier, Purolite, Bala Cynwyd, PA Chromate RemovalPeter a Meyers, ResinTech, Inc, West Be Cindy Gresham, Therm Oxyanion RemovalEdward Rosenberg, Ph.D., UniversityTerry Heller, Purolite Corporation, Bala Cynwyd, PA Immobilization of Jay Renew, Southern Research InstitPeter Meyers, ResinTech, West Berlin, NJ WRC Overview Jeff Wilson, Southern Company, Bir Peter Meyers, ResinTech, West Berlin, NJ Navigating the New Colleen Layman, HDR Engineering, Kristin Glikbarg, Burns and McDonnell, Kansas City, MO Boiler Feed Water Khalid Farooq, Pall Corporation, Port Kevin Drake, P. Eng., Drake Consulting, Calgary, AB Canada Chemistry of ProduMartin Godfrey, Ph.D., Ecolab, Eaga Kevin Brown, Chemtreat The Case for Elim Michael Dejak, P.Eng., Eco-Tec and S Donald Downey, Purolite Corporation, Paris, Ontario Water Management Scott a Allen, P.E., Golder Associates Diane Martini, Sargent & Lundy, LLC, Chicago, IL Municipal ReclaimTimothy Eggert, W. Dan Harbs, Robin Bruce Alderman, Desalitech, Eastampton, NJ Municipal Recycle Daniel Sampson, WorleyParsons, ValJoel M. Davie, M.S., P.E., Bechtel Power Corporation, Frederick, MD Advanced Membran Riad Al-Samadi, Ph.D., P.Eng., Advanced Water Solutions, Burlington, ON, Canada A Water Toolbox t Leo Kobylka, Statoil Canada Ltd., CaJason Monnell, GAI Consultants, Homestead, PA Oil Fields (Produ Rafique Janjua, P.E., Fluor Enterprise, Inc., Sugar Land, TX Mitigating Risk WhVincent A. Como, P.E., Black & Veatch Mitigating Risk WhSarahann Rackl (Dow), Marrone Bio Innovations, Davis, CA Conserving Gallons Jeanette Shoemaker, Bechtel Power Corp., Frederick, MD Using Reclaimed M Jim Braun, AVANTech, Inc. Using Reclaimed M Michael Wilson, CH2M Hill, Boston, MA Deciphering the ChCharles Meurer, P.E., Stanley Consultants, Inc. Deciphering the ChJames Harwood, GE Power & Water, Oakville, ON Canada Sequential PrecipiTimothy Keister, ProChemTech International, Inc., Brockway, PA On-site Field Anal Scott Tucker, Hach Company, Loveland, CO Opportunities for Timothy Keizer, Nalco, Naperville, IL A Sustainable, En Li An, Veolia Water Systems, Moon Twp., PA Chemical CleaningGlenn Matys, Chemtreat Chemical CleaningKenneth Hansen, The Babcock and Wilcox Company, Barberton, OH Comparative MS2 Daryl B Gisch, The Dow Chemical Company, Midland, MI Design, Build, Op Doug Kellogg, Seimens Design, Build, Op Steven Gagnon, Avantech Inc, Columbia, SC Removal of Trace George Crits, Consultant Removal of Trace Peter Meyers, ResinTech Inc, West Berlin, NJ Arsenic Removal by James Sabzali, Aldex Chemical Company Ltd. Arsenic Removal by Atul Bhagwat, Ion Exchange Global, , India Sulfate Removal t William Tuck, Anderson Water Systems, Inc. Sulfate Removal t David Kratochvil, BioteQ Environmental Technologies, Inc., Vancouver, BC Canada Perchlorate SpecifCharles Drewry, Calgon Carbon Corporation, Fischer, TX Sulfate Discharge Diane Martini, Sargent & Lundy, LLC, Chicago, IL Pilot-Scale Demons Ganesh Kamatkar, Aquatech International Corp. Pilot-Scale Demons Yongheng Huang, Texas A&M University, College Station, TX

Extending the Lin Fredrick Vance, Kemira, Atlanta, GA Chemistry of FGD Ray Post, ChemTreat Chemistry of FGD Thomas Higgins, CH2M HILL, Chantilly, VA Shale Gas Produce James Silva, GE Global Research, Niskayuna, NY ClO2 for TreatmentJosh Prusakiewicz, HDR, Inc. ClO2 for TreatmentGreg Simpson, NRCCA, Grapevine, TX Field Operation Res John Schubert, HDR, Inc. Field Operation Res Charles Kozora, Aquatech International Corp., Canonsburg, PA VSep Solution for Mark Galimberti, New Logic Research – VSEP, State College, PA Toxic Dissolved Dick Meijer, Veolia MPP Systems/ Veolia Water, Ede, Gelderland The Netherlands Wet Air Oxidation Bryan Kumfer, Siemens Industry, Inc, Rothschild, WI Characterization oFrank Castaldi, Ph.D., Golder Associates Characterization oDavid Pecina, General Electric, The Woodlands, Texas Selenium RemovalDaniel Schwarz, Nalco, an Ecolab Company, Naperville, IL Increasing Water Sy Nandan T. Vani, D.Sc., PE, Bechtel Corporation Increasing Water Sy Katariina Majamaa, Dow Chemical Ibrica, S.L., Tarragona, Catalonia Spain Control of Biofou Ken Pandya, AWTS, Inc. Control of Biofou Paul Schook, The Dow Chemical Company, Buffalo Grove, IL A Novel Appoach tDale Wynkoop, Veolia Water (Crown Solutions) A Novel Appoach tWilliam Collentro, Consultant, Plymouth, MA Development of a P M o Malki, American Water Chemicals, Inc. Deepak Musale Development of a P Musale, Nalco, An Ecolab Company, Naperville, IL Constructed Wetlan Diane Martini, Sargent & Lundy, LLC Constructed Wetlan Christopher Snider, Burns & McDonnell, Kansas City, MO A Pilot Demonstra Greg Mandigo, Aquatech International Corporation A Pilot Demonstra Jason (Xinjun) Teng, Southern Company, Birmingham, AL Start-up and Oper Herman Nebrig, Southern Co. Start-up and Oper H Robert Goltz, The Dow Chemcial Company, Midland, MI Design and Start-uJosh R. Prusakiewicz, HDR Inc. Design and Start-uAntonio Lau, Infilco Degremont, Inc., Richmond, VA Silicate Deposit C Corbin Ralph, Nalco, an Ecolab Company, Naperville, IL Silicate Scale Inh Edward Van Doorn, Baker Hughes Inc., Calgary, AB Canada Advances in the CDavid Pernitsky, Suncor Energy, Inc., Calgary, AB Canada Monitoring PPB Lev Glenn Weagle, Champion Technologies, Sherwood Park, AB Canada Intelligent Sensor Jeanette Shoemaker, Betchel Power Intelligent Sensor David Gray, Mettler-Toledo Thornton, Inc., Bedford, MA Nanodex(TM) FilteTerry Heller, Purolite Bala Nanodex(TM) FiltePeter A. Yarnell, Graver Technologies, LLC, Glasgow, DE Effects of Zinc Ad Rachel DeVito, Westhinghouse Electric Company, Cranberry Township, PA Emergency Respons Peter Meyers, ResinTech Emergency Respons James Braun, Avantech Incorporated, Columbia, SC Detection of Orga Christopher B. Wilson, Nalco, An Ecolab Company, Naperville, IL USA Pretreatment and R Yakup Nurdogan, Bechtel National, Inc., Pueblo, CO A Water ConservatRay Post, Chemtreat A Water ConservatGianna Cooley, CDM Smith, Houston, TX Producing DeminerBrian Clarke, Kiewit Producing DeminerDheneshree Lalla, Eskom Holdings SOC Ltd., Johannesburg, Gueteng South Africa Removal of Selen Anna Casasus, Kemira, Atlanta, GA Ash Pond ReplaceWilliam Moore, WesTech Inc. Ash Pond ReplaceThomas Higgins, CH2M HILL, Chantilly, VA Selenite and Sele William Kennedy, P.E., Duke Energy

Selenite and Sele Nancy Sherwood, MAR Systems Inc., Solon, OH FGD Evaporation P J. Michael Marlett, Aquatech International Corp., Hartland, WI Recovery of BTEXPeter Lamke, Golder Associates Inc. Recovery of BTEXH. Robert Goltz, Dow Chemical, Midland, MI SAGD: Produced Wa Arun Mittal, Aquatech International Corp. SAGD: Produced Wa Claude Gauthier, The Purolite Company, Burlington, ON Canada 0ptimization of a Steve Portelance, Worley Parsons 0ptimization of a Melonie Myszczyszyn, Canadian Natural Resources Limited, Calgary, AB Canada Overview of ProduPeter Midgley, Degremont north America – Infilco Dunidas Overview of ProduAvijit Dey, Bechtel Corporation, Houston, TX Modern Film-FormiAnthony Rossi, General Electric Modern Film-FormiWolfgang Hater, BK Giulini GmbH, Duesseldorf, Germany The Continuing CrTony Banweg, Nalco, an Ecolab Company The Continuing CrBrad Buecker, Kiewit Power Engineers, Lenexa, KS Controlling Conde Ted Beardwood, Ashland Technologies Controlling Conde Robert Bartholomew, Sheppard T. Powell Associates, LLC, Baltimore, MD Proper Sampling MRoger Light, DOW Chemical Proper Sampling MDavid Daniels, M&M Engineering, Austin, TX Zero-Liquid Disch Robert Bradley, Veolia Water Solutions & Technologies Zero-Liquid Disch Matthias Loewenberg, GEA Process Engineering Inc., Columbia, MD Treatment of Cool Marvin Drake, Indiantown Cogeneration LP, Indiantown, FL A Unique All MembMichele Migliavacca, Veolia Water Solutions & Technologies A Unique All MembStanley Karrs, Siemens Industries, Inc., Warrendale, PA No Easy Answers: William Moore, Westech Engineering No Easy Answers: Daniel Sampson, WorleyParsons, Vallejo, CA Enhanced Iron CorYongheng Huang, Texas A&M University, College Station, TX Revolutionary Sil Richard Johnson, ITOCHU Chemicals America Inc., Houston, TX Modifying an Exis Dave Malkmus, ResinTech Modifying an Exis Ivan Cooper, Civil & Environmental Consultants, Charlotte, NC An Integrated App Peter Midgley, Degremont North America- Infilco Dundas An Integrated App Dejan Blagojevich, Nalco Company, Naperville, IL Controlling Conde Robert D. Bartholomew, Associate, Sheppard T. Powell Associates, LLC Controlling Conde Kenneth Chen, Fluor Enterprises, Inc., Irvine, CA Silica and Sodium Vickie Olson, Honeywell Altanta, GA Silica and Sodium Randy Turner, Swan Analytical USA, Wheeling, IL Fluidized Bed BiorDavid Enegess, Envirogen Technologies, Inc., Kingwood, TX Zero Liquid Waste:Joseph Swearman, CONSOL Energy, Pittsburgh, PA Integration of Ben Neal Gallagher, Golder Associates Inc., Lakewood, CO New Legionella S Loraine Huchler, MarTech Systems, Inc. New Legionella S Janet Stout, Special Pathogens Laboratory, Pittsburgh, PA MIC (Microbiologi Paul Puckorius, Puckorius & Associates, Inc., Arvada, CO An Integrated ApprDon Holt, Ashland Water Technologies Ajax An Integrated ApprCharles Ascolese, GE Water & Process Technologies, Trevose, PA Field Trial Experi George Dimotsis, Dripping Wet Water Mineral Scale PredMichael Bluemle, Ashland Water Technologies Mineral Scale PredRobert Ferguson, French Creek Software, Inc. Development of anLoraine Huchler, MarTech Systems Development of anDustin Mobley, Black & Veatch Boiler Deposit ConRobert Bartholomew, Sheppard T. Powell Associates LLC Boiler Deposit ConJames Robinson, GE Power and Water A Holistic Water Stephen Frank, GenOn Energy

A Holistic Water Michael Preston, Black & Veatch Public- Private Pa Jonathan Shimko, GAI Consultants Public- Private Pa Javier Gaztelu, INIMA-OHL Group Using Patents to MThomas C. McThenia, Jr., GrayRobinson, P.A. Using Patents to MClifton McCann, Venable LLP Organic Removal wi Rafique Janjua, Fluor Enterprises, Inc. Organic Removal wi Emmanuel Quagraine, Saskatchewan Power Corporation, Shand Power Station SAGD ZLD approaPrit Kotecha, P.E., M.E., Suncor Energy SAGD ZLD approaMark Nicholson, P.E., HPD, a Veolia Water Solutions & Technologies company On-site Sodium Hyp Michele Funk, Bechtel Power Corporation On-site Sodium Hyp Luis Diaz, NextEra Energy Hydrodynamic Cavi Jeri Penrose, Sargent & Lundy, LLC Hydrodynamic Cavi Philip Vella, Ph.D., VRTX Tech Carbon Electrode-b Frank DeSilva, ResinTech Carbon Electrode-b John Barber, GE Power and Water In Search of the HiPeter Yarnell, Graver Technologies In Search of the HiAlan Knapp, Siemens Industry, Inc. Water Technologies Double Pass ElectrMichael Snow, Ph.D., SnowPure Water Technologies Double Pass ElectrSteven Gagnon, AVANTech, Inc Advanced On-line Jim Cairns, ThermoFisher Scientific Advanced On-line David Gray, Mettler-Toledo Thornton, Inc. Factors Impacting Jasbir Gill, The Nalco Company Factors Impacting Zahid Amjad, Lubrizol Advanced Materials, Inc. Using Permeat Suct Peter Waldron, Toray Membrane USA. Using Permeat Suct Awad El-Shamy, Crane Environmental, Inc. Anti-fouling Memb Doug Frick, Porex Filtration Anti-fouling Memb Joon Min, BKT Membrane PretreatKelly Lange-Haider, Dow Water and Process Solutions Membrane PretreatJonathan Dietrich, Dietrich Consulting Group, LLC Optical Based SoftKent Peterson, Fluid Imaging Technologies, Inc. New No Solvent Met Duane Germenis, Turner Designs Hydrocarbon Instruments, Inc. Analytical Method E. J. Van Doorn, Baker Hughes Inc. On-site LaboratorySudhir Parab, ConocoPhillips Canada Experiences and Ch Ivan Morales, MEG Energy Selenium Control Tom Higgins, Aquatech International Corp. Selenium Control Katherine Searcy, Trimeric Corporation Solidification of F Patricia Scroggin, Burns & McDonnell Solidification of F Mark Owens, Degremont Technologies The Use of Constru Dan Sampson, Worley Parsons The Use of Constru Jared Morrison, Westar Energy, Inc. Concepts in Zero-LBrad Buecker, Kiewit Power Engineers Concepts in Zero-LMatthias Loewenberg, Ph.D., GEA Process Engineering Inc. Filll Selection an Tony Selby, Water Technology Consultants, Inc. Filll Selection an Brad Buecker, Kiewit Power Engineers A Review of the CoBruce Chamberlain, Ashland Water Technologies A Review of the CoElizabeth Harrelson, Enviro Tech Chemical Services, Inc. A Novel Anti-Scali Emmanuel Quagraine, Ph.D., Saskatchewan Power Corporation A Novel Anti-Scali Jie Lu, Ph.D., World Minerals, Inc. Challenges with thVickie Olson, Honeywell Challenges with thJasbir Gill, Ph.D., Nalco Company Iron Oxide Empla Jim Knoll, Graver Technologies

Iron Oxide Empla T.S. Abia II, Texas A&M University – Biological and Agricultural Engineering Dept. Absorbent Technolo Bob Goltz, The Dow Chemical Company Absorbent Technolo Gina Sacco, MAR Systems Inc. Advanced OxidatioBarbara Schilling, Ozonia N.A. Advanced OxidatioIvano Aglietto, Ph.D., SA Envitech s.r.l. Chromate RemovalTerry Heller, The Purolite Company Chromate RemovalDean Neshem, CH2M HILL Plateau Remediation Company Solidification: A Peter Midgley, Degeremont Technologies Solidification: A Scott Tavaglione, GE Water & Process Technologies An Innovative Sol Steve Portelance, WorleyParsons Canada An Innovative Sol Susan Sun, New Technology Oil Sands Cenovus Corp. Filtration Proces Rudy Tamayo, Husky Energy Filtration Proces Ramesh Sharma, ConocoPhillips Company Produced Water So George Crits, Aqua-Zeolite Sciences Produced Water So Stephen Moylan, The Purolite Company Evaluation of The Bill Shaw, Veolia Water Solutions & Technologies Evaluation of The Herman Nebrig, Southern Company Services Case Study on SelVince Como, Black & Veatch Case Study on SelMichael Soller, Bowen Engineering Corporation Demonstration TesKristin Collier, Burns & McDonnell Demonstration TesThomas Higgins, CH2M HILL Evaluation of CarbWilliam Kennedy, Orion Engineering, PLLC Evaluation of CarbAntonio Lau, Infilco Degremont, Inc. Innovative MonitorRay Post, ChemTreat Inc. Innovative MonitorBruce Chamberlain, Ashland Water Technology Novel Biocide DeliTom Armon, H-O-H Water Technology Novel Biocide DeliDorothy Reynolds, GE Power & Water, Water & Process Technologies Important ConsiderRick Kreuser, RTK Technologies Inc. Important ConsiderPaul Puckorius, Puckorius & Associates, Inc Design of the Biot Jill Sonstegard, GE Water Design of the Biot Yakup Nurdogan, Bechtel National, Inc. Thirty Years of SucJohn Schubert, HDR Engineering Thirty Years of SucDavid Pickard, Premier Magnesia LLC Filtration of Solu F. Tepper, Argonide Corporation Nutrient Control – Bob Goltz, The DOW Chemical Company Nutrient Control – John Richardson, ChemTreat Enzymatic RemovalJohn Christiansen, CDM, Inc. Enzymatic Removal Greg DeLozier, Novozymes A/S Ceramic Membranes Rick Szilagyi, WesTech Engineering Ceramic Membranes R. Gay-de-Montella, Worley Parsons Canada Challenges and PrJohn Christiansen, CDM Piloting ConventioJeff Easton, WesTech Engineering, Inc. Piloting ConventioRichard Mah, Suncor Energy Use of Mobile TechJames Braun, AVANTech Use of Mobile TechJeanette Shoemaker, Bechtel Power Corp Upgrade of Condens Khalid Farooq, Pall Corp. Operating ExperieLewis Crone, Dominion Nuclear Connecticut, Inc. Monitoring of Trac Dave Silverman, Advanced Water Engineering, Inc. Monitoring of Trac Kenneth Ogan, Advanced MicroLabs Study of The ScaliBarbara Moriarty, Nalco Company Study of The ScaliOlivier Horner, EDF R&D

Achieving Complete Diane Martini, Sargent & Lundy Achieving Complete John Williamson, Infilco Degremont, Inc Monochloramine Re Stephanie Carr, Calgon Carbon Corporation Monochloramine Re William Collentro, Worcester Polytechnic Institute Forward Osmosis A Donald Kirk, University of Toronto Forward Osmosis A Peter Nicoll, Modern Water plc AMD Reuse and Oth Louiza Bell, Teck Metals Ltd AMD Reuse and Oth Scott Quinlan, P.E., C.B.C., GAI Consultants, Inc. Sulfate Removal frJoseph Swearman, Consol Energy Sulfate Removal frDavid Kratochvil, Ph.D., chúng tôi BioteQ Environmental Technologies Mine Waste Clean-U Tom Rutkowski, Golder Associates Inc. Mine Waste Clean-U Edward Rosenberg, Ph.D., University of Montana (UM)/Purity Systems Inc(PSI) The Real Cost of ZDan Dudek, Siemens Industry, Inc. The Real Cost of ZWilliam Shaw, P.E., HPD, LLC, a Veolia Water Solutions & Technologies company Evaluation of BracJohn Schubert, HDR Engineering, Inc. Evaluation of BracBethany Kurz, Energy & Environmental Research Center NORM Removal fro Jerry Penland, Chester Engineers NORM Removal fro James Silva, Ph.D., General Electric Global Research Seawater RegenerDave Dally, Lanxess Seawater RegenerDonald Downey, Purolite Company Molybdate RemovaClaude Gauthier, The Purolite Company Molybdate RemovaAndrew Bishop, ResinTech Amine Chemistry -Jim Wiegand, ChemTreat, Inc. Amine Chemistry -Gregory Bachman, Technology and Lab Services Municipal Wastewat Robert Bradley, Veolia Water Solutions & Technologies Municipal Wastewat Katariina Majamaa, Dow Chemical Ibérica, S.L. Recovery and Recyc Arun Mittal, Aquatech International Corporation Recovery and Recyc Michael Chan, Duraflow, LLC Principles of a Li William Moore, Aquatech International Corporation Principles of a Li Bruce Folkedahl, Ph.D., Energy & Environmetal Research Center MBBR AND DAF Chuck Hewell, P.E.-CLH Consulting, LLC MBBR AND DAF Chandler H. Johnson – World Water Works, Inc. Horizontal EvaporaSudhir D. Parab, P.Eng.-ConocoPhillips Canada Limited Horizontal EvaporaJ. Michael Marlett, P.E., chúng tôi – Aquatech International Corporation Field Comparison o Donald Weakley – Ashland Hercules Water Technologies Hydrodynamic Cavi Phil Vella, Ph.D. – VRTX Technologies A New MechanicalYoung W Cho, Ph.D. – Drexel University Water Recovery viJoseph Tinto – GE Water & Process Technologies – RCC Thermal NORM Removal from Jerry Penland-Chester Engineers NORM Removal from James Silva – GE Global Research Center Precipitation ReacScott C. Quinlan, P.E.-GAI Consultants, Inc. Precipitation ReacJohn Schubert, P.E. – HDR Engineering, Inc. Water Management: Bradley D. Wolf, P.E.-Navigant Consulting Water Management: Chuck Kozora – Aquatech Carbon Capture TeWayne Micheletti-Wayne C. Micheletti Carbon Capture TeSandra Kolvick, P.E. – Fluor Enterprises, Inc. Experiences UsingPeter Lemke, PE-MWH Americas, Inc. Experiences UsingRobin Kluck – GE Water and Process Technologies Incorporating SafeJames Braun-AVANTech, inc. Incorporating SafeMichele Funk, P.E. – Bechtel Power Corporation Automated Selenium Steve Gagnon-AVANTech, Inc.

In Situ Monitoring Jon Vernon-Lyondell Chemical Company In Situ Monitoring Olivier Horner – Electricite de France R&D State of the Art of Michael H. Dorsey-DuPont Engineering State of the Art of David Hasson – Rabin Desalination Laboratory, Grand Water Research Institute, Technion Barium and StrontRickey E. Smith-Southern Company Services Barium and StrontRobert Ferguson – French Creek Software, Inc. Successful Pilot T David Marrs, P.E.-Valero Energy Corporation Successful Pilot T Arun Mittal – Aquatech International Corporation Resource Recovery Jared Meiser – Veolia Water State of Options Bill Moore, P.E.-Aquatech State of Options Rafique Janjua P.E. – Fluor Calcium CarbonatePaul Puckorius – Puckorius & Associates, Inc. Development of NeVance Lumme-GE Water & Process Technologies Development of NeRay Post – ChemTreat, Inc Produced Water Re Lewis Krause – Eco-Tec Inc. Corrosion Control Melonie Myszczyszyn-Canadian Natural Resources Limited Corrosion Control Jasbir Gill, Ph.D. – Nalco Company Subsurface Wash a Claude Gauthier P. Eng.-The Purolite Company Subsurface Wash a Tamer Antar, EIT – Cenovus Energy ABMet: Setting th A. Paul Togna, Ph.D.-Environmental Operating Solutions, Inc. ABMet: Setting th Jill Sonstegard – GE Power & Water Technologies Case Study: Cost-Thomas Lawry-HDR Engineering, Inc. Case Study: Cost-Tony Lau, Ph.D. – Infilco Degremont, Inc. Selenium Speciati Corey A. Tyree, Ph.D.-Southern Company Services Selenium Speciati Gary Blythe – URS Corporation Study of DeaeratorDouglas Dewitt-Dick-Champion Technologies Study of Deaerator Richard of Peterson – Nalco Company Evaluating the Correlation Low Conductivity Boiler Water Pitting and Generalized Corrosion Analysis Compared to Coupons Chemistry Jim Robinson-GE Water & Process Technologies Evaluating the Correlation of Low Conductivity Boiler Water Pitting and Generalized Corrosion Analysis Compared to Coupons Chemistry Vickie G. Olson – Honeywell Field Solutions Research Evaluatio Roger Light-Dow Chemical Company Research Evaluatio Rosa Crovetto – GE Power & Water, Water & Process Technologies Removal of Water Y Pongheng Huang, Ph.D., P.E.-Impaired Water Treatment Removal of Water A Pdva Zach-Maor – Technion Biological TreatmeDavid D. Friese-Applied Process Technologies Biological TreatmeTom Rutkowski – Golder Associates Inc. Emerging BiologicJoon H. Min, Ph.D.-Psomas Emerging BiologicYakup Nurdogan, Ph.D., PE – Bechtel National, Inc. Generating Power Peter Lemke, PE-MWH Americas, Inc. Generating Power Juan Josse – HDR Physico-Chemical F Trank Johns, PE-Tetra Tech Physico-Chemical A Tnal Chavan, Ph.D. – Siemens Information Systems Ltd. Commissioning of Rafique Janjua, P.E.-Fluor Commissioning of Eric Blumenstein – Golder Associates Inc. Integrating Polluti Nandan Vani-Bechtel Power Corporation Integrating Polluti David Nystuen – SES Environmental Patent Law DeveloDavid A. Velegol-Chester Engineers Patent Law DeveloClifton E. McCann – Venable LLP A Report On ResinWilliam Moore – Aquatech Amine Form Operat Lewis Crone – Dominion Nuclear Connecticut, Inc. Design ConsideratiGerald (Jerry) Alexander – Siemens Water Technologies Aspects of Cycle Moderator: Deborah Bloom, Nalco Company, Naperville, IL

HRSG & Boiler TubModerator; Robert Bartholomew, Sheppard T. Powell Associates LLC, Baltimore, MD Optimizing Chemica IRVIN J. COTTON, Arthur Freedman Associates, Newport, RI Optimizing Chemica ROBIN W. KLUCK, JAMES O. ROBINSON, GE Water and Process Technologies, Trevose, PA Optimizing Chemica ROBIN W. KLUCK, JAMES O. ROBINSON, GE Water and Process Technologies, Trevose, PA There is a Hurric ROGER W. LIGHT, The Dow Chemical Company, Freeport, TX There is a Hurric CHRISTOPHER VOTAVA, Cottonwood Energy, Deweyville, TX; J. C. DROMGOOLE, Fort Bend Services, S Advances in AnalytHAKAN GURLEYUK, RUSSELL GERARDS, Applied Speciation and Consulting, LLC, Tukwila, WA Development of anCLAUDE GAUTHIER, P.Eng., The Purolite Company (Canada), Paris, ON, Canada Development of anJEON SOO MOON, PYL YANG PARK, JAE KUEN LEE, Korea Electric Power Research Institute, Daejoon, Evaluating On-LineVICKIE G. OLSON, Honeywell Field Solutions, Atlanta, GA; JODIRAH GREEN, Progress Energy Carolinas Application of NewJOHN C. VOGT, Chemtreat, Inc, Glen Allen, VA Application of NewCHRISTINA FLEMING, JOHN OSTBERG, Nalco Company, Naperville, IL Selective RemovalROBERT J. FERGUSON, French Creek Software, Inc., Kimberton, PA Selective RemovalSHIRISH NAIK, SHARDUL KSHIRSAGAR, R. PARASHTEKAR, S KRISHNAN, KIRAN DESHPANDE, Ther Selective Silica R LYLE KIRMAN, Tangent Company, LLC, Chagrin Falls, OH Selective Silica R PETER MEYERS, ResinTech, Inc., West Berlin, NJ The Improvement of GEORGE J. CRITS, WILLLIAM RUNYAN, Idreco USA, West Chester, PA The Improvement of SHOJI AOKI, JUNICHI KANNO, HIDEO KAWAZU, Ebara Clean Environment Co., Ltd., Fujisawa, Kanagaw Beyond “Green TecAL NEBRIG, Southern Company Generation, Birmingham, AL Beyond “Green TecMARY GLASS, Mexel USA, LLC, Arlington, VA; DENNIS HUNTER YANKEE, Tennessee Valley Authority, K Beyond “Green TecMARY GLASS, Mexel USA, LLC, Arlington, VA; DENNIS HUNTER YANKEE, Tennessee Valley Authority, K A Field Evaluatio LORAINE HUCHLER, PE, CMC, MarTech Systems, Inc., Lawrenceville, NJ A Field Evaluatio M. P. PATTON, D. W. ALLEY, Clearwater Systems Corporation, Essex, CT EPA Rule 316b – CWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA EPA Rule 316b – CHENRY C. HUNT, Ranney Collector Wells, Columbus, OH Cost-Effective NewDAVE CHRISTOPHERSEN, Crown Solutions, a subsidiary of Veolia Water Solutions & Technologies, Vand Cost-Effective NewSANG-HEA SHIM, Justeq LLC, Northbrook, IL Automated Feed ofGEORGE DIMOTSIS, Dripping Wet Water, Inc., Boerne, TX Automated Feed ofJOHN W. BYRNE, BASF Corporation, Wyandotte, MI Automated Feed ofJOHN BYRNE, RICHARD McCAFFREY, BASF Catalysts, LLC, Iselin, NJ; KEITH HIRSCH, BASF Corporat Produced Water Re DAN BJORKLUND, Aquatech International Corporation, Hartland, WI Optimization of S NATHAN S. HARALSON, GREGG L. WILSON, WILLIAM F. HEINS, GE Water & Process Technologies, Be Treatment of Oilfi MILIND KULKARNI, Hatch & Associates, Calgary, AB, Canda Treatment of Oilfi PETER R. LEMKE, KEVIN W. CONROY, Golder Associates Inc., Denver, CO; ISAAC JURAWAN, PE., T.N. Reuse of Water forDOROTHY NEU, HPD/Veolia Water Solutions & Technologies, Pewaukee, WI Reuse of Water forJASBIR S. GILL, PhD., Nalco Company, Naperville, IL New Low-TemperatTHOMAS E. HIGGINS, PhD., PE., CH2M Hill, Chantilly, VA New Low-TemperatWILLIAM A. SHAW, PE., HPD, LLC, a Veolia Water Solutions & Technologies Company, Pewaukee, WI ZLD Achieved for tGREG MANDIGO, Aquatech ICD, Hartland, WI Evaluating Biologi F. DAVID FITZGERALD, PE, Progress Energy (Contract), Raleigh, NC Evaluating Biologi ERIC BLUMENSTEIN, KEVIN CONROY, JIM GUSEK, Golder Associates, Inc, Lakewood, CO A Zero Liquid Dis ROBERT L. HAMILTON, PE., Hamilton Engineering, Inc., Denver, CO A Zero Liquid Dis H. ROBERT GOLTZ, PhD., CHRIS EICHER, The Dow Chemical Company, Midland, MI; NAOMI LEVY, Infil Application Study ARTHUR J. FREEDMAN, PhD, Arthur Freedman Associates, Inc., East Stroudsburg, PA Application Study JAMES JEON SOO MOON, JAE KUEN YANGInc., PARK, KoreaTX Electric Power Research Institute, Daejoon, C. DROMGOOLE, Fort LEE, BendPYL Services, Stafford, An Update of ReusIWC-09-47D An Update of ReusPAUL R. PUCKORIUS, Puckorius & Associates, Inc., Arvada, CO The Impact of WatPETER ELLIOTT, GE Water & Process Technologies, Trevose, PA Organic Chemical JTAMES ROBINSON, ROBIN KLUCK, ANTHONY ROSSI, GE Water and Process Technologies, Trevose, PA Cycle Chemistry PK. ANTHONY SELBY, Water Technology Consultants, Inc., Evergreen, CO Real-time CorrosioJOE ZIMMERMAN, CHEMTRAC Systems, Inc, Norcross, GA

The Impact of OxidDANIEL C. SAMPSON, Nalco Company, Vallejo, CA; JAMES MOEN, Roseville Energy Park, Roseville, CA Toxic Dissolved a EDWARD V. CARR, Buckman USA, St. Louis, MO Toxic Dissolved a DICK TH. MEIJER, VWS Oil & Gas / MPP Systems, Veolia Water Solutions & Technologies, Ede, The Neth Innovative Ion ExcSTEPHEN DOMINICK, Siemens Water Technologies Corp., Chicago, IL Innovative Ion ExcMICHAEL BRATTY, DAVID KRATCHOVIL, DAVID SANGUINETTI, SONGLIN YE, TERYL MURRAY, BioteQ Innovative Water JEFFREY S. MALLORY, Sargent & Lundy, LLC, Chicago, IL Innovative Water PETER G. DEMAKOS, PE, CHRIS IMIOLA, Niagara Blower Heat Transfer Solutions, Buffalo, NY A Case Study: ProKEVIN DRAKE, AMEC BDR Limited – Calgary, AB, Canada A Case Study: ProBRIAN D. HUFF, ERIC BLUMENSTEIN, Golder Associates Inc., Lakewood, CO SAGD Fit-for-Purpo SUDHIR PARAB, Conoco Phillips Canada Limited, Calgary, AB, Canada SAGD Fit-for-Purpo KEITH R. MINNICH, PE., VWS Oil & Gas, Calgary, AB, Canada; DOROTHY NEU, Veolia Water Solutions & SAGD Fit-for-Purpo KEITH R. MINNICH, PE., VWS Oil & Gas, Calgary, AB, Canada; DOROTHY NEU, Veolia Water Solutions & Steam Generation MELONIE MYSZCZYSZYN, Canadian Natural Resources Ltd., Bonnyville, AB, Canada Steam Generation MARTIN R. GODFREY, Nalco Company, Naperville, IL Trace Mercury Rem WILLIAM M. KENNEDY, PE., Orion Engineering, PLLC, Charlotte, NC Trace Mercury Rem MARK OWENS, PE., Degremont Technologies, Richmond, VA; H. ROBERT GOLTZ, PhD., The Dow Chem Intercomparison ofFREDERICK W. VANCE, PhD, Dow Water & Process Solutions, Midland, MI Intercomparison ofRUSSELL GERARDS, JACOB MEYER, Applied Speciation and Consulting, LLC, Tukwila, WA Monitoring the ChlCOREY A. TYREE, PhD., Southern Company Services, Birmingham, AL Monitoring the ChlJOHN J. KIM, PE., PhD., JJK Water Technology, Inc., Huntersville, NC; ROBBIN M. JOLLY, Duke Energy C Source Water ChaMILTON OWEN, URS Corporation, Austin, TX Source Water ChaTHOMAS LAWRY, HDR Engineering, Inc, Pittsburgh, PA A Review of Hydr DAVID DANIELS, M&M Engineering Associates Inc., Austin, TX Oxygen Corrosion EWA a M. LABUDA, ROBERT D. BARTHOLOMEW, Sheppard T. Powell Associates, LLC, Baltimore, MD HRSG FAC BILL BOYD, MARK VOGT, Dynegy Generation, O’Fallon, IL The Impact of FeeMEL J. ESMACHER, PE., GE Water & Process Technologies, The Woodlands, TX; ANTHONY ROSSI, GE Evaluation of RO, K. ANTHONY SELBY, Water Technology Consultants, Inc., Evergreen, CO Evaluation of RO, EMMANUEL QUAGRAINE, PhD., Shand Power Station, SaskPower, Estevan, SK, Canada; KEITH HILL, F Evaluation of RO, EMMANUEL QUAGRAINE, PhD., Shand Power Station, SaskPower, Estevan, SK, Canada; KEITH HILL, F Zero Liquid Disch DAVID VELEGOL, Jr., Chester Engineers, Moon Township, PA Zero Liquid Disch LAWRENCE V. KRZESOWSKI, General Motors, Warren, MI; LNSP NAGGHAPPAN, DAVID E. PARKER, N Zero Liquid Disch LAWRENCE V. KRZESOWSKI, General Motors, Warren, MI; LNSP NAGGHAPPAN, DAVID E. PARKER, N Achieving the Firs DIANE R. MARTINI, Sargent & Lundy, LLC, Chicago, IL Achieving the Firs BRUCE KASNITZ, GREGG WILSON, ROBERT SOLOMON, GE Water and Process Technologies, Bellevu Contaminant Reduc BARBARA SCHILLING, Degremont Technologies, Elmwood Park, NJ Contaminant Reduc ANDREW COLLENTRO, Water Consulting Specialists, Inc., Doylestown, PA Technology Selectio GREGORY OSEN, Christ Water Technology Americas, LLC, New Britain, CT Technology Selectio ROCH LAFLAMME, ROBERT GERARD, Montreal-Amsterdam GE Water and Process Technologies Technology Selectio ROCH LAFLAMME, ROBERT GERARD, Montreal-Amsterdam GE Water and Process Technologies, Refinery WastewatRAFIQUE JANJUA, Fluor Corporation, Sugar Land, TX Refinery and HeavMIKE BRADFORD, Jacobs Consultancy, Houston, TX; IAN BUCHANAN, Jacobs Consultancy Canada, Cal Oil-In-Water Fluor VADIM MALKOV, PhD., Hach Company, Loveland, CO; DIETMAR SIEVERT, PhD., Hach Lange GmbH, Du Circular API SeparMIKE L. BRADFORD, SHAUN T. KUSEK, PhD., Jacobs Consultancy Inc., Houston, TX Selenium RemovalAVIJIT DEY, PhD., RAJENDRA KULKARNI, Bechtel Corporation, Houston, TX Meeting Environmen COLLEEN M. LAYMAN, PE, Bechtel Power Corporation, Frederick, MD Meeting Environmen RAYMOND M. POST, PE., RICHARD H. TRIBBLE, HELEN R. CERRA, ChemTreat, Inc., Glen Allen, VA; TO Cycle Chemistry Ch RONALD M. HORN, GE Energy, SANDY J. SCHEXNAILDER, GE Water & Process Technologies, Dallas, T Cycle Chemistry Ch EARNESTINE JOHNSON, JEANETTE SHOEMAKER, CHRISTOPHER HUTH, KUMAR SINHA, Bechtel Po Cycle Chemistry Ch EARNESTINE JOHNSON, JEANETTE SHOEMAKER, CHRISTOPHER HUTH, KUMAR SINHA, Bechtel Po Well, How Long is AMARDEEP VENUS KAUR, Southern California Edison, Westminster, CA The Hybrid CeramBUM SOO CHOI, LEE SOO YOON, DONG SUN SHIN, Kwater, Daejeon, South Korea

Use of Unique FracBILL LOYD, The Dow Chemical Company, Minneapolis , MN Use of Unique FracDEVESH MITTAL, V. J. NATHAN, Aquatech International Corporation, Canonsburg, PA; NARENDER SING Prevention of Cal GREGG POPPE, SCOTT BEARDSLEY, The Dow Chemical Company, Minneapolis, MN Prevention of Cal STEPHEN P. CHESTERS, FERNANDO DEL VIGO, EDWARD G. DARTON, Genesys International Limited, Case History of a JONATHAN WOOD, Siemens Water Technologies, Lowell, MA Case History of a STEVEN R. GAGNON, JEFF HARTMAN, AVANTech, Inc., Columbia, SC Instrumentation a JANE KUCERA, Nalco Company, Naperville, IL Instrumentation a JOSEPH P. VANDEHEY, Global Water Treatment Biological Nitrog MICHAEL L. PUDVAY, Infilco Degremont, Inc., Richmond, VA Biological Nitrog PAMELA EDRICH, ERIC BLUMENSTEIN, PAUL PIGEON, Golder Associates Inc., Denver, CO; JOE MIDDL Sustainable ReuseBRENT W. COWAN, CSC Technology, Inc., Coatesville, PA Sustainable ReuseANTON G. CALLERY, ITT Water & Wastewater S.A., Callao, Lima, Peru Biological TreatmeBRIAN P. FLYNN, PE, BCEE, MRE Associates, Austin, TX Biological TreatmeSHASHI GORUR, Siemens Water Technologies, Warrendale, PA; ROBERT MILLER, Mariani Packing Co., Patent Pitfalls: W RICHARD POSA, JOHN WALISZEWSKI, SAMCO Technologies, Inc., Buffalo, NY Patent Pitfalls: W CLIFTON E. McCANN, CHRISTOPHER S. CROOK, Venable LLP, Washington, DC Evaluation of Tre CLAUDE GAUTHIER, P.Eng., The Purolite Company, Paris, ON, Canada Evaluation of Tre KYLE SMITH, Dow Water and Process Solutions; Midland, MI; ANTONIO O. LAU, PhD., Infilco Degremont MBBR TechnologyJERRY P PENLAND, Chester Engineers, Moon Township, PA MBBR TechnologyCHRISTIAN P CABRAL, EVA SANTOS, MARK SMOCK, WILLIAM KELLY, N.A. Water Systems, a Veolia Wat MBBR TechnologyCHRISTIAN P CABRAL, EVA SANTOS, MARK SMOCK, WILLIAM KELLY, N.A. Water Systems, a Veolia Wat Three-Phase Mining ANDREW STERN, Chester Engineers, Charleroi, PA Three-Phase Mining CHRISTOPHER HOWELL, DAVE CHRISTOPHERSEN, Crown Solutions Co., LLC – a Veolia Water Solutio How A Media FilterJOHN SCHUBERT, PE., HDR, Pittsburgh, PA How A Media FilterJOE HALIGOWSKI, Filtra Systems Company, Farmington Hills, MI How A Media FilterJOE HALIGOWSKI, Filtra Systems Company, Farmington Hills, MI DAVID KLANECKY, Global Research & Development, (R&D), Director of Dow Water & Process Solutions, E Water Treatment i Panel Discussiom Moderator: James Robinson, GE Water & Process Technologies, Trevose, PA Trace ContaminantPETER RITCHEY, Severn Trent Water Purification, Inc., Pittsburgh, PA Trace ContaminantH. ROBERT GOLTZ, PhD, Dow Chemical, Midland, MI Porous Polymers Vi WILLIAM C. ZAVORA, PE, Calgon Carbon Corporation, Santa Fe Springs, CA Porous Polymers Vi ROBERT L. ALBRIGHT, PhD, Albright Consulting, Southampton, PA Use of High Efficiency Reverse Osmosis (HERO), Brine Concentration and Crystallization at the ARUN World’s MITTAL, First Two Aquatech Zero Liquid International Discharge Corporation, (ZLD) Ethanol Canonsburg, Plants PA Use of High Efficiency Reverse Osmosis (HERO), Brine Concentration and Crystallization at the RUSSELL World’s First VANDENBERG, Two Zero Liquid NIMAI Discharge MILLER (ZLD) , GE Water Ethanol & Process Plants Technologies, RCC Thermal Products, Be Navajo GeneratingMICHAEL L. WISDOM, P. E., ContourGlobal, Houston, TX Navajo GeneratingROBERT B. PETERSON, JERRY N. KOGER,, Salt River Project, Page, AZ; TIMOTHY J. RITTOF, HPD, LL Evaporation of Was MICHAEL C. PRESTON, Black & Veatch Corporation, Overland Park, KS Evaporation of Was WILLIAM A. SHAW, P.E., HPD, LLC, a Veolia Water Solutions & Technologies Company, Pewaukee, WI Achieving ReliableWILLIAM E. MOORE, Fluor Enterprises, Inc., Sugar Land, TX Achieving ReliableLANNY WEIMER, CAROLINA GONZALEZ, ROBERT SOLOMON, GE Water & Process Technologies, Ellic Water Conservation JOHN T. LUCEY, Jr., HDR Engineering, Pittsburgh, PA Water Conservation ANDREW MARKLE, P.Eng., MPR Associates Inc., Alexandria, VA Technical Assessme ALFONSO SALINAS, Crown Solutions, Vandalia, OH Technical Assessme CHRISTOPHER STACKLIN, PE., JERRY EVANGELISTA, PE., Orange County Sanitation District, Fountain Technical Assessme CHRISTOPHER STACKLIN, PE., JERRY EVANGELISTA, PE., Orange County Sanitation District, Fountain Design ConsideratCHRISTIAN CABRAL, NA Water Systems, Pittsburgh, PA Design ConsideratBRIAN AYLAIAN, HONG YIN, Metcalf & Eddy/AECOM, Laurel, MD Keeping it Green aBRIAN AYLAIAN, Metcalf & Eddy/AECOM, Laurel, MD Keeping it Green aCRISTINA DEL PICCOLO, Led Italia srl., Zoppola, PN, Italy; TINA MASTERS ODUM, P.E. Crown Solutions Evaluating Impact J. C. DROMGOOLE, Fortbend Services, Inc., Stafford, TX Evaluating Impact FRANK J. CASTALDI, Brown and Caldwell, Austin, TX; JEFF ALLEN, P.E., Brown and Caldwell, Saint Paul,

Biological TreatmeENOS L. STOVER, Stover & Associates, Inc., Stillwater, OK; MICHAEL PUDVAY, ROBERT F. KELLY; ANTO Closed Cooling SyBARBARA MORIARTY, JOAN CHAO, DAN CICERO, CRAIG MYERS; F. PHILIP YU, Nalco Company, Nap Review Of Closed JAY FARMERIE, SUSAN REY, GARY REGGIANI, Cyrus Rice Water Consultants, Pittsburgh, PA, & Associ Case Histories: UsWILLIAM STAPP, AS Inc., Santa Rosa, CA; PAUL PUCKORIUS, Puckorius and Associates, Arvada, CO; CL Water (Resource) PETER G. DEMAKOS, P.E., Niagara Blower Co., Buffalo, NY Tracking MolybdenVADIM B. MALKOV, Hach Company, Loveland, CO; BLAINE NAGAO, ChemCal, Inc. Pushing the Limi TRACY A. BARKER, AVANtech, Inc., Oak Ridge, TN Pushing the Limi CRAIG A. BRODEN, Dow Water Solutions, Edina, MN; CLIFFORD D. GILBERT, Dow Water Solutions, Mou EDI Performance aJEFFREY TATE, Agape Water Solutions, Inc., Harleysville, PA EDI Performance aJOHN BARBER, GE Water and Process Technologies, Guelph, ON, Canada; DAVID F. TESSIER, GE Wate Commission of a W STEVEN R. GAGNON, AVANTech, Inc., Columbia, SC ZLD Solutions for PATRICIA M. SCROGGIN, PE, Burns and Mcdonnell, Kansas City, MO ZLD Solutions for GREG MANDIGO, Aquatech ICD, Hartland, WI Evaporation of Pu MICHAEL C. PRESTON, Black & Veatch Corporation, Overland Park, KS Evaporation of Pu MARK C. NICHOLSON, PE, HPD LLC, Plainfield, IL Considerations ImPAUL CHU, EPRI, Palo Alto, CA Considerations ImCOLLEEN A. CHAPMAN,. COLLEEN M. LAYMAN, PE, Bechtel Power Corporation, Frederick, MD Designing Construc CYNTHIA MURRAY-GULDE, F. DOUGLAS MOONEY, ENTRIX, Atlanta, GA; GEORGE M. HUDDLESTON I Cooling Tower RetrJAMES W. CUCHENS, P.E., Southern Company Services, Inc, Birmingham, AL Cooling Tower RetrBRUCE A. LARKIN, PE, Black & Veatch, Overland Park, KS; SUSAN CINELLI, Board of Public Utilities (BP Cooling Tower RetrBRUCE A. LARKIN, PE, Black & Veatch, Overland Park, KS; SUSAN CINELLI, Board of Public Utilities (BP Calcium HypochlorFARAH D. AZARNIA, Albemarle Corporation, Baton Rouge, LA Calcium HypochlorSTANLEY R. PICKENS, PH.D, PPG Industries, Inc., Monroeville, PA Benefits of Soft W JAMES G. KANUTH, ChemTreat, Inc. Glen Alllen, VA Benefits of Soft W WILLIAM F. HARFST, Harfst and Associates, Inc., Crystal Lake, IL Considerations in DOUGLAS DEWITT-DICK, Ashland Water Technologies, Portland, TX; CARLOS BENAVIDES, Topaz Energ EPRI and ASME Gu WILLIAM MOORE, Calpine Corporation, Houston, TX Lay-Up and ReturnMICHAEL CARAVAGGIO, Ontario Power Generation, Courtright, ON, Canada Declining PressureWILLIAM H. STROMAN, Primary Energy, San Diego, CA A Novel Approach T t OM PIKE, Western Farmers Electric Co-operative, Fort Towson, OK; DOUGLAS DEWITT-DICK, Ashland W I Have to Lay Boil JAMES C. DROMGOOLE, Fort Bend Services, Inc., Stafford, TX Affect and ChallenMELONIE MYSZCZYSZYN, Canadian Natural Resources Limited, Bonnyville, AB, Canada Waste Water Treatm KAREN KWASNIEWSKI, Encana Oil and Gas Partnership, Calgary, AB, Canada Waste Water Treatm STEVE N. PORTELANCE, WorleyParsons MEG, Calgary, AB, Canada Water Reuse PlantCAROLINE WILSON MUSSBACHER, Encana Integrated Oilsands Division, Calgary, AB,Canada Water Reuse PlantROWENA PENARANDA, RAFAEL GAY-DE-MONTELLA, Colt Engineering Corporation, Calgary, AB, Canad Equipment DesignJOHN E. FAIR P. ENG., Fair Canada Engineering Ltd., Calgary, AB, Canada Equipment DesignROBERT HOLLOWAY, Holloway Associates, Etobicoke, ON, Canada; GORDON PAGE, Page Technology L Equipment DesignROBERT HOLLOWAY, Holloway Associates, Etobicoke, ON, Canada; GORDON PAGE, Page Technology L Fate of Arsenic, T R. DAVID G. PYNE, ASR Systems LLC, Gainesville, FL Legionella: Proble GEORGE LICINA, Structural Integrity Associates, Inc., San Jose, CA Cutting-Edge Techn ZHE ZHANG, Ph.D., San Air Technologies Laboratory, Inc., Powhatan, VA; PAUL PUCKORIUS, Puckorius & An Overview of Leg DIANE MISKOWSKI, MPH EMSL Analytical Inc., Westmont, NJ Use of IQ-CHECK™ HÉLÈNE L FRENKIEL-LEBOSSÉ, SOPHIE PIERRE, VELIANA TODOROVA, FRÉDERIC MARTINEZ, Bio-Ra Ion Exchange Resin GORDON ROSSITER, IXSEP, Houston, TX Ion Exchange Resin HEIKKI MONONEN, Finex Oy (Ltd), Kotka, Finland Evaluation of Tri-BGREG VERO, Orica Watercare Inc., Oxford, NC Evaluation of Tri-BSHIRISH NAIK, S. V. MOKASHI, SUJATA KULKARNI, KIRAN DESHPANDE, Thermax Ltd, Pune, India Structural Design ROBERT L. ALBRIGHT, Ph.D, Albright Consulting, Churchville, PA Structural Design EDWARD ROSENBERG, MARK HUGHES, JESSICA WOOD, Department of Chemistry, University of Mont Selective Ion Exc FRANCIS BOODOO, The Purolite Company, Bala Cynwyd, PA

Selective Ion Exc STEFAN NEUMANN, Ph.D, Lanxess Deutschland GmbH, Leverkusen, Germany; PHIL FATULA Lanxess / S Treatment of WastROBERT SOLOMON, Ph.D, LANNY WEIMER, RCC Thermal Products, Division of GE Water, Ellicott City, M Treatment of WastWILLIAM A. SHAW, PE, Veolia Water Solutions & Technologies Company, Pewaukee, WI Controlling ChemisROBERT D. BARTHOLOMEW, Shepherd T. Powell & Associates, Baltimore, MD Controlling ChemisKATHI KIRSCHENHEITER, MICHAEL CHUK, COLLEEN LAYMAN, KUMAR SINHA, Bechtel Power Corpor Increased Water TrARUN MITTAL, Aquatech International Corporation, Canonsburg, PA Increased Water TrRUSI KAPADIA, MICHAEL SHEEDY, DONALD SWAINE, Eco-Tec Inc., Pickering, ON, Canada, PATRICIA M ZLD: New Silica BaDAN DUKE, Water Conservation Technology International, Inc., Temecula, CA Leading Chemical JAMES M MOEN, Roseville Electric, Roseville, CA Leading Chemical AJANTA M SARKAR, SACHIN KUKADE, Aquatech Systems (Asia) Pvt. Limited, Pune, India Achieving Zero Bl WILLIAM A. SHAW, PE, HPD LLC, Pewaukee, WI Achieving Zero Bl SAM R. OWENS, RICK H. MAXEY, CHEMICO International, Inc., Corpus Christi, TX Application Of Ad ENOS L. STOVER, PH.D,; The Stover Group, Stillwater, OK Application Of Ad ALI R. AHMADI MOTLAGH, JABIN K. JOSEPH, TIMOTHY M. LAPARA, MICHAEL J.SEMMENS, Departme Treatment Efficien PAUL TOGNA, Ph.D., Shaw Environmental Inc., Lawrenceville, NJ Treatment Efficien ARI KETONEN, Eimco Water Technologies, Salt Lake City, UT; M. KIERIKKA, P. PAJUNIEMI, Eimco Water Enhanced CalciumSUN-JIP KIM, JIN-YOUNG PARK, WON-KWON LEE, G&G Co., Ltd., Suwon, Korea, YONG-WOO LEE, JA Side-By-Side Perf MICHAEL C. PRESTON, Black & Veatch Corporation, Overland Park, KS Side-By-Side Perf SCOTT S. BEARDSLEY, CRAIG R. GRANLUND, Dow Water Solutions, Edina, MN High-Efficiency Fi JASON P. FUES, JANE KUCERA, Nalco Company, Naperville, IL Synthesis and Char JONATHAN WOOD, USFilter, Lowell, MA Synthesis and Char ANIL KUMAR, SONNY SACHDEVA, Department of Chemical Engineering, Indian Institute of Technology, K Application of Act GARY L. HATCH, Ph.D., TARA KOELE, Pentair Filtration, Inc., Sheboygan, WI Experiences AssocPanel Moderator: Edward (Ted) Beardwood, Ashland Chemical Drew Division, Ajax, Ontario, Canada Legionella; Panel Panel Moderator: David Alley, Clearwater Systems Corp., Essex, CT None Dr. WILLIAM H. JOYCE, CEO Nalco Company, Naperville, IL ZLD: General TopicDANIEL BJORKLUND, AquaChem ICD, Intellectual ProperKEITH G. HADDAWAY, Ph.D., Esq., CLIFTON E. McCANN, Esq., Venable LLP, Washington, DC Business ContinuitMICHAEL SMITH, ReadySmith, Inc., Oakville, ON, Canada Water Consumption JOHN GREENWALD, Mechanical Operations Co., Inc. & BOMA, Pittsburgh, PA Water, Water Everyw ERIC J. BECKMAN, Ph.D., University of Pittsburgh, Pittsburgh, PA Environmental Effec CHRISTOPHER J. NALEPA, Ph.D., Albemarle Corp., Baton Rouge, LA Environmental Effec RUDOLF C. MATOUSEK, Severn Trent Services, Sugar Land, TX; DAVID W. HILL, Severn Trent De Nora L Opportunities and RAYMOND M. POST, P.E., GE Water & Process Technologies, Trevose, PA Opportunities and ANTON G. CALLERY, ITT Advanced Water Treatment/Portacel, Inc., The Woodlands, TX Simultaneous CleaARTHUR J. FREEDMAN, Ph.D., Arthur Freedman Associates, Inc., East Stroudsburg, PA Simultaneous CleaJASBIR S. GILL, Ph.D., AMIT GUPTA, Nalco Company, Naperville, IL Enhanced Tube Fou JAY FARMERIE, Cyrus Rice Water Consultants, Inc., Pittsburgh, PA Enhanced Tube Fou EDWARD S. BEARDWOOD, Ashland Water Technologies, Ajax, ON, Canada Enhanced Tube Fou EDWARD S. BEARDWOOD, Ashland Water Technologies, Ajax, ON, Canada; GEORGE F. HAYS, STEVEN Analytical and PreHAKAN GURLEYUK, Ph.D., RUSS GERARDS Applied Speciation and Consulting, LLC, Tukwila, WA New European Regu LAWRENCE M. GURNARI, N.A. Water Systems, a Veolia Water Solutions & Technologies company, Moon Mercury SeparatioJOE LALLY, BOB GEC, Degussa Corporation, Parsippany, NJ; RUEDIGER PELDSZUS, Degussa AG Mixed Bed Resin SPETER MEYERS, FRANK DESILVA ResinTech, Inc., West Berlin, NJ Unusual Items andCLAUDE GAUTHIER, P.Eng., The Purolite Company, Kitchener, ON, Canada Unusual Items andGEORGE J. CRITS, Idreco, USA, West Chester, PA New Procedure forWILLIAM E. BORNAK, Recirculation Technologies Inc., Warminster, PA Condensate PolishKENNETH FREDERICK, Ion Exchange Associates, LLC, Smithville, WV Condensate PolishROBERT A. APPLEGATE, MICHAEL J. O’BRIEN, Graver Water Systems, LLC, Cranford, NJ Maximizing Membra JOHN CLARK, Chemtrac Systems, Inc., Norcross, GA Design, Construct CHRISTINA HO, Consolidated Edison, New York, NY; JONATHAN WOOD, Siemens Water Technologies, Io

Reverse Osmosis T MICHAEL C. PRESTON, Black & Veatch Corporation, Overland Park, KS Reverse Osmosis T JANE KUCERA, Nalco Company, Naperville, IL Reverse Osmosis T JANE KUCERA, Nalco Company, Naperville, IL Measurement of IrCHARLES KUHFELDT, Ashland Water Technologies, Drew Industrial, Boonton, NJ Failure Analysis C DAVID KOTWICA, GE Water & Process Techologies, The Woodlands, TX Chemistry ChallenDAVID G. DANIELS, Mechanical & Materials Engineering, LLP, Austin, TX Designing HRSG’sLEWIS R. DOUGLAS, PE, JOSEPH E. SCHROEDER, Nooter Eriksen, Fenton, MO Metal Removal andPHILIP W. FATULA, Sybron Chemicals Inc., a Lanxess Company, Pittsburgh, PA Metal Removal andPAUL PAJUNEN, P.Eng., MICHAEL SHEEDY, P.Eng., Eco-Tec Inc., Pickering, ON, Canada Performance ImproNACEUR JEMAA, MICHAEL PALEOLOGOU, Pulp And Paper Research Institute Of Canada, Pointe Clair, Q Performance ImproEDWARD ROSENBERG, Ph.D., CAROLYN HART, MARK HUGHES, VARADHARAJAN KAILASAM, JESSE Towards a Resin-inCRAIG J. BROWN, P.Eng., Chemionex, Pickering, ON, Canada Towards a Resin-inBEREND WASSINK, MARVIN NEUFELD & DAVID DREISINGER University of British Columbia – Dept. of M Towards a Resin-in BEREND WASSINK, MARVIN NEUFELD, DAVID DREISINGER University Experience and Use of Oxidation Reduction Potential (ORP) Measurements in Power Plant of British Columbia – Dept. of M Applications STEPHEN J. SHULDER, Constellation Energy, MD Online IonConductivity Chromat BEVERLY NEWTON, Dionex Corporation, Sunnyvale, CA Mississaugua, Is Cation LUISMonitoring CARVALHO, Relevant PE,, GE ForWater Today’s & Process Combined Technologies, Cycle Power Plant? – Yet ON, Canada; THOMAS JAMES, Another Case Study I Says It Is Not. A Comphensive D L AVID M. GRAY, Mettler-Toledo Thornton, Inc., Bedford, MA Volatilization of LAWRENCE OWOPUTI, Ph.D., P.E., JERRY PENLAND, JOHN BALEWSKI, P.E., ATS-Chester Engineers, Innovated Bio-ecolGODEFROID BUKURU, YANG JIAN, Tongji University, Shanghai, China Submerged Membran BALAKRISHNAN VISWANATH, TAO GUIHE, KIRAN KEKRE, Centre for Advanced Water Technology, Sing Department of Ene SARA M. PLETCHER, BARBARA A. CARNEY, Cooling SARAHSystems FORBES, U.S. Department of Energy – National En Water (Resource) Conservation Using Closed-Loop, Evaporative For Process and Power Applications PETER G. DEMAKOS, Niagara Blower Co., Buffalo, NY Water Recovery fr ESAM ELSARRAG, Technical Studies Institute, Abu Dhabi, UAE Innovative Membran ERIK W. NOTTLESON, Inima USA, Brockton, MA A New High Perform EMMANUEL DOUSSIERE, Aquasource, Toulouse, France Sand Ballasted HigCHRIS HOWELL, Crown Solution, Inc., Vandalia, OH Sand Ballasted HigCHARLES D. BLUMENSCHEIN, P.E., DEE, ERICA LATKER, KASHI BANERJEE, N.A. Water Systems, a V Effective Monitor ROBERT HOLLOWAY, Holloway Associates, Etobicoke, ON, Canada Two Stage Clarifi ROBERT APPLEGATE, Graver Water Systems, LLC, Cranford, NJ Two Stage Clarifi RICHARD R. ROSS, PE, Siemens Water Technologies, Chalfont, PA, ERIC A. LAWRENCE, Siemens Wate Water Reuse in AlbCAROLINE WILSON MUSSBACHER, EnCana Oil and Gas Partnership, Calgary, AB, Canada, GORDON P Water Reuse in AlbRAFAEL GAY-DE-MONTELLA, SHULAMIT KULTNER, Colt Engineering Corporation, Calgary, AB, Canada Fresh, Produced, MELONIE MYSZCZYSZYN, Canadian Natural Resources Limited, Bonnyville, AB, Canada High Efficiency DeMICHAEL K. BRIDLE, WorleyParsons MEG Ltd., Calgary, Alberta, Canada Desalination of P CAROLINE WILSON MUSSBACHER, Encana Oil & Gas Partnership, Calgary, AB, Canada Desalination of P LNSP NAGGHAPPAN, N.A.Water Systems, a Veolia Water Solutions & Technology Company, Pittsburgh, P ZLD Systems for PMICHAEL PRESTON,Data Black & Veatch Corporation, Overland Park,Using KS Start-up, Commissioning, andC.Operational From the World’s First SAGD Facilities Evaporators to Treat LANNY Produced WEIMER, WaterWILLIAM for BoilerHEINS, Feedwater GE Water & Process Technologies, Bellevue, WA FGD Waste Water DEVESH B MITTAL, JACK HOSKIN, Aquatech International Corporation, Canonsburg, PA Evaluation of Scal JESUS MARIN-CRUZ, ARQUEMEDES ESTRADA MARTINEZ, Instituto Mexicano del Petróleo, C.P., Mexic Prediction and MonDAVID SCHLOTTENMIER, Bridger Scientific, Sagamore Beach, MA Prediction and MonEDWARD S. BEARDWOOD, Ashland Water Technologies, Toronto,ON., Canada Prediction and MonDOUGLAS B. DEWITT-DICK, Ashland Specialty Chemical Company, Portland, TX; EDWARD S. BEARDWO On-Line Analysis SAL ESPOSITO, GE Water And Process Technologies, Trevose, PA On-Line Analysis JOHN RICHARDSON, RICHARD H. TRIBBLE, RICHARD E. GEISLER, DALE P. STUART, ChemTreat, Inc Effect of pH and SMICHAEL GOTTLIEB, PETER MEYERS, ResinTech, Inc., West Berlin, NJ Calcium Removal ALLEN L. FARBER, Bechtel National, Frederick, MD Calcium Removal PETER MEYERS, ResinTech, Inc., Inc., West Berlin, NJ Multi-Parameter AnCOLLEEN M. LAYMAN, P.E., Bechtel Power Corporation, Frederick, MD Multi-Parameter AnSCOTT CROSIER, Hach Company, Fairport, NY

Rapid Legionella aGEORGE ASPREY, Microbiological Labs, Streetsboro, OH; PAUL PUCKORIUS, Puckorius & Associates, In Sampling and Analy DOUGLAS TOAL, Aerotech P&K, Cherry Hill, NJ Legionella and BioCHARLES ASCOLESE, GE Water & Process Technologies, Trevose, PA Legionella and BioMARTIN NOSOWITZ, ARKEMA Inc., King of Prussia, PA, PAUL PUCKORIUS, Puckorius & Associates, Inc Legionella and BioMARTIN NOSOWITZ, ARKEMA Inc., King of Prussia, PA, PAUL PUCKORIUS, Puckorius & Associates, Inc The Legionella ShiWILLIAM F. McCOY, Ph.D., Phigenics, Inc., Naperville, IL The Legionella ShiMALCOLM TURVEY, Ashland Netherlands, B.V., Drew Industrial, Barendrecht, The Netherlands; JOANNE Start-ups and Cyc Moderator: Edward Beardwood, Ashland Chemical Drew Division, Ajax, Ontario, Controlling Water Moderator: Deborah Bloom, Nalco Company, Naperville, IL, None JEFFREY HOFFMAN, USA. Department of Energy, National Testing Laboratories High Rate Combine JERRY PHIPPS, P.E., USFilter, Ames, IA, GLEN SUNDSTROM, USFilter, Rockford, IL A New Easy, Accura BONNIE L. HARRIS, Nalco Company, Burlington, Ontario, Canada, ROGER W. FOWEE, Nalco Company, Boiler Feed Water WILLIAM S. MILLER, GE Mobile Water, Inc., Norfolk, VA Boiler Feed Water BRIAN MILLER, JORGE MUNOZ, FRED WIESLER, Membrana – Industrial Separations, Charlotte, NC Outsourced Water RUSSELL HOURIGAN, GABRIEL NICOLAIDES, Ontario Power Generation, Pickering, Ontario, Canada, M A Triple MembraneALLISON M. MOODY, GE Mobile Water, Norfolk, VA Removal of WeaklyDAVID F. TESSIER, Ph.D., GE Water & Process Technologies, Guelph, Ontario, Canada Removal of WeaklyAVIJIT DEY, Ph.D., Omexell, Inc., Houston, TX Microfiltration P JOHN POTTS, Kimley-Horn and Associates, Inc., West Palm Beach, FL Microfiltration P GLEN P. SUNDSTROM, USFilter Memcor Products, Rockford, IL, RUSS SWERDFEGER, USFilter Memco Microfiltration P GLEN P. SUNDSTROM, USFilter Memcor Products, Rockford, IL, RUSS SWERDFEGER, USFilter Memco Membrane Technolo PETER S. CARTWRIGHT, P.E., Cartwright Consulting Company, Minneapolis, MN Selenium Reduction MAREK MIERZEJEWSKI, USFilter, a Siemens Business, Glen Allen, VA Selenium Reduction TERRY SCHEURMAN, Applied Specialties, Inc., Avon Lake, OH ABMet Biological DONALD VACKER, Bechtel Corporation, Houston, TX ABMet Biological JILL SONSTEGARD, TIM PICKETT, Zenon Environmental, Salt Lake City, UT Constructed WetlaF. DOUGLAS MOONEY, ENTRIX, Inc., Atlanta, GA, ROBERT R. WYLIE, Duke Energy Corp., Charlotte, NC Designing Construc MARK A. JANICK, Bechtel Power Corporation, Frederick, MD Designing Construc JOHN H. RODGERS, Jr., Ph.D.., Clemson University, Clemson, SC, GEORGE M. HUDDLESTON, Entrix, I Operating ExperieMICHAEL L. PUDVAY, Infilco Degremont, Richmond, VA Continuous Real TiARTHUR J. FREEDMAN, PH.D., Arthur Freedman Associates, Inc., East Stroudsburg, PA Continuous Real TiROBERT GOLDSTEIN, Technical Associates, Canoga Park, CA, DANIEL DOTSON, Jefferson UV Disinfection EqKEITH BIRCHER, ELLIOTT WHITBY, SCOTT FRENZ, Calgon Carbon Corporation – UV Technologies Divis MIEX Advanced DOC WAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA MIEX Advanced DOC PAUL SMITH, MICHAEL BOURKE, DAVID SHELBACH, S. MITCHELL, SHANE JONES, Orica Watercare I CARIX – 20 Years ALFRED J. NICKELS, Watertech Inc., Twin Falls, ID CARIX – 20 Years WOLFGANG H. HOLL, Ph.D., Forschungszentrum Karlsruhe, Karlsruhe, Germany, KLAUS HAGEN, Krüge Legionella 2003: HELEN CERRA, Chemtreat, Inc., Glen Allen, VA Legionella 2003: WILLIAM PEARSON, II, Southeastern Laboratories, Goldsboro, NC Guidelines for Tr PAUL R. PUCKORIUS, Puckorius & Associates, Evergreen, CO Guidelines for Tr J. PATRICK SISK, JAY FARMERIE, CHARLES D. HAMRICK, CWT White Rust: An IndJASBIR S. GILL, Ph.D., Nalco Company, Naperville, IL White Rust: An IndROBERT LEE, R2J Chemical Services, Inc., Largo, FL, GARY M. REGGIANI, New Paradigm ForJAMES BELLOWS, PH.D.., Siemens Power Generation, Orlando, FL, MICHAEL RZIHA, Siemens Power G On-line Strategy f TOM PIKE, Western Farmers Electric Cooperative, Fort Towson, OK, DOUGLAS DEWITT-DICK, Ashland S Origin and BehavioSTEFAN A. HUBER, Ph.D., DOC-LABOR Dr. Huber, Karlsruhe, Germany Corrosion Steam TDAVID G. DANIELS, Mechanical & Materials Engineering, LLP, Austin, TX Particle Size Anal JIM SUMMERFIELD, The Dow Chemical Company, Midland, MI Unusual Items andMICHAEL HALDEMAN, Tenergy Christ Water, LLC, New Britain, CT Unusual Items andGEORGE J. CRITS, Idreco USA, Corp., Ardmore, PA Sixteen Years of RWAYNE BERNAHL, W. Bernahl Enterprises Ltd., Elmhurst, IL

Sixteen Years of RWILLIAM E. BORNAK, Recirculation Technologies, Inc., Warminster, PA Case Study of AnaROBERT HICKEY,MICHAEL Ecovation,DELVECCHIO, Inc., Victor, NYShaw Environmental, Inc, Lawrenceville, NJ,. TODD WEBSTE A. PAUL TOGNA, Water Quality and BOB GARYHINES, A. LORETITSCH, PAUL R. PUCKORIUS, & Associates, Treating High Concentrations of USFilter, Perchlorate Envirex in Groundwater Products, Waukesha, Using aPuckorius Two-Stage Wisconsin, Advanced Inc., Evergreen, CO Bioreactor Design MARY CHEUNG, Veolia Water North America Operating Services, LLC, Solon, Ohio, L. KEITH BAILEY, Ke Performance of Mul M. N. RAO, Aquatech International Corporation, Canonsburg, PA Performance of Mul Dr. C. H. KRISHNAMURTHI RAO, T. JAYACHANDER, Membrane Research Technology Singapore PTE LT Performance of Mul Dr. C. H. KRISHNAMURTHI RAO, T. JAYACHANDER, Membrane Research Technology Singapore PTE LT Innovative MBR PrGOMES GANAPATHI, Bechtel International Systems Inc., Amman, Jordan Innovative MBR PrALASDAIR DONN, ITT Aquious, Basingstoke, Hants, UK, MICHAEL DIMITRIOU, ITT Advanced Cooling Tower FilmDOUGLAS DEWITT-DICK, Ashland Specialty Chemical, Boonton, NJ Cooling Tower FilmJOHN A. ZIMOWSKI, Dupont Co., Orange, TX Cooling Tower DiagCHARLES W. H. FOSTER, Diagnostic Cooling Solutions Inc., Burlington, Ontario, Canada Dynamic Control oDANIEL M. CICERO, Nalco Company, Naperville, IL Water (Resource) JAMES C. DROMGOOLE, Fort Bend Services, Stafford, TX Water (Resource) PETER G. DEMAKOS, Niagara Company, NYusing a Combination of Removal of Recalcitrant Organic Compounds from Blower Stabilized Landfill Buffalo, Leachate Ammonium Stripping WILLIAM and GAC S. MILLER, Adsorption GE Mobile Water, Inc., Norfolk, VA Removal of Recalcitrant Organic Compounds from Stabilized Landfill Leachate using a Combination of Ammonium Stripping TONNI and AGUSTIONO GAC Adsorption KURNIAWAN, GILBERT Y. S. CHAN, WAI-HUNG LO, The Hong Kong Polytechnic Un The Ion ExchangeLARRY GOTTLIEB, ResinTech, West Berlin, NJ Use of Hydrolysis ALLAN FARBER, CHRISTINE TEVIS, Bechtel National Inc., Gunpowder, MD, CRAIG HATFIELD, Battelle M Silica Polyamine MICHAEL GOTTLIEB, ResinTech, West Berlin, NJ Silica Polyamine EDWARD ROSENBERG, DAN NIELSEN, PAUL MIRANDA, University of Montana, Missoula, MT, CAROLY Solar Based Distil M. UMAMAHESWARAN, Anna University, College of Engineering, Chennai, India The Island Water ABRANDON HENKE, STEVEN DOVER, PHILIP NOE, The Island Water Association, Inc., Sanibel, FL Pretreatment for MARK A. JANICK, Bechtel Power Corporation, Frederick, MD Pretreatment for KENNETH IRWIN, Ph.D., GE Water & Process Technologies, Trevose, PA, IAN RAMROOP, Desalination C Performance Charac JAMES F. KLAUSNER, YI LI, RENWEI MEI, University of Florida, Gainesville, FL Overview of DesalMICHAEL C. PRESTON, Black & Veatch, Overland Park, KS Drawing the Line: CLIFTON E. McCANN, ESQ., Venable LLP, Washington, DC Short-Bed Ion-Exch DANIEL B. RICE, DR. DARYL GISCH, The Dow Chemical Company, Midland, MI Short-Bed Ion-Exch MICHAEL PALEOLOGOU, NACEUR JEMAA, RICHARDPrinciple: BERRY, Two PulpCase and Paper Research Institute of Can Value Added Products from Innovative Applications of Donnan Membrane Studies ROBERT L. ALBRIGHT, PH.D.., Albright Consulting, Southampton, PA Case Value Added Products from Innovative Applications of Donnan Membrane Principle: Two Studies ARUP SENGUPTA, Ph.D.., LUIS CUMBAL, PRAKHAR PRAKASH, Lehigh University, Bethlehem, PA Hybrid Resin TechHEIKKI MONONEN, Finex Oy, Kotka, Finland Hybrid Resin TechPETER MEYERS, MICHAEL GOTTLIEB, ResinTech, Inc., West Berlin, NJ Capacitive DeionizJONATHAN WOOD, USFilter, Lowell, MA Capacitive DeionizTOBIE J. WELGEMOED, P.E., CDT Systems, Inc., Dallas, TX The Use of Heat TSCOTT BEARDSLEY, The Dow Chemical Company, Minneapolis, MN The Use of Heat TJONATHAN HUNT, GE Water & Process Technologies, Gambs UK The Use of Heat TJONATHAN HUNT, SIMON GARE, GE Water & Process Technologies, Gambs UK, MATTHEW WHITE, GE New Oxygen Scave FEDERICA PERPIGLIA, Università degli Studi di Genova – D.I.M.S.E.T., Genova, Italy Pilot Testing of I JINGDONG ZHANG, SUNMING GAO, YANWEI LIU, CHANGZHENG LI, FANG ZHANG, HUIMING ZEN, C Solving Tube FailuANTON BANWEG, Nalco Company, Naperville, IL Solving Tube FailuFELIX TAMPUBOLON, PE., MUHAIMIN, PE., PT Badak NGL, Bontang, East Kalimantan, Indonesia Benefits of Oil an LARRY SMITH, GE Infrastructure Water & Process Technologies, Trevose, PA Benefits of Oil an NORMAN D. FAHRER, ChemTreat, Inc., Glen Allen, VA, JEFF BRAUN, RUSSELL McLAUGHLIN, Nucor S Case History of th SAEED DARIAN, Connestoga-Rovers & Associates, Atlanta, GA Case History of th DON VACKER, DAVID DYBELL, Bechtel Corporation, Houston, TX Case History of th DON VACKER, DAVID DYBELL, Bechtel Corporation, Houston, TX Assuring Mill Reli VIRGINIA DURHAM, Hercules Incorporated, Philadelphia, PA Assuring Learned Mill Reli from THOMAS W.Outbreaks WOLFE, Wolfe Water Treatment Services, LLC, Baton Rouge,inLA, JAMES J. KEANE, NewPa Lessons Recent of Legionnaires Disease and Guidelines for Control Cooling Water PAUL PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO

Legionella ControlDAVID ALLEY, FRANK ROGERS, Clearwater Systems Corp., Essex, CT Zen and the Art o DAVID CHARLES ASCOLESE, GE Infrastructure Water & Process Technologies, Trevose, PA W. HILL, Severn Trent De Nora LLC, Sugar Land, TX, Environmental Effec RUDOLF C. MATOUSEK, Severn Trent Services, Sugar Land, TX, R.P. HERWIG, Ph.D. School of Aquatic Particle Removal EFRED TEPPER, LEONID KALEDIN, Argonide Corporation, Sanford, FL Design and OperatPETER MIDGLEY, Anderson Water Systems, Dundas, Ontario, Canada Design and OperatBENJAMIN T. ANTRIM, CHINH HOANG, Koch Membrane Systems, Inc., Wilmington, MA Sensitive Integrit STUART A. McCLELLAN, Successfully Applied Membranes, Inc., Palm Beach Gardens, FL Sensitive Integrit STEVEN D. JONS, Ph.D., TOM N. HAYNES, JAMES S. NATHAN, FilmTec Corporation, a subsidiary of The Membrane FiltratioJANE KUCERA, Nalco Company, Naperville, IL Membrane FiltratioGLEN P. SUNDSTROM, USFilter Memcor Products, Rockford, IL, DEAN M. WEYENBERG, Nuclear Manag Membrane FiltratioGLEN P. SUNDSTROM, USFilter Memcor Products, Rockford, IL, DEAN M. WEYENBERG, Nuclear Mana An Alternative to SROBERT L BRYANT, Chemtrac Systems, Inc., Norcross, GA Providing Water toVIRAJ DE SILVA, Ph.D., P.E., BCEE, WALID HATOUM, P.E., PARSONS, Tampa, FL Challenges in Rebui MOUSTAFA HASAN, Metito (Overseas) Ltd, Amman, Jordan, TAREK GHANDOUR, Metito International Inc Refurbishing Power ISAAC, PE, KATHRYN HYAM, PE, KUMAR SINHA, Power Corporation and Bechtel S Startup of Multiple JULIUS Water Treatment Plants & Operator Training in Southern IraqPE, – A Bechtel Rewarding Experience GOMES GANAPATHI, Ph.D., CHMM, LUKE SANTEE, JASSIM AJEEL, ALI GHAREEB, OSAMA ABD AL KA ZLD Today – MatchWILLIAM E. MOORE, Calpine, Houston, TX, TIMOTHY RITTOF, HPD LLC, Plainfield, IL Selection of Zero KUMAR SINHA, Bechtel, Frederick, MD Selection of Zero CHRIS LEWIS, Burbank Water & Power, City of Burbank, CA, DEVESH MITTAL, Aquatech International Co The Bayswater-LidWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA The Bayswater-LidTHOMAS Wagner PPI, Callaghan, NSW, Australia, Supercritical Carbon Dioxide BRYANT, ExtractionConnell of Leachable Organics from IX Resins – A Novel ERROL Resin BURTON, Macquarie Genera Cleaning Technique WILLIAM E. BORNAK, RTI, Warminster, PA Supercritical Carbon Dioxide Extraction of Leachable Organics from IX Resins – A Novel Resin Cleaning Technique SHELTON A. DIAS, Ph.D, Kinectrics, Inc., Toronto, Ontario, Canada Use of Fractal LiquCRAIG BROWN, Eco-Tec Inc., Pickering, Ontario, Canada Use of Fractal LiquJAY MIERS, Jr., Rohm and Haas Company, Philadelphia, PA Optimizing Single-FRANK CRAFT, Mobile Processing Technology, Memphis, TN Optimizing Single-PETER MEYERS, FRANCIS DESILVA, ResinTech, Inc., West Berlin, NJ Impact of Low MolJIM SUMMERFIELD, The Dow Chemical Company, Midland, MI Advances in ProduKUMAR JAIN, Fluor Corporation, Calgary, Alberta, Canada Advances in ProduM. J. PLEBON, MARC SAAD, SERGE FRASER, EARTH (Canada) Corporation, Montreal, Quebec, Canad Advances in ProduM. J. PLEBON, MARC SAAD, SERGE FRASER, EARTH (Canada) Corporation, Montreal, Quebec, Canad Study on RecyclingGORDON PAGE, Page Technology Ltd., Calgary, Alberta, Canada, CAROLINE WILSON, Oil Recovery, En Study on RecyclingJIACAI XIE, XILIN LIU, Petrochina Liaohe Oilfield Company, Panjin, China Corrrosion and Dep Panel Moderator: Allan Harvey, Ph.D., National Institute of Standards & Technology, Boulder, CO Water Scarcity Solu JEFFREY CONNELLY, Vice President & General Manager, GE Water & Process Technologies Effective Dust SupJOEY S. SWART, Water & Environmental Research, Sasol Technology R&D, Sasol, South Africa Introduction of Sp JONATHAN WOOD, USFilter, Lowell, MA Introduction of Sp AVIJIT DEY, Ph.D., Omexell, Inc., Houston, TX Introduction of Sp AVIJIT DEY, Ph.D., GUANGHUI LI, Omexell, Inc., Houston, TX EDI Performance: C OHERYL G. SAWYER, Cogentrix, Suffolk, VA EDI Performance: W O ILLIAM T. HARVEY, TED PRATO, Ionics Incorporated, Watertown, MA Process and SysteJONATHAN WOOD, JOSEPH GIFFORD, USFilter, Lowell, MA Risk Mitigation by SUSAN BAINES, KATHI KIRSCHENHEITER, KUMAR SINHA, Bechtel Power Corporation, Frederick, MD Fear and LoathingBEVERLY NEWTON, MIKE DOYLE, Dionex Corporation, Sunnyvale, CA, LUIS CARVALHO, IAN SCARTH pH and CO2 Determ DAVID M. GRAY, Mettler-Toledo Thornton, Inc., Bedford, MA Total Oxidizable TUNG-JEN WEN, Institute of Nuclear Energy Research, Taoyuan, Taiwan, ROC Application of MemJANTJE JOHNSON, FilmTec Corporation, Edina, MN Application of MemBENJAMIN T. ANTRIM, BRIANtoM. KILCULLEN, Membrane Systems, Wilmington, MA The Use of Reverse Osmosis as a Pretreatment a High PressureKoch Steam System at a PowerInc., Plant in Thailand MICHAEL C. PRESTON, Black & Veatch Corporation., Overland Park, KS The Use of Reverse Osmosis as a Pretreatment to a High Pressure Steam System at a Power Plant in Thailand MATTHEW WHITE, Ecolochem International, Ltd., Peterborough, UK, MIKE DUNHAM, Glow Power Co. Ltd Reverse Osmosis KENNETH J. KOZELSKI, E.I. DuPont de Nemours, Camden, SC

Reverse Osmosis JANE KUCERA, Nalco Company, Naperville, IL Application of a N ALBERT BURSIK, PowerPlant Chemistry GmbH, Neulussheim, Germany Use of Amines in WILLIAM MOORE, Calpine, Houston, TX Hazards of Amine JAMES C. BELLOWS, Ph.D., Siemens Westinghouse Power Corporation, Orlando, FL Amine Use in HighMICHAEL W. ROOTHAM, Mike Rootham & Associates, Delmont, PA Emerging Contamina MARK V. ROWZEE, Water Quality Association, Lisle, IL I Hear What You’reJOHN D. MORTON, P.E., ALCOA, Inc., Pittsburgh, PA Technology BasedDEBORAH NAGLE, US EPA – Permits Division, Washington, DC Modeling LeachingPAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO Modeling LeachingMARK A. JANICK, LAWRENCE GASPER, Bechtel Power Corporation, Frederick, MD Design & OperatioDOUGLAS B. DEWITT-DICK, Ashland Specialty Chemical Co., Portland, TX, EDWARD S. BEARDWOOD, Case History of a TORRY J. TVEDT, T.J. Tvedt & Associates, Ltd, Angleton, TX Heat Recovery SteIRVIN J. COTTON, Arthur Freedman Associates, Inc., East Stroudsburg, PA, JOHN OBERMAIER, Deltak L Avoiding Costly WLUIS CARVALHO, P.Eng., GE Water & Process Technologies, Mississauga, Ontario, Canada HRSG Chemistry G DAVID G. DANIELS, Mechanical & Materials Engineering, LLC, Austin, TX Applications of I JOHN SCHUBERT, P.E., EnCoss, Monroeville, PA Applications of I JAMES W. HOTCHKIES, P.Eng., ZENON Environmental Inc., Oakville, Ontario, Canada Non-Chemical Devic EDWARD S. BEARDWOOD, Drew Industrial Division of Ashland Canada Corp., Ajax, Ontario, Canada Non-Chemical Devic TIMOTHY KEISTER, ProChemTech International, Inc., Brockway, PA On-Site Chlorine DLEONARD R. OLAVESSEN, Buckman Laboratories International, Inc., Memphis, TN On-Site Chlorine DMAREK NOWOSIELSKI, International Dioxcide, Inc., A Dupont Company, North Kingstown, RI Zebra Mussels in an RAYMOND M. POST, P.E. GE Water Process Technologies, Trevose, PA Zebra Mussels in an DAN BUTTS, ASI Group Ltd., Orchard Park, NY Demonstration of aJEFFERY TEAGUE, Refineries’ Indianapolis PowerTower and Light , Petersburg, IN, ROB LAWLER, TIMOTHY KEISTER, P Water Reuse Experience in Petroleum Cooling Water Systems – Lessons Learned R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO The Cost of Water PAUL in an Arid Climate: Water Conservation and Reuse -Source Segregation and Treatment JUSTIN R. MOSES, P.E., SCOTT A. GRIECO, PE, O’Brien & Gere Engineers, Inc., Syracuse, NY Legionnaires’ DiseBRIAN G. SHELTON, PathCon Laboratories, Norcross,forGA Chlorine Dioxide for Legionella Control inMPH, a Hospital Water System: Monitoring Disinfection By-Products ZHEDioxide ZHANG, D. VIDIC, Ph.D., Pittsburgh SchoolCare of Engineering, Pittsburgh, PA, C Evaluation of Chlorine inRADISAV Potable Water Systems for University LegionellaofControl in an- Acute Hospital Environment MARK HODGSON, Clayton Edison, NJ Evaluation of Chlorine Dioxide in Potable WaterEnvironmental, Systems for Legionella Control in an Acute Care Hospital Environment GREGORY TheWater JohnsSystems Hopkins for Hospital, Baltimore, PAUL SHARPE, Evaluation of Chlorine Dioxide BOVA, in Potable Legionella ControlMD, in an Acute Care Water Chemical Service, I Hospital Environment GREGORY BOVA, The Johns Hopkins Hospital, Baltimore, MD, PAUL SHARPE, Water Chemical Service, I Developing a JCAH MATTHEW R. FREIJE, HC Information Resources, Fallbrook, CA Developing a JCAH JAYIon FARMERIE, Rice Water Consultants, Pittsburgh, PA at WE The Use of Short Bed ExchangeCyrus Technology for the Production Inc., of High Purity Water Energies’ AL Prairie NEBRIG, Power Southern Plant Company Hoover, AL Purity Water at WE The Use ofPleasant Short Bed Ion Exchange Technology for Services, the Production of High Energies’ PleasantMICHAEL Prairie Power SHEEDY, Plant Eco-Tec Inc., Pickering, Ontario, Canada; PETER G. KUTZORA, We Energies, Fossil Hollow Fiber UF PiJERRY ALEXANDER, USFilter, La Canada, CA Hollow Fiber UF PiDAVE CHRISTOPHERSEN, CROWN Solutions, Inc., Vandalia, OH High Rate Compss MICHAEL C. PRESTON, Black & Veatch Corporation, Overland Park, KS High Rate Compss WILLIAM D. KUNZMAN, Schreiber LLC, Trussville, AL Dissolved Air FlotaMARK JANICK, Bechtel Power Corporation, Frederick, MD Dissolved Air FlotaWILLIAM E. HAAS, Ecolochem, Inc., An Ionics Company, Norfolk, VA; ADRIAAN VAN DER BEEK, Nijhuis W Phased Municipal LEE A. LUNDBERG, Veolia Water, Moon Township, PA Phased Municipal BUKURU GODEFROID, YANG JIAN , Tongji University, Shanghai, China Mobile WastewaterCOLLEEN M. KULICK, Parsons E & C, Reading, PA Mobile Wastewater BILL PERPICH USFilter, Tampa, FL;Lessons CHRIS Learned SOULE, from USFilter, Schaumburg, IL; SAM ZAMANI, P.E., Fl Cooling Tower Water Systems andJR., Legionnaires’ Disease: Recent USA IncidentsTower and Outbreaks CHRISTOPHER and the Corrective J. NALEPA, Actions Ph.D., That Albemarle Should Be Corporation, Taken from Baton Rouge, LA Cooling Water Systems and Legionnaires’ Disease: Lessons Learned Recent USA Incidents and Outbreaks PAUL R. and PUCKORIUS, the Corrective Puckorius Actions That & Associates, Should BeInc., Taken Evergreen, CO Risk Assessment: T ARTHUR J. FREEDMAN, Ph.D., Arthur Freedman Associates, Inc., East Stroudsburg, PA Risk Assessment: T ANDREW COOPER, HOWARD BARNES, NOEL CHRISTOPHER, THOMAS LINDLEY, ERIC MYERS, Nal Pros and Cons of APanel Moderator: Deborah Bloom, Nalco Company, Naperville, IL Current Ideas for Panel Moderator: Deborah Bloom, Nalco Company, Naperville, IL

None PETER J. CENSKY, Water Quality Association, (WQA), Lisle, IL Meeting Semicondu ROBERT A. ESCOBEDO, JUDY PUCKETT, Ecolochem, Inc., Norfolk, VA An Innovative ApprJ. C. DROMGOOLE, Fort Bend Services, Inc., Stafford, Texas, USA An Innovative ApprCHARLES D. BLUMENSCHEIN, P.E. DEE, KASHI BANERJEE, PH.D., P.E., DEE, MARK DI NARDO, ERIC Does Your DeaeratROBERT HOLLOWAY, Holloway Associates, Etobicoke, Ontario, Canada Does Your DeaeratJACK B. PRATT, SIP Consultants, Inc. – Consultant to Altair Equipment Co., Inc., Ft. Worth, TX, ORIN HOL Troubleshooting WDAVID G. DANIELS, M&M Engineering, Austin, TX Troubleshooting WMARK A.JANICK, PE., P. KUMAR SINHA, ALLAN FARBER, Bechtel Power Corporation, Frederick, MD 15 years of Crud, K. ANTHONY SELBY, Water Technology Consultants, Inc., Evergreen, CO 15 years of Crud, PHILIP K. HAZEN, Texas Municipal Power Agency, Bryan, TX, VANCE LUMME, Ondeo Nalco, Houston, TX Arsenic ReductionJAMES C. DROMGOOLE, Fort Bend Services, Inc., Stafford, TX Arsenic ReductionBILL SAULSBERRY, Ecolochem, Inc., Baytown, TX, RICHARD M. MODE, Ecolochem, Inc., St. Peters, MO THM Destruction bJASON P. BUFE, Ecolochem, Inc., East Hartford, CT Boron Removal witDONALD D. DOWNEY, The Purolite Company, Kitchener, Ontario, Canada, MICHAEL WUNSCH,Ondeo In Integrated Approa RONALD D. NEUFELD, PhD, University of Pittsburgh, Pittsburgh, PA Integrated Approa DAVE CISZEWSKI, Ionics RCC, Bellevue, WA Process Selection WAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA Process Selection CHARLES H. FRITZ, MICHAEL C. PRESTON, Black & Veatch Corporation, Overland Park, KS Process Selection CHARLES H. FRITZ, MICHAEL C. PRESTON, Black & Veatch Corporation, Overland Park, KS Nanofiltration Fol WILLIAM MOORE, Calpine, Houston, TX Nanofiltration Fol PETER K. ERIKSSON, GE Osmonics, Vista, CA, , WILLIAM K. TAYLOR, Saskferco Products, Inc., Belle Pl Field Experience JONATHAN WOOD, US Filter, Lowell, MA, ERIN WESTBERG, Alliant Energy, Cedar Rapids, IA, DENNIS Improvement in PeCHERYL G. SAWYER, Cogentrix, Suffolk, VA Improvement in PeTED PRATO, Ionics, Inc., Watertown, MA,, JEFF PURVIS, Hinds Energy, BRIAN AHEARN, Maine Independ Basis for CharacteJOHN H. BARBER, DAVID F. TESSIER, E-Cell Corporation, Guelph, Ontario, Canada Chemical CleaningMARK A. JANICK, P.E., Bechtel Power Corp., Frederick, MD Chemical CleaningMO D. MAJNOUNI, Aramco Services Company, Houston, TX, ARIF E. JAFFER, Baker Petrolite Corporatio From Raw Water Ana JAMES C. BELLOWS, Siemens Westinghouse, Orlando, FL From Raw Water Ana JULIUS ISAAC, LAWRENCE GASPER, KUMAR SINHA Bechtel Power Corporation, Frederick, MD The Application of OTAKAR JONAS, LEE MACHEMER, Jonas, Inc., Wilmington, DE The Application of PETER J. MILLETT, JOSEPH D. BATES, DENNIS F. HUSSEY, iSagacity, Half Moon Bay, CA Part per Billion D RICHARD W. PRESLEY, Hach Company, Loveland, Co Metallurgical ExamSTEPHEN M. McINTYRE, DOUGLAS B. DeWITT-DICK, DUNN JOSEPH HOFILENA, Ashland Specialty C Optimizing the Pe GLENN VICEVIC, Zenon Environmental Systems, Oakville, Ontario, Canada Optimizing the Pe RICHARD N. FRANKS, Hydranautics, Oceanside, CA Economy of Scale:PETER S. CARTWRIGHT, PE, Cartwright Consulting Co., Minneapolis, MN Economy of Scale:AMIT SENGUPTA, CHARLES J. RUNKLE, BETH A. KITTERINGHAM, Membrana-Charlotte, Charlotte, NC New Process For CRAIG J. BROWN, Eco-Tec, Inc., Pickering, JON H. XIAO, Andmir Environmental Hi-Tech Inc., Toronto, On Challenges in ManDAN ROBINETTE, CH2M Hill, Englewood, CO, RICH D’AMATO, CH2M Hill, Charlotte, NC, BOB HOLDEN, Advanced Environme LAWRENCE C. HALE, Drew Ameroid Singapore, Ashland Speciality Chemical Co, Jurong, Singapore Utilizing Mine WateKENNETH J. SEMMENS, DANIEL MILLER, West Virginia University, Morgantown, WV New Stable Biodegr MATTHIAS SCHWEINSBERG, W. HATER, J. VERDES, Henkel KGaA, Dusseldorf, Germany Chemical vs. Non-ARTHUR J. FREEDMAN, PhD., Arthur Freedman & Associates, Inc., East Stroudsburg, PA Chemical vs. Non-KEVIN A. KITZMAN, EDWARD F. MAZIARZ, Alcoa, Inc., Pittsburgh, PA, BOBBY J. PADGETT, Alcoa, Goos The On-line Remova GARRY LORETITSCH, Puckorius & Associates, Inc., Evergreen , CO The On-line Remova GEORGE J. MINCAR, Ashland Specialty Chemical, Boonton, NJ, BRIAN R. DAVIDSON, Ashland Specialty The On-line Remova GEORGE J. MINCAR, Ashland Specialty Chemical, Boonton, NJ, BRIAN R. DAVIDSON, Ashland Specialty Monitoring of Org BALAKRISHNAN VISWANATH, KIRAN KEKRE, GUIHE TAO, PhD., Centre for Advanced Water Technolog Commissioning Chal KUMAR JAIN, Fluor Daniel, Inc., Aliso Viejo, CA, ANGELO O. CANALES RUBIO, Agua Process, Veronica A Microfiltration Sy KUMAR SINHA, Bechtel Power Corp., Frederick, MD Microfiltration Sy GARRY CRAIG, Eraring Energy, Dora Creek, NSW, Australia

Eight Years of Se DON BOUDREAU, DAWN GUENDERT, USFilter, Plainfield, IL Eight Years of Se CHUCK DALE, Ionics Incorporated, San Jose, CA Utilizing Reverse JEFF TATE, Omexell, Inc., Hatfield, PA Utilizing Reverse SCOTT B. GORRY, Ecolochem, Inc., EastHartford, CT, JOHN E. KRISTENSEN, Entergy Nuclear, Buchana Cost Effective RO HARBANS KOHLI, US Filter Corporation, Signal Hill, CA Cost Effective RO JULIA E. NEMETH, PE, Harn R/O Systems, Inc., Venice, FL, THOMAS F. SEACORD, P.E., Carollo Enginee Pre-Commissioning EDWARD S. BEARDWOOD, Drew Canada Division of Ashland Canada Corp., Ajax, Ontario, Canada Amine Use in SteaMICHAEL W. ROOTHAM, Mike Rootham and Associates, Delmont, PA, J. C. BELLOWS, Siemens Westing Operating Guideli DAVID G. DANIELS, M&M Engineering, Austin, TX Sampling and MoniIRVIN J. COTTON, Arthur Freedman Associates, Stroudsburg, PA, ROGER LIGHT, The Dow Chemical Co. The Costs of Impr ROBERT HOLLOWAY, Holloway Associates, Etobicoke, Ontario, Canada Introduction to t JAMES O. ROBINSON, GE Betz, Trevose, PA The Use of PortablDAVE CHRISTOPHERSON, Crown Solutions, Dayton, OH The Use of PortablWILLIAM KOEBEL, ResinTech, Inc., West Berlin, NJ Membrane Technolo WAYNE E. BERNAHL, W Bernahl Enterprises Ltd, Elmhurst, IL Membrane Technolo PETER S. CARTWRIGHT, PE, Cartwright Consulting Co., Minneapolis, MN The Application o RICK MULLIGAN, Dow Chemical Canada, Inc., Sarnia, Ontario, Canada, STEPHEN NAJMY, The Dow Che Modernization ConDAN RICE, Dow Chemical Co., Midland, MI Modernization ConDAVID T. DALLY, Sybron Chemicals, Inc., A Bayer Co., Birmingham, NJ, PAUL W. GROSS, DARIN CUNNI RO Pretreatment wi GLENN P. VICEVIC, SUSAN GUIBERT, ZENON Environmental, Oakville, Ontario, Canada, RICKY BRYAN Ultrafiltration as RICHARD FRANKS, Hydranautics, Inc., Oceanside, CA Ultrafiltration as MERRILEE GALLOWAY, ANTONIA VON GOTTBERG, SCOTT WHITAKER, Ionics, Inc., Watertown. MA A Step-Change Imp PETER F. METCALFE, Toray Membrane America, San Diego, CA A Step-Change Imp DAVE PAULSON, MIKE MADSEN, JOE SZCZEPANSKI, BILL LAIDLAW, GE Osmonics, ASME Research Com Moderator :Richard Jacobsen, Bechtel BWXT Idaho, Idaho Falls, ID None JACK BOGUT, 1320-WJAS, Pittsburgh, PA An Integrated ApprWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA An Integrated ApprW. D. SAUNDERS, Sasol Technology (Pty) Ltd, Secunda, South Africa; S. E. WILKENS, Buckman Laborato An Integrated ApprW. D. SAUNDERS, Sasol Technology (Pty) Ltd, Secunda, South Africa; S. E. WILKENS, Buckman Laborato How a Power PlantSUSAN BAINES, Bechtel Power Corporation, Frederick, MD Identifying and Mi THOMAS W. WOLFE, Wolfe Water Treatment Services, LLC, Baton Rouge, LA Identifying and Mi ROBERT STRANDBERG, Covanta Energy, Bainbridge, PA, TERRY McCOY, ChemTreat, Inc., Lancaster, P Corrosion MonitoriTZU-YU CHEN, RODNEY H. BANKS, JEFFREY BRESHEARS, NARASIMHA M. RAO, STEVEN N. NICOL Proper Initial PassPAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO Chemical CleaningJAMES o E. FARMERIE, HERC Products, Inc., Wexford, PA Beneficial Use of KEVIN J. SHIELDS, R. BARRY DOOLEY, EPRI, Palo Alto, CA Process for CleaniJASBIR S. GILL, PhD, ONDEO Nalco Company, Naperville, IL Responding to BoiK. ANTHONY SELBY, Water Technology Consultants™, Inc., Evergreen, CO The Impact of Wate MEL J. ESMACHER, PE., JOHN F. FARRARO, PE, GE Betz, The Woodlands, TX A Multi-Parameter DAVID M. GRAY, Thornton/Mettler-Toledo, Waltham, MA Operating To Prev ROBERT HOLLOWAY, Holloway Associates, Etobicoke, Ontario, Canada Dissolved Oxygen RICHARD M. MODE, Ecolochem, Inc., St. Peters, MO; WILLIAM E. HAAS, Ecolochem, Inc., Norfolk, VA; W HERO™ Application PETER ERIKSSON, Osmonics, Vista, CA HERO™ Application NANDAN T. VANI, PE., DSc., Bechtel Corporation, Frederick, MD; JOEL KASPER, PE., Aquagenics Inc., W Electrochemical EvJESÚS MARÍN-CRUZ, ARQUIMEDES ESTRADA, Instituto Mexicano del Petróleo México City, DF, México; An Investigation ofJOHN P. COULTER, CESAR A. SILEBI, C. ALEXIS BENNETT, Lehigh University, Bethlehem, PA; PAUL Q. Non-chemical Water SAMUEL B. DILCER, Jr., Cyrus Rice Water Consultants, Pittsburgh, PA Non-chemical Water LORAINE A. HUCHLER, PE, MarTech Systems, Inc., Lawrenceville, NJ Organic Water TreJULIUS ISAAC, PE., KUMAR SINHA, PE, Bechtel Power Corporation, Frederick, MD Improved ChemicalMARK A. JANICK, PE, Bechtel Power Corporation, Frederick, MD Improved ChemicalKAJ RONDUM, KEITH FIELDING, Ashland Specialty Chemical Company, Boonton, NJ

New Analytical Te RICK DUNN, Ionics Instruments, Boulder, CO Microsand-BallasteBILL MOORE, Calpine Corporation, Houston, TX, WILLIAM SULLIVAN, USFilter, Cary, NC Treatment of Waste NARENDRA P. SHINKAR, Government Polytechnic, Nanded, Maharashtra, India, TAPAS NANDY, National Reuse of Secondary ROBERT P. HELWICK, USFilter, Moon Township, PA Combined Cycle AC BRIAN AYLAIAN, PE, Bechtel Power Corporation, Frederick, MD Propagation of RaiLOU KOENIG, Zinkan Enterprises, Twinsburg, OH, CHARLES K. BLANKENSHIP, PE, Duquesne Light Com Self-Cleaning Fil MARCUS N. ALLHANDS, PhD, PE, Amiad Filtration Systems, Oxnard, CA The Impact of MonE. H. K. ZEIHER, B. HO, B. ANDREWS, ONDEO Nalco Company, Naperville, IL Membranes Operati JIM SUMMERFIELD, The Dow Chemical Company, Midland, MI Alleviation of Pro PHILIP K. HAZEN, Texas Municipal Power Agency, Carlos, TX, J. C.DROMGOOLE, Fort Bend Services, In Ozone For CoolingRICHARD M. AHLGREN, Ahlgren Associates LLC, Waukesha, WI Ozone For CoolingDANIEL J. TIERNEY, Space Gateway Support, Kennedy Space Center, FL Biofilms that ContrTHOMAS K. WOOD, Univ. of Connecticut, Storrs, CT; BARRY C. SYRETT, Electric Power Research Institut Advanced Cooling GARY A. LORETITSCH, Puckorius & Associates, Inc., Evergreen, CO Advanced Cooling DANIEL A. MEIER, BARBARA E. MORIARTY, JEFFREY P. RASIMAS, DAVID L. STONECIPHER, ONDEO Development of PeEDWARD S. BEARDWOOD, Drew Industrial Division, Ashland Canada Corp., Ajax, Ontario, Canada; MAR Evaluating MicrobiWAYNE H. DICKINSON, Buckman Laboratories, Memphis, TN Innovative BusinesBOB GORGOL, JOHN SCHUBERT, STEVE HOPPER, USFilter Operating Services Water Treatment aPAUL R. BROWN, CPM., Texas Genco, LP, Houston, TX, MICHAEL D. BAILEY, Reliant Resources Corpora Plant Design ConsiRONALD MADDEN, Res-Kem Corp., Aston, PA Plant Design ConsiR. T. TAYLOR, Jr., Ecolochem, Inc., Norfolk, VA Copper Transport in KEVIN J. SHIELDS, R. BARRY DOOLEY, Electric Power Research Institute, Palo Alto, CA Condensate PolishiMICHAEL A. SADLER, , Bristol, Portishead, Bristol, UK Renovation or RepROBERT D. BARTHOLOMEW, DAVID A. CLINE, Jr., GARY H. ROBERTS, Sheppard T. Powell Associates, An Outline of the ALROY ASCHOFF, Electric Power Research Institute Consultant, BARRY DOOLEY, Electric Power Resea Performance Expec TERRENCE HELLER, The Purolite Company, Bala Cynwyd, PA Design Criteria fo JOSEPH F. GIANNELLI, Finetech Inc., Mountain Lakes, NJ Design Criteria fo STEFAN HILGER, Bayer AG, Leverkusen, Germany, PHILIP W. FATULA, Sybron Chemicals, Inc., a Bayer Four Years of OperPHILIP W. FATULA, Sybron Chemcials, Inc., a Bayer Company, Pittsburgh, PA Four Years of OperDONALD D. DOWNEY, The Purolite Company, Kitchener, Ontario, Canada Four Years of OperDONALD D. DOWNEY, The Purolite Company, Kitchener, Ontario, Canada Current Trends in VIJAY K. PURI, Ionex Water Treatment, Inc., Pittsburgh, PA Current Trends in WAYNE BERNAHL, W Bernahl Enterprises Ltd., Elmhurst, IL, CARL ROSSOW, ONDEO Nalco Company,N Current Trends in WAYNE BERNAHL, W Bernahl Enterprises Ltd., Elmhurst, IL, CARL ROSSOW, ONDEO Nalco Company,N Case Histories on DAVID F. GEARY, D. F. Geary Consultants, LLC, Annapolis, MD Case Histories on PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO An Effective Bioc RADISAV D. VIDIC, University of Pittsburgh, Department of Civil and Environmental Engineering, Pittsburgh An Effective Bioc G. A. GANZER, The Dow Chemical Company, Buckingham, PA; M. G. FREID, The Dow Chemical Compan Legionella TreatmeANDREW J. COOPER, ONDEO Nalco Company, Naperville, IL Legionella TreatmeLINDA RUSZNAK, KEVIN PIDANE, Ashland Specialty Chemical, Boonton, NJ Legionella TreatmeLINDA RUSZNAK, KEVIN PIDANE, JOANNE KUCHINSKI, Ashland Specialty Chemical, Boonton, NJ Practical Experie IRVIN J. COTTON, ARTHUR J. FREEDMAN, PhD, Arthur Freedman Associates, Inc., East Stroudsburg, PA Plant ConstructionDAVID M. SMYTH, MICHAEL RZIHA, LIAM McLOUGHLIN, Siemens Power Generation, Erlangen, German Chemical CleaningJOHN M. SULLIVAN, HydroChem, Inc., Youngstown, OH, JOHN McGRAW, HydroChem, Jacksonville, FL Pre-Operational ClKENNETH E. HANSEN, Babcock and Wilcox Company, Barberton, OH, JOHN M. JEVEC, McDermott Tech Pre-Operational CSEAN MACDONALD, CEDA Reactor, Ltd. Sherwood Park, Alberta, Canada, BRAD BUECKER, Consultant Development Of 14SALLY FISHER, Puricon Inc., Malvern, PA Development Of 14TAKESHI IZUMI, MASAHIRO HAGIWARA, TAKAO INO, TOSHI TAKAI, Ebara Corporation, Japan; MARVI Operation of PowdBRUCE A. LARKIN, Black & Veatch Corporation, Kansas City, MO Operation Condensate of PowdPHILLIP J. Performance D’ANGELO, Jordan Technologies, Glen Processing Mills, PA., J. STUART BROWN, Liquid Six, Inc., York, M Improved Resin Using Post-Manufacturing Systems PETER A. YARNELL, Graver Technologies, Inc., Glasgow, DE

New And EmergingRICHARD T D. MILLER, PhD, University of Louisville, Louisville, KY New And EmergingJANET T E. STOUT, PhD, VA Medical Center, Pittsburgh, PA Strategies For Eff CHRISTOPHER J. NALEPA, JONATHAN N. HOWARTH, Albemarle Corporation, Baton Rouge, LA Boiler Feedwater CPanel Moderator: James Robinson, GE Betz, Trevose, PA EPRI Boiler WaterPanel Moderator: MIKE ROOTHAM, Mike Rootham & Associates Commisioning Guid Panel Moderator: RICHARD T. JACOBSEN, Bechtel BWXT Idaho, LLC, Idaho Falls, ID An Innovative and WAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA An Innovative and ROBERT KELSEY, DAVE KOONTZ, WILEY WANG, VRTX Technologies, San Antonio, TX Biological Control DENNIS OPHEIM, Ph.D., Quinnipiac University, Hamden, CT, JOHN LANE, Clearwater Systems, Essex, C Cooling Water BactSUSANA SILVA MARTINEZ, ALBERTO A. ALVAREZ GALLEGOS, Instituto de Investigaciones Non-Chemical Wate YOUNG I. CHO, Drexel University, Philadelphia, PA Reducing UltrapurRON CHIARELLO, Ph.D., JUSTIN FOSTER, Etalon Technologies & Center for Environmentally Benign Sem HERO Process Volu RAFIQUE JANJUA, Duke Fluor Daniel, Sugar Land, TX HERO Process Volu CHARLES H. FRITZ, Black & Veatch Corporation, Kansas City, MO, BIPIN R. RANADE, Aquatech Internati Zero Liquid Disch GERALD N. TRUDEL, JR., AEP-PRO SERV, South Portland, MA, WILLIAM SANDERSON, Eautech, Redm Comparing the CruRICHARD HETHERINGTON, Epicor Incorporated, Linden NJ Comparing the CruDAVID F. RYAN, Exelon Nuclear, Limerick Station. Sanatoga, PA Evaluation of NewLORAINE HUCHLER, MarTech Systems, Inc., Lawrenceville, NJ Evaluation of NewMELISSA MELISSA KEGLAY, ONDEO Nalco Chemical Company. Naperville, IL, JIM DAUGHERTY, Armstrong Intern Evaluation of New Armstrong International, Inc., Three Rivers, MI Improvements in Co VINCE STEPHENSON, Ashland Specialty Chemical Company, Kansas City, KS, JOANNE KUCHINSKI, As An Overview of th WAYNE E. G. YEARWOOD, C. ANDREA JORDAN, WAYNE O’B. PRESCOD, Barbados Light and Power C Pre-Operational CSEAN MACDONALD,CEDA Reactor, Ltd., Sherwood Park, Alberta, Canada, BRAD BUECKER, CEDA, Inc. Are You Aware of HROB FERGUSON, Kimberton , PA Are You Aware of HDR. DAN VANDERPOOL, Laurel Functional Chemicals, Northport, AL Are You Aware of HDR. DAN VANDERPOOL, Laurel Functional Chemicals, Northport, AL Using New and OldLEROY RUGG, ONDEO Nalco Chemical Company, Pittsburgh, PA Using New and OldSCOTT J. WHITLOW, E.I. DuPont de Nemours, Beaumont, TX, THOMAS W. WOLFE, Puckorius and Asso Part III: Cooling PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO Understanding WetDANIEL E. SHANNON, Unipure Corporation, Houston, TX Understanding WetWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA, JOHN M. BURNS, PE, Burns Engin Understanding WetWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA, JOHN M. BURNS, PE, Burns Engin Audit Program RiskMARK CHERESNOWSKY, ONDEO Industrial Solutions, Richmond. VA Audit Program RiskSCOTT R. DISMUKES, DKW Law Group, PC, Pittsburgh, PA A Successful OutsPAULO S.S. SANTIAGO, JOILTON SILVA, Nalco, Brazil, JOAO B. LINS NETO, AMILCAR A. SALES, Cope New Reverse Osmos RON KOCHIK, ONDEO Nalco Chemical Co., Naperville, IL New Reverse Osmos BOB KIMBALL, Hydrometrics, Incorporated, Helena MT A New Membrane AMIT De SENGUPTA, FRED WIESLER, BETH KITTERINGHAM, Celgard Inc., Charlotte, NC Ultrafilter Membra PETER ERIKSSON, Osmonics, Vista, CA Ultrafilter Membra GLENN VICEVIC, ZENON Environmental Inc., Oakville, Ontario, Canada, STEVE KROLL, ZENON Environ Reverse Osmosis:RANDY C. TURNER, Southern Company, Atlanta, GA Reverse Osmosis:STEPHEN Y. MARR, Parsons Infrastructure & Technology Inc., Reading, PA, FRANK B. ARIETA, Procter & Controlling LegionMARK HODGSON, Clayton Group, Edison, NJ Anthrax Presentat JANET E. STOUT, PH.D., Infectious Disease Section,VA Medical Center Pittsburgh, PA Treating Water SysJANET E. STOUT, Ph.D., Infectious Disease Section,VA Medical Center Pittsburgh, PA Protozoa – Accompl SHARON G. BERK, Tennessee Technological University, Cookeville, TN Controlling Corros CHUN-SONG YE, CHENG-XIN ZHANG, XUN-JIE GONG, Wuhan University, College of Chemistry and Mol On-Line Improveme S. RONNIE PATE, Southern Company, Atlanta, GA On-Line Improveme THOMAS H. PIKE, Western Farmers Electric Cooperative, Fort Towson, OK, EMERY LANGE, Ashland Spe EPRI ChemExpert:JAMES C. BELLOWS, Siemens Westinghouse Power Corporation, Orlando, FL EPRI ChemExpert:OTAKAR JONAS, LEE MACHEMER, BARRY DOOLEY, Jonas, Inc., Wilmington, DE

Stability of Organ K. ANTHONY SELBY, Water Technology Consultants, Inc., Evergreen, CO Stability of Organ MARTIN R. GODFREY, ONDEO Nalco Chemical Company, Naperville, IL Pretreatment MethKUMAR SINHA, Bechtel Power Corporation, Frederick, MD173 Pretreatment MethSIMON GARE, Ecolochem International, Inc., Peterborough, UK A Statistical Revi EDWARD G. DARTON, MAQSOOD FAZEL, ONDEO Nalco Chemical Company, Northwich, UK New EDI ApplicatioTED PRATO, CHRIS GALLAGHER, Ionics Incorporated, Watertown, MA Effect of PAC DosiDr. PETER A. YARNELL, Graver Technologies, Inc., Glasgow, DE Effect of PAC DosiAVIJIT DEY, GARETH THOMAS, Chemitreat Private Limited, Singapore, KIRAN ARUN KEKRE, TAO GUIH Cation Conductivi WILLIAM MOORE, Calpine Central LP, Houston, TX Steam Purity for JAMES C. BELLOWS, Siemens Westinghouse Power Corporation, Orlando, FL Cation ConductivitL. CARVALHO,, P. SEHL, BetzDearborn, A Division of Hercules Canada, Mississauga, Ontario, Canada, R. Where’s the OrganiDEBORAH BLOOM, ONDEO Nalco Chemical Company, Naperville,IL Shallow Shell ResiWILLIAM FRIES, The Purolite Company, Bala Cynwyd, PA An Economical ProKHALED MOFTAH, Aquatech International Corp., Canonsburg,PA An Economical ProBOB BRADLEY, Hydrometrics, Incorporated, Houston, TX, MARK REINSEL, Hydrometrics, Incorporated, H Operating Efficien HAROLD ARONOVITCH, Hungerford & Terry, Inc., Clayton, NJ Operating Efficien MIKE AYRES, MIKE NELSON, Appleton Papers Inc., West Carrollton, OH, TERRANCE HELLER, The Puro Utilization of Ultr DOUGLAS J. GUMP, Severn Trent Services, Inc.-UltraDynamics, Colmar, PA Managing Microbiol RICK COLCLASURE, AquaComp, Inc., St. Charles, MO Severe Organic CoCLAUDIO L.V. DA COSTA, Deten, Brazil, PAULO S.S. SANTIAGO, ARMANDO GENTILE, Nalco, Brazil A Variant Form of RUSS GREEN, Amergen Energy, Middletown, PA Use of Ion ChromaBEVERLY NEWTON, Dionex Corporation, Sunnyvale, CA An Improved Metho THOMAS H. PIKE, Western Farmers Electric Cooperative, Fort Towson, OK, EMERY LANGE, Ashland Spe A New Approach t DAVID M. GRAY, Thornton Inc., Waltham, MA DNA Hybridization T.J. TVEDT, Jr., Puckorius & Associates, Inc., Angleton, TX, GEORGE ASPERY, Microbiological Laboratorie A New, Bromine-Rel TIMOTHY E. KEISTER, CWT, ProChemTech International, Inc., Brockway, PA A New, Bromine-Rel J. N. HOWARTH, C. J. NALEPA, Albemarle Corporation, Baton Rouge, LA A Review and Comp ALAN SMITH, ONDEO Nalco Chemical Company, Pittsburgh, PA A Review and Comp RICHARD W. LUTEY, Ph.D., RW Lutey & Associates, Inc., Memphis, TN, ARTHUR STEIN, Stone & Webste A Quantum Leap i TOMOAKI ITO, YUSUKE NAGETA. Orgallo Corporation, Tokyo, Japan, SHINTARO TSUZUKI, Rohm and H New Ion ExchangeELI SALEM, E. Salem & Associates, Inc., Deal, NJ New Ion ExchangeSTEPHEN W. NAJMY, The Dow Chemical Company, Midland, MI, The Sulfate Probl MICHAEL A. SADLER, Michael A. Sadler Consultant, Portishead, Bristol, UK The Sulfate Probl SALLIE FISHER, MICHELLE DONNELLY-KELLEHER, Puricons, Inc., Malvern, PA Panel Discussion: Panel Moderator – Mark Wilson, BetzDearborn, Horsham, PA Panel Discussion TPanel Moderator: ROBERT HOLLOWAY, Holloway Associates, Etobicoke, Ontario, Canada Keynote Speech DONALD P. FUSILLI, Jr.,, Chairman and CEO, Michael Baker Corporation, Coraopolis, PA Biocide Optimizati GEORGE J. LICINA, Structural Integrity Associates, Inc., San Jose, CA, LARS P. VENHUIS, KEMA Nederla Oxidation of Orga ALBERTO ALVAREZ-GALLEGOS, Instituto de Investigaciones Electricas, Morelos Mexico, DEREK PLETC Electrochemical TrPETERSON B. MORAES, CARLOS R. CORSA, ,EDERIO D. BIDOIA, State University of Sao Paulo Purification and R C. RAVI, Aquatech International, Pittsburgh, PA Treatment of GrouJEFF McBRIDE, Infilco Degremont Inc., Richmond, VA Effective Upgrade ROBERT A. ESCOBEDO, Ecolochem, Inc., Norfolk, VA, (SCOTT B. GORRY, Ecolochem, Inc., East Hartfor Filtration Options DAVID A. CLINE, Jr., Sheppard T. Powell Associates, LLC, Baltimore, MD Operation and Oper JOHN A. ZAVRL, TIMOTHY J. ZAK, First Energy Corporation, Eastlake, OH, BERNIE MACK, Ionics, Inc.,W An Innovative ApprJOSEPH D. GIFFORD, DEVEN ATNOOR, USFilter, Lowell, MA A Quantitative AnaK. ANTHONY SELBY, Water Technology Consultants, LLC, Evergreen, CO A Quantitative AnaDANIEL M. CICERO, CAROL B. BATTON, Nalco Chemical Co., Naperville, IL, DAVID A. BIESZK, P.E., Arm Industrial ISFET p SHANE FILER, FARID AHMAD, Honeywell, Fort Washington, PA On-Line PPB-LevelDEBORAH BLOOM, Nalco Chemical Company, Naperville, IL On-Line PPB-LevelPHILIPPE SERIZOT, Zellweger Analytics, Inc., Paris, France, VICKIE OLSON, Zellweger Analytics, Atlanta,

Using EDI to MeetTED PRATO, CHRISTOPHER JAY GALLAGHER, Ionics, Watertown, MA EDI Operation for JOHN H. BARBER, GLENN TOWE, DAVID F.TESSIER, E-Cell Corporation, Guelph, Ontario, Canada Experiences with CKUMAR SINHA, Bechtel Power Corporation, Frederick, MD Experiences with J. STUART BROWN, Liquid Six Inc., York. ME, RANDY GERMAN, TransCanada Power, North Bay, Ontario Biological Pretre P. TEMPLE BALLARD, Degremont North American Research & Development Center, Kichmona, VA, ALBE Chemistry ChallenS. S. MITRA, Dabhol Power Plant, Maharashtra, India, P. K. SINHA, Bechtel Corporation, Frederick, MD How to Handle SusKAI-UWE HOEHN, Dow Chemical Limited, Frenchs Forest, Australia, ANDRE MEDETE, Dow Deutschland, The Effect of Serv HERMAN C. HAMANN, JOSEPH V. D’ALESSANDRO, The Purolite Co., Philadelphia, PA, JAMES A. MANS Softening with WAGUY J. MOMMAERTS, Ion Exchange Services (Canada) Inc., Elora, Ontario, Canada The Cost of ProducPHIL FATULA, Bayer Corporation, Pittsburgh, PA, STEFAN HILGER, Bayer AG, Leverkusen, Germany, FRE Performance of HiTAKESHI IZUMI, TOSHI TAKAI, AKIRA MATSUMOTO, TAKAO INO, MASAHIRO HAGIWARA, Ebara Corpo Pilot and Plant Sc BENNETT P. BOFFARDI, Boffardi and Associates, Bethel Park, PA Pilot and Plant Sc W. D. SAUNDERS, Sasol Technology (Pty) Ltd. Secunda, South Africa, C. P. BRAND, Sasol Synthetic Fuel Redefining Coolin EDWARD S. BEARDWOOD, Drew Industrial pision, Ashland Canada, Inc., Ajax, Ontario, Canada, KEVIN Major ImprovementRICHARD O. YOUMANS, Buckman Laboratories International, Inc., Memphis, TN Major ImprovementPAULO S. S. SANTIAGO, Nalco Chemical Company, Sao Paulo, Brazil, RICARDO PRADO SANTOS, Ultra Use of Comphensi MEL J. ESMACHER, BetzDearborn, the Woodlands, TX, GEORGE BODMAN, George H. Bodman, Inc., Kin Determining the NeK. ANTHONY SELBY, Water Technology Consultants, Inc., Evergreen, CO The Impact of Wate ROBERT HOLLOWAY, Holloway Associates, Etobicoke, Ontario, Caanada Boiler Failure Me DOUGLAS DEWITT-DICK, STEPHEN McINTYRE, JOSEPH HOFILENA, Ashland Specialty Chemical Co., An Examination of GLENN REYNOLDS, ROBERT GLOD, New Logic International, Emeryville, CA Enhancing the Meas STAN LUECK, RODI Systems Corp., Aztec, NM Low Fouling PolyaPATRICK H. KINGHORN, Ecolochem, Inc., Baytown, TX, WILLIAM E. HAAS, Ecolochem, Inc., Norfolk, VA Treatment of SmelWAYNE BATES, Hydranautics, Rockton, IL Treatment of SmelBOB KIMBALL, Hydrometrics, Inc., Helena, MT Reverse Osmosis S PIMON G. GARE, Ecolochem International, Inc., Peterborough, UK, MATTHEW J. WHITE, Ecolochem Inte Calcium CarbonateROBERT W. ZUHL, BF Goodrich Co/Specialty Additives Group, Cleveland, OH Calcium CarbonateGORDON R. BURTON, M. E. BLIMKIE, C. W. TURNER, Atomic Energy of Canada, Chalk River, Ontario, C Part II: Cooling W ROBERT J. FERGUSON, French Creek Software, Kimberton, PA Part II: Cooling W PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO Chemical Condition Dr. FABIO GIGLI, CHIMEC S.p.A., Rome, Italy Quad Polymer Techn THOMAS W. WOLFE, Puckorius & Associates, Baton Rouge, LA Quad Polymer Techn JOHN RICHARDSON, MICHAEL G. TRULEAR, Chem Treat, Inc., Richmond VA Maintaining Stora J. HELMRICH, Florid a Power & Light, Florid a City, FL, WILLIAM E. HAAS, Ecolochem, Inc., Norfolk, VA, J Nuclear Plant Ste KARA PRENTICE, PHILIP BATTAGLIA, ART BYERS, , DEBORAH FARNSWORTH, JONNA PARTEZANA, A Study on the AppRONALD D. WINE, Ronald D. Wine Consulting, Harrisburg, PA A Study on the AppJEON-SOO MOON, KWANG-KYU PARK, SEOK-WON YUN, Korea Electric Power Research Institute,(KEP System MonitoringWILLIAM MOORE, Calpine Corp., Houston, TX System MonitoringSARA JANE GEARY, JAMES C. BELLOWS, Siemens Westinghouse Power Corp., Orlando, FL System MonitoringSARA JANE GEARY, JAMES C. BELLOWS, Siemens Westinghouse Power Corp., Orlando, FL Pilot Testing High BRIAN POWERS, The Dow Chemical Company, Kingwood, TX Pilot Testing High ROBERT BRADLEY, Hydrometrics, Inc., Houston, TX Interstage Energy GEOFFREY K. HART, Montgomery Watson, Sunrise, FL Interstage Energy JULIA E. NEMETH, P.E., Ham R/O Systems, Inc., Venice, FL, STEVEN DURANCEAU, Ph.D., P.E., Boyle E The Impact of Eli BRETT ANDREWS, JOSEPH MAZUR, PermaCare US A Inc., San Diego, CA Dosing Liquid Dis Dr. ANDREW W. COURT, DAVID GRANT, A. G. CALLERY, Portacel Disinfection Systems, Hampshire, UK Algicidal Perform ROBERT D. BARTHOLOMEW, PE, Sheppard T. Powell Associates, LLC, Baltimore, MD Algicidal Perform ANDREW J. COOPER, ANTHONY W. DALLMIER, Nalco Chemical Company, Naperville, IL Mixed-Oxidant Appl EDWARD S. BEARDWOOD, Drew Industrial Division of Ashland Canada, Ajax, Ontario, Canada Mixed-Oxidant Appl WESLEY L. BRADFORD, Ph.D., Los Alamos Technical Associates, Inc., Los Alamos, NM, PAUL PETERSO Eliminating Oxidi R. JONES, BetzDearborn, Trevose, PA, JAMES F. ECHOLS, SIDTEC Services, Inc., Houston, TX, TOM FR

The Use of Membra DOUG GLANZ, Betz Dearborn Argo Scientific, Cincinnati, OH The Use of Membra DAVID THRELFALL, Ecolochem International, Inc., Peterborough, UK Design-Build of a CHARLES FRITZ, Black & Veatch, Overland Park, KS Design-Build of a IAN A. CROSSLEY, DONALD BRAILEY, Hazen and Sawyer, P.C., New York, NY; JAMES HOFF, USFilter, R Condensate PolishGEORGE CRITS, Aqua-Zeolite Sciences, Inc., Ardmore, PA Condensate PolishGERALD P. GALL, GPU/Genco, New Florence, PA Deaeration – the P ELI SALEM, Ecodyne Limited, Cranford, NJ Deaeration – the P BILL RUNYAN, JAMES ABRUZZO, Cochrane, Inc., King of Prussia, PA Corrosion BehaviorMINORU KOBAYASHI, HAJIME HIRASAWA, Toshiba Corporation Power Systems & Services Company, Yo High Pressure BoilTHOMAS H. PIKE, Western Farmers Electric Cooperative, Ft. Towson, OK Corrosion-ProductBARBARA D. SAWICKA, JERZY A. SAWICKI, AECL, Chalk River Laboratories, Maintaining Effec Dr. JOHN RICHARDSON, ChemTreat, Inc., Ashland, VA Maintaining Effec RICARDO DE ARAUJO FERNANDES, BetzDearborn Brazil; ALTINO ALVES BENTO, Petroquimica Uniao, Development of anSERGIO CASTRO, Instituto Mexicano Del Petroleo (IMP), Mexico, D.F., Mexico High Cycle CoolingJ. E. HOOTS, DON A. JOHNSON, J. D. LAMMERING, D. A. MEIER, B. YANG, Nalco Chemical Company, N Cooling Water ScaWAYNE C. MICHELETTI, Wayne C. Micheletti Inc., Charlottesville, VA Cooling Water ScaPAUL R. PUCKORIUS, GARY R. LORETITSCH, Puckorius & Associates, Inc., Evergreen, CO Cooling Water ScaPAUL R. PUCKORIUS, GARY R. LORETITSCH, Puckorius & Associates, Inc., Evergreen, CO The Effect of SandZHONGFANG LEI, Fudan University, Shanghai, China; QINAI BAO, Shanghai Petrochemical Co. Ltd., Sha Large Scale Zero WAYNE C. MICHELETTI, Wayne C. Micheletti, Inc. Charlottesville, VA Large Scale Zero JOHN P. TOOHIL Jr., USFilter Industrial Wastewater Systems Billerica, MA Innovative MethodMICHAEL S. DALTON, Baker Petrolite, Sugarland, TX Innovative MethodSUSAN BAINES, KUMAR SINHA, Bechtel Power Corporation, Frederick, MD Follow-up Report tCHRISTOPHER HEADLEY, US Generating Company, Swedesboro, NJ Cooling Towers anROBERT DEBENEDETTO, Public Service Elec. & Gas Co., Ridgefield, NJ; ARTHUR J. FREEDMAN, Arthu The Chemicals for MICHAEL S. DALTON, Baker Petrolite, Sugarland, TX The Chemicals for QINAI BAO, Shanghai Petrochemical Co., Ltd., Shanghai, China Halogen-Tolerant ARTHUR FREEDMAN, Arthur Freedman Associates, Inc., East Stroudsburg, PA Halogen-Tolerant JASBIR S. GILL, SUSAN P. REY, MONICA A. YORKE, Calgon Corporation, Pittsburgh, PA Development of a JENNIFER C. HORNE, NARASIMHA M. RAO, JOHN E. HOOTS, Nalco Chemical Company, Naperville, IL An Evaluation of MORIN HOLLANDER, HOLLANDER Technologies, Inc. Jamison, PA An Evaluation of MWILLIAM E. GARRETT, JR., Alabama Power Company, Birmingham, AL; TERRY SELF, PAULINE BROWN An Evaluation of MWILLIAM E. GARRETT, JR., Alabama Power Company, Birmingham, AL; TERRY SELF, PAULINE BROWN Water Reuse in anSUSAN L. COULTER, US Filter, Rockford, IL Water Reuse in anTIMOTHY KEISTER, CWT, ProChemTech International, Inc., Brockway, PA A Unique High RecDENNIS McBRIDE, Intel Corp., Rio Rancho, NM A Unique High RecMICHAEL HUMPHREYS, , Membrane Systems Corporation, Cape Coral, FL;,; GEORGE .PATRICK, Parso Water Conservation S. S. MITRA, ENRON, Ratnagiri, India Water Conservation K. DAS. D.. K. PANDE, Krishak Bharati Co-Operative Ltd, Surat, India Water Conservation K. DAS. D.. K. PANDE, Krishak Bharati Co-Operative Ltd, Surat, India Recovery of ReverLAWRENCE KRZESWSKI, General Motors, Detroit, MI; A. BERLANGA, D. VALLE, GM de Mexico; RAMO Zero Discharge Pro MICHAEL SIECKMANN, ISSI/Amrox, Pittsburgh, PA Zero Discharge Pro DOUGLAS R. OLSEN, Green Technology Group, Pawling, NY; CHARLES D. BLUMENSCHEIN, Chester E Advances in WaterDENNY DORAN, Nalco Chemical Co., Pittsburgh, PA Advances in WaterROSS K.FULLER, CHRISTOPHER R. LEITZ, Ashland Chemical Company, Drew Chemical Division, Boont Removal of SeleniKASHI BANERJEE, USFilter/Chester Engineers, Moon Township, PA Removal of SeleniSTEPHEN P. ELLIS, Ecolochem, Inc., Norfolk, VA Removal of SeleniSTEPHEN P. ELLIS, Ecolochem, Inc., Norfolk, VA Gas Transfer MemR. ALLEN PITTMAN, CELGARD LLC, Charlotte, NC Speciation of Weakl JOHN H. BARBER, CANDIDA OLIVEIRA, DAVID TESSIER, E-Cell Corporation, Guelph, Ontario, Canada Improvement in anRONALD PICHT, Environmental Dynamics Corporation, Sharon, WI

Improvement in anLYNDON FLEMING, Ionics, Incorporated, Columbia, AL; TED PRATO, LI ZHANG, BRIAN HERNON, Ionics Advances in ContinDICK SAMPSON, Halox Technologies Corporation, Bridgeport, CT Advances in ContinANIL D. JHA, JOSEPH D. GIFFORD, US Filter Corporation, Lowell, MA SWRO for AquaculDONALD TERRILL, George Scott International, Pensacola, FL Vapor Compssion SHARIF DISI, Mechanical Equipment Company, Inc., New Orleans, LA Reducing RO Opera STAN LUECK, RODI Systems Corp, Aztec, NM Biological Control JOHN CURRIER, Tennessee Valley Authority, Chattanooga, TN Biological Control ROBERT D. BARTHOLOMEW, P.E., Sheppard T. Powell Associates, LLC, Baltimore, MD Capillary UF as R PHIL ROLCHIGO, Osmonics, Inc. Minnetonka, MN Capillary UF as R WAYNE T. BATES, Hydranautics, Rockton, IL Water Pretreatment SCOTT BEARDSLEY, Dow Filmtec, Edina, MN Water Pretreatment WIL F. PERGANDE, Osmonics Inc., Minnetonka, MN Ceramic Membrane RICKY BRYANT, Southern Company Services, Birmingham, AL Ceramic Membrane DAVID THRELFALL, Ecolochem International, Inc., Peterborough, UK Efficacy of Biocid EDWARD McCALL, JANET E. STOUT, VICTOR L. YU, VA Medical Center, Pittsburgh, PA; RADISAV VIDIC The Elimination of ARTHUR HARRIS, Feedwater Ltd, Moreton, Wirral, UK; MIKE RENDELL, BTG International Ltd, London, U Cooling Tower SysPAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, CO HRSG Design – WaFRANK GABRIELLI, ABB Combustion Engineering Ultrapure Steam f JAMES C. BELLOWS, MICHAEL McMANUS, JON DONOVAN, Siemens Westinghouse Power Corporation Performance MonitANTON BANWEG,, Nalco Chemical Company, Naperville, IL Combined Cycle an K. ANTHONY SELBY, Water Technology Consultants, LLC, Evergreen, CO Changing Boiler WR. HENRY WEED, BetzDearborn, Horsham, PA Heat Exchanger Des RAYMOND M. POST, P.E., Betz Dearborn, Trevose, PA Heat Exchanger Des GEORGE F. HAYS, Ashland Chemical Company, Drew Industrial Division, Boonton, NJ; JAMES G. KNUDS Heat Exchanger Des GEORGE F. HAYS, Ashland Chemical Company, Drew Industrial Division, Boonton, NJ; JAMES G. KNUDS Condenser Chemistr ERIC HALE, Nalco Chemical Co., Naperville, IL Condenser Chemistr BRAD BUECKER, CEDA, Lawrence, KS The Importance ofTERRY LATERRA, Graver Water Systems, Cranford, NJ The Importance ofSHAN S. SUNDARAM, PE., AMBI-Design, Inc., Rockford, IL Economics of ResMICHAEL O’BRIEN, Graver Water Systems, Cranford, NJ Economics of ResAMY H. LETTOFSKY, Rohm and Haas Company, Philadelphia, PA Total Installed Co ROBERT BRADLEY, MECO, New Orleans, LA Total Installed Co KENNETH G. EXCELL, P.E., Water & Power Technologies, Inc., Salt Lake City, UT; CARL T. LINCOLN, P.E Nine Mile Point U R F. GREEN, Niagara Mohawk Power Corporation, Oswego, NY Use of Low Level EClois D. Fears, Delta Applied Technology, Inc., Pittsburgh, PA Impacts and Contro CHARLES R. O’NEILL, Jr., New York Sea Grant, Brockport, NY Strategic SourcingDENNIS SHEA, Solutia, Inc., Alvin, TX Strategic SourcingJOHN T. MEAKIM, Infilco Degremont, Inc., Richmond VA The Water Audit asCARSON BARRY, Westvaco Corporation, Luke, MD The Water Audit asKAREN L. ASHER, USFilter, Lowell, MA; HARRY C. DELONGE, USFilter, Amenia NY The Business Aspec FRED W. KILE, US Filter, Rockford, IL Seawater Desalinat Panel Discussion Moderator: Dave Morris, Chester Engineers, Pittsburgh, PA Legionella: Updat Panel Discussion Moderator: Dileep Thatte, Calgon Corporation, Pittsburgh, PA HRSGs Don’t HavePanel Moderator; James Robinson, BetzDearborn, Horsham, PA Keynote Address REBECCA MARK, Vice Chair, Enron Corporation, Chair and Chief Executive Officer, Azurix Use of Modern Comp STEWART W. TAYLOR, Bechtel Power Corporation, Frederick, MD Use of Modern Comp RUSSELL ELLIOTT, R.V. Elliott Enterprises, Toronto, Ontario, Canada, ROBERT BROBERG, AEA Technolo Experience on Orga VALENTINA CIMKINA, MOSENERGO, Moscow, Russia TetrakishydroxymeANTHONY W. DALLMIER, Nalco Chemical Company, Naperville, IL TetrakishydroxymeDR. R. E. TALBOT, B. L. DOWNWARD, Albright & Wilson, Birmingham, U.K.; THOMAS K. HAACK, Albright A “Greener”, Cost-ARTHUR J. FREEDMAN, Arthur Freedman Associates, Inc., East Stroudsburg, PA

A “Greener”, Cost-BRIAN K. FAILON, R. G. GABRIEL, Albright & Wilson Americas, Inc., Glen Allen, VA Bromine-Based Bioc ROBERT D. BARTHOLOMEW, Sheppard T. Powell Associates LLC, Baltimore, MD Optimizing Wastewa CARLETON P. BOWEN, ICARUS Corporation, Rockville, MD Using Computers NICHOLAS J. FURIBONDO, Nalco Chemical Company, Naperville, IL Use of Computer SLEONARD FREEDMAN, LXF Incorporated, Wilmington, DE Pricing a SeawaterDr. IRVING MOCH, Jr, I. Moch & Associates, Inc., Wilmington, DE Pre-Treatment of JAMES D. MAVIS, CH2M HILL, Bellevue, WA Pre-Treatment of ALLAN L. FARBER, Bechtel Limited, Frederick, MD; ANDREW R. MILSTED, Bechtel Limited, London, UK; Seawater as an InTHOMAS M. MISSIMER, ROBERT R. WRIGHT, Missimer International, Inc., Fort Myers, FL Desalination of SePAUL A. W. CHOULES, Dr. ALASDAIR MACIVER, MECO, New Orleans, LA Wastewater ReclaROBERT P. ALLISON, Ionics, Incorporated, Watertown, MA Wastewater ReclaROBERT T. TAYLOR, JR., Ecolochem, Inc., Norfolk, VA Installation and S BRUCE A. LARKIN, Black & Veatch LLC, Kansas City, MO Installation and S TOBY C. WAGNER, ROBERT M. McGRAW, Ecolochem, Inc., Norfolk, Virginia; SAM HARVEY, Tennessee Economic EvaluatiJOHN CASTAGNA, American Aquasciences, Parsippany, NJ Economic EvaluatiSHAN S. SUNDARAM, P.E., AMBI-Design, Inc., Rockford, IL Economics of Water KENNETH FREDERICK, Ion Exchange Associates, Inc., Reading, PA Economics of Water HUGH PATTERSON, City of Lakeland Florida, Lakeland, FL; KENNETH R. WEISS, Black & Veatch, Kansa A New Halogen-Resi DONALD A. JOHNSON, Nalco Chemical Co., Naperville, IL A New Halogen-Resi KURT M. GIVEN, ROGER C. MAY, CLAUDIA C. PIERCE, BetzDearborn Water Management Group, Trevos A New Halogen-Resi KURT M. GIVEN, ROGER C. MAY, CLAUDIA C. PIERCE, BetzDearborn Water Management Group, Trevos Carbon Dioxide Ass JOHN C. TVERBERG, P.E., Trent Tube, East Troy, WI; JEROME T. WELZ, Miller Brewing Company, Milwau Concentration of CFRANCIS M. CUTLER, FM Consulting & Engineering, Irvine, CA Concentration of CHIROSHI YAMAUCHI, TAKASHI HONDA, TAKESHI ONODA, Hitachi, Ltd.; TAKAYUKI MIZUNO, SHOUJI N Concentration of CHIROSHI YAMAUCHI, TAKASHI HONDA, TAKESHI ONODA, Hitachi, Ltd.; TAKAYUKI MIZUNO, SHOUJI N Good Lay-up PractiROBERT HOLLOWAY, Holloway Associates, Islington, Ontario, Canada Good Lay-up PractiDAVID G. DANIELS, Mechanical & Materials Engineering, Austin, TX Water Chemistry CK. ANTHONY SELBY, GARY A. LORETITSCH, Puckorius & Associates, Inc., Evergreen, CO Use of a Patented CHRISTOPHER BREW, Gainesville Regional Utilities, Gainesville, FL Use of a Patented PETE AVERELL, Sithe Energies, Inc., Massena, NY; PETER H. WREDE, DOUG CALVEY, AL FLORANCE, Complete Reuse ofMICHAEL S. DALTON, Baker Petrolite, A Baker Hughes Company, Sugar Land, Txi Complete Reuse ofALLAN L. FARBER, Bechtel Power Corporation, Frederick, Maryland; EMMA E. HARNIMAN, Bechtel Limite The Advantage of M DENNIS McBRIDE, Intel Corporation, Rio Rancho, NM The Advantage of M ROBERT SOLOMON, Ph.D., KAREN E. SCHOOLEY, Ionics RCC, Bellevue, WA; SAMUEL J. GRIFFIN, Or Wastewater Reduct BRIAN TOWNES, Simplot Canada Limited, Brandon, Manitoba, Canada PHILIP FATULA, Bayer Corporation, Pittsburgh, PA; FRED MUIR, Ecodyne Limited, Start-Up and OperBurlington, Ontario Performance of HiSALLIE FISHER, Puricons, Inc., Malvern, PA Performance of HiTAKESHI IZUMI, TATSUYA DEGUCHI, AKIRA MATSUMOTO, TAKAO INO, MASAHIRO HAGIWARA, EBAR Packed-Bed Techno GUY J. MOMMAERTS, Ion Exchange Services (Canada) Inc., Elora, Ontario, Canada Double Flow HydroCHARLES H. FRITZ, Black & Veatch LLP, Kansas City, MO Double Flow HydroZHANG CHENGXIN, Department of Applied Chemistry, Wuhan University of Hydraulic & Electric Engineerin Cooling Water ChloPHILIP JONES, U.S. Generating Company, Kennerdell, PA Marine Biofouling ARTHUR J. FREEDMAN, Arthur Freedman Associates Inc., E. Stroudsburg, PA Marine Biofouling YASUHIRO TANIMURA, JUNJI HIROTSUJI, KAZUHIRO MIYA, Mitsubishi Electric Corporation, Amagasaki Innovative TroubleTHOMAS S. McCURDY, JILL TEITELBAUM McCURDY, The Breakthrough Group, Cherry Hill, NJ Development and O TATSUYA DEGUCHI, TAKAO INO, EBARA Corporation, Fujisawa-shi, Kanagawa, Japan; MASAHIRO HAG Total Organic CarbANTHONY BEVILACQUA, Ph.D., Thornton Associates, Waltham, MA Total Organic CarbKAREN CLARK, JOHN STILLIAN, JERRY KIRKPATRICK, Anatel Corporation, Boulder, CO Sources of TOC inDANIEL B. RICE, Dow Chemical Company, Midland, MI Sources of TOC inPETER S. MEYERS, ResinTech, Inc., Cherry Hill, NJ Trace Organics in WILLIAM E. BORNAK, Aqueous Solutions, Inc., Richboro, PA

Advances in ElectrELI SALEM, Ecodyne Limited, Cranford, NJ; DAVID F. TESSIER, E-Cell Corp., Guelph, Ontario, Canada Removal of WeaklyBRIAN P. HERNON, HILDA ZANAPALIDOU, TED PRATO, LI ZHANG, Ionics, Incorporated, Watertown, MA Characterizing Was PAUL CHU, Electric Power Research Institute, Palo Alto, CA; PHILIP BENSON, DENNIS FINK, CH2M HILL Chemically Foam C JAMES R. RUCK, CHARLES D. FOSTER, HydroChem Industrial Services, Inc., Pine Grove, CA The Special Probl ZHANG CHENGXIN, Department of Applied Chemistry, Wuhan University of Hydraulic & Electric Engineerin Impact of Deregul MICHAEL L. WISDOM, P.E., ENRON Engineering & Construction Company, Houston, TX Impact of Deregul RON KOSAGE, Bechtel Power Corp., Pine Grove, CA Design, Start-Up, STEPHEN P. WOOD, Carney’s Point Generating Station, Carneys Point, NJ Design, Start-Up, JOEL M. DAVIE, Bechtel Power Corp., Frederick, MD; KEITH YEGERLEHNER, Indiantown Generating Co. The Behavior of StDr. RODNEY DONLAN, Calgon Corporation, Pittsburgh, PA The Behavior of StRICHARD E. AVERY, Nickel Development Inst., Londonderry, NH; S. LAMB, Nickel Development Institute, H Case History: ContRICHARD W. LUTEY, Buckman Laboratories International, Memphis, TN MIC Mitigation in RICHARD W. LUTEY, Buckman Laboratories International, Memphis, TN Case Study of Comp DONALD L. GIBBON, CHRISTOPHER L. WIATR, CECILIA M. McGOUGH, Calgon Corporation, Pittsburg Copper Fouling of ANDREW HOWELL, New Century Energies, Englewood, CO Steering Power PlaFRANK GABRIELLI, ABB Power Plant Laboratories Combustion Engineering, Inc., Windsor, CT Steam and Water Pr JAMES C. BELLOWS, Siemens Westinghouse Power Generation, Orlando, FL Keys to On-Line MLORAINE A. HUCHLER, P.E., MarTech Systems, Inc., Lawrenceville, NJ; WILLIAM J. HERBERT, Sr., Johns Oxidation/Reducti SHANE FILER, Honeywell Analytical, Fort Washington, PA; GARRY CRAIG, Eraring Power Station, New So Upgrading a CycleORIN HOLLANDER, Holland Technologies, Inc., Jamison, PA Upgrading a CycleJOEL R. KASPER, Aquagenics Incorporated, Woburn, MA; DAVID MILES, U.S. Generating Co., Salem, MA Innovative New Pro BEVERLY NEWTON, Dionex Corporation, Sunnyvale, CA Future Trends in SCOTT S. BEARDSLEY, The Dow Chemical Company, Minneapolis, MN Troubleshooting PDAVID H. PAUL, David H. Paul Inc., Farmington, NM Optimize RO SysteLEE A. DURHAM, Argo Scientific Inc., San Marcos, CA; MARK BOARDMAN, Argo Scientific Ltd., Heriot-Wa A Difficult Iron R WILLIAM E. BORNAK, Aqueous Solutions, Inc., Richboro, PA What Do I Need toWAYNE BERNAHL, Nalco Chemical Company, Naperville, IL Developing Ceramic RAMESH R. BHAVE, U.S. Filter/Memcor, Timonium, MD Developing Ceramic W. E. HAAS, Ecolochem, Inc., Norfolk, VA; GREG BARTLEY, Tennessee Valley Authority, Chattanooga, TN Reducing the FoulPETER K. ERIKSSON, Osmonics /Desal, Vista, CA Reducing the FoulWAYNE T. BATES, Hydranautics, Rockton, IL Reverse Osmosis RANDALL J. MAJERLE, Argo Scientific, San Marcos, CA Reverse Osmosis ROBERT Y. NING, Ph.D., King Lee Technologies, San Diego, CA Screen Filtration TTHOMAS E. HAMILTON, Power Products & Services Co., Battleground, WA High Silt Density ROBERT L. BRADLEY, U.S. Filter, Houston, TX High Silt Density GREG BARTLEY, Tennessee Valley Authority, Chattanooga, Tennessee; ARTHUR J. ACKERMANN, RUSS Avoiding Potential MEL J. ESMACHER, P.E., BetzDearborn, The Woodlands, TX Boiler Tube Failu DAVID A. CLINE, JR., CHARLES A. GREENE, Ph.D., Sheppard T. Powell Associates, LLC, Baltimore, MD Failure Analysis a MEHROOZ ZAMANZEDEH, Ph.D., MATCO Associates, Inc., Pittsburgh, PA Panel Discussion: Moderator; Torry Tvedt, Puckorius & Associates, Angleton, TX. Panelists: HAROLD CHAGNARD, Dow Ch Keynote Address RICHARD HECKMANN, Chairman of the Board, President and CEO, U.S. Filter Corporation, Palm Desert, Using Predictive TBENNETT BOFFARDI, Boffardi and Associates, Pittsburgh, PA Using Predictive TV. K. CHEXAL, J. S. HOROWITZ, DOUGLAS P. MUNSON, K. SHYE, C. S. SPALARIS, Electric Power Res Application of the JEFFREY GARDNER, Pacific Gas & Electric, Avila Beach, CA Using Carbohydrazi PETER D. HICKS, RICHARD J. MOUCH, Nalco Chemical Company, Naperville, IL Application of the JOEL W. McELRATH, Consumers Energy, Covert, MI Optimization of WaPETER J. MILLETT, CHRIS WOOD, Electric Power Research Institute, Palo Alto, CA Microfiltration Te TONY. TONELLI, ZENON Environmental Systems, Inc, Burlington, Ontario, Canada, STEVE KROLL, ZENO Ceramic Membrane SCOTT WITTWER, Graver Separations, Inc., Glasgow, DE Ceramic Membrane JOSEPH HOOLEY, U.S. Filter, Middleburg Heights, OH, JEFF J. PETERS, JOHN F. BOSSLER, U.S. Filter, A New High Flow M LI ZHANG, Ionics Incorporated, Watertown, MA

A New High Flow M WILLIAM E. HAAS, Ecolochem, Inc., Norfolk, VA, JAY GRAFTON, TVA Brown’s Ferry Nuclear Plant, Athens Reduction of TOC/KUMAR SINHA, Bechtel Corporation, Gaithersburg, MD Reduction of TOC/WANDA R. HARRISON, PECO Energy Company, Sanatoga, PA, SANDY J. SCHEXNAILDER, Ionics, Incor RO System DesignMARK A. THOMPSON, Malcolm Pirnie Incorporated, Newport News, VA RO System DesignWAYNE BATES, Hydranautics, Oceanside, CA, RANDY MAJERLE, Argo Scientific, San Marcos, CA Use of SurfactantsANTHONY W. DALLMIER, Nalco Chemical Company, Naperville, IL Use of SurfactantsRODNEY M. DONLAN, DAVID L. ELLIOT, DONALD L. GIBBON, Calgon Corporation, Pittsburgh, PA Evaluation of a NoWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, VA Evaluation of a NoT. OGAWA, T. TAMURA, S. FUJIWARA, T. OZAWA, , Mitsubishi Electrical Corporation, Kobe, Hyogi, Japan Enzyme Technology RICHARD W. LUTEY, Ph.D., Buckman Laboratories International, Inc., Memphis, TN Five Years of Ope ROBERT Taprogge Americas, NY Texas Utilities JAMES F.RENFFTLEN, ECHOLS, Betz Dearborn, Houston,Inc., TX,Woodbury, RON JONES, Five Years of Ope Longview, TX Five Years of Ope JAMES F. ECHOLS, Betz Dearborn, Houston, TX, RON JONES, Texas Utilities, Longview, TX Cost of Retrofit o DANIEL B. RICE, Dow Chemical Company, Midland, MI Cost of Retrofit o PETER S. MEYERS, ResinTech, Inc., Cherry Hill, NJ Making Old MakeuSALLIE C. FISHER, Puricons, Inc., Malvern, PA Ion Exchange Demin BILL RUNYAN, Cochrane Environmental Systems, King of Prussia, PA Ion Exchange Demin KEN D. PANDYA, Advanced Water Technologies Services, Inc., Plano, TX Demineralizer RecJOHN D. ZUPANOVICH, Chemtreat, Inc., Glen Allen, VA Demineralizer RecKARA Products Company, Ferndale, WA, GLORIA K. SHELTON, Baker Industrial Che JOHN MILLHOLLIN, J, SCHUBERT,ARCO Modular Environmental Technologies, Inc., Optimization of WaPittsburgh, PA Optimization of WaDEMETRI P. PETRIDES, J. CALANDRANIS, Intelligen, Inc., Scotch Plains, NJ Engineering SoftwMARK ISAACS, AEA Technology, Pittsburgh, PA, PETER MARTIN, AEA Technology plc., Didcot, Oxfordshir CFD Modeling of thHARVEY S. PORDAL, AEA Technologies Pittsburgh, PA CFD Modeling of thMARK W. WENDEL, P. T. WILLIAMS, J. H. PLATFOOT, Oak Ridge National Laboratory, Oak Ridge, TN Amoco Chemical Co ANDRE MEDETE, Dow Chemical Germany Amoco Chemical Co WENDY N. WATSON, FRANK C. MARINUCCI, Amoco Chemical Company, Decatur, AL, AMY H. LETTOFS The Properties andPETER YARNELL, Graver Technologies, Newark, NJ The Properties andJAMES IRVING, JAMES A. DALE, The Purolite Company Ltd., Wales, UK, TERRENCE L. HELLER, The Pu Selective RemovalARUP K. SENGUPTA, DONGYE ZHAO, Lehigh University, Bethlehem, PA Problems of H2O2SALLIE FISHER, Puricons, Inc., Malvern, PA Problems of H2O2DIRK-MICHAEL PFENNING, Bayer AG, Leverkusen, Germany Control of Flow-AcV. K. CHEXAL, R. BARRY DOOLEY, DOUGLAS P. MUNSON, RICHARD M. TILLEY, Electric Power Rese Pathology of SinglT. J. TVEDT Jr., Puckorius & Associates, HAROLD A. CHAGNARD, The Dow Chemical Co., Plaquemine, L Flow-Accelerated C PAUL BURGMAYER, Betz Dearborn, Trevose, PA Microbiologically RICHARD W. LUTEY, Buckman Laboratories, Memphis, TN Microbiologically GREG KOBRIN, Nickel Development Institute, Beaumont, TX, STEPHEN LAMB, Nickel Development Instit Microbially-Influe DONALD L. GIBBON, RODNEY M. DONLAN, Calgon Corporation, Pittsburgh, PA Use of a New Brom RODNEY H. SERGENT, Chattem Chemical Company, Chattanooga, TN Use of a New Brom ROBERT M. MOORE, C. J. NALEPA, G. L. GOLSON, Albemarle Corporation, Baton Rouge, LA, T. W. WOL The DeterminationC. H. POLLEMA, S. F. ACEVEDO, Hach Chemical Company Loveland, CO A Novel Approach R t ENEE J. JACOBS, MARYKE KRUGER, Sasol Chemical Industries, Sasolburg, South Africa Cation Conductivi DAVID M. GRAY, ANTHONY C. BEVILACQUA, Thornton Associates, Inc., Waltham, MA Fluorescent TracedMICHAEL J. CHMELOVSKI, Nalco Chemical Company, Naperville, IL Optimization of ChJEON-SOO MOON, KWANG-KYU PARK, DO-YEONG WEON, Korea Electric Power Research institute, Ta A Real-Time, ContJOHN WEBB, Nalco Chemical Company,Naperville, IL A Real-Time, ContROBERT D. BARTHOLOMEW, Sheppard T. Powell Associates, LLC, Baltimore, Maryland A Real-Time, ContBROWN T. HAGEWOOD, JEFFREY J. KOST, Long Island Lighting Company, Glenwood Landing, NY Development and Te WAYNE W. FRENIER, HydroChem Industrial Services, Houston, TX Experimental Study JAMES C. BELLOWS, Westinghouse Electric Corporation, Orlando, FL Experimental Study HIROSHI YAMAUCHI, TAKASHI HONDA, TAKESHI ONODA, Hitachi Ltd., Hitachi-city, Japan, TAKAYUKI M

Waterside Inspectin DENNIS A. FREY, Dennis Frey Consultants, Canton, OH Waterside Inspectin SAMUEL B. DILCER JR., Cyrus Rice Water Consultants, Bradford Woods, PA, JAMES C. DROOMGOOLE New Calcium Carbo PATRICK H. GILL, Calgon Corporation, Pittsburgh, PA New Calcium Carbo DANIEL PARKER VANDERPOOL, Laurel Functional Chemicals, Tuscaloosa, AL A New Corrosion I MICHAEL E. ROGERS, Syncrude Canada Ltd., Alberta, Canada A New Corrosion I M. MONACO, Usina Santa Cruz, Ometto Paven Group Américo Brasiliense, San Paulo, Brazil, WILTON JO A New Environmenta JIM HAFF, Calgon Corporation, Pittsburgh, PA A New Environmenta BRIAN L. DOWNWARD, Albright & Wilson U.K. Ltd., Warley, West Midlands, U.K., BRIAN K. FAILON, Albrig Rust Cleaning at BRUCE L. LIBUTTI, JOSEPH MIHELIC, Ashland Chemical Company Drew Divisions, Boonton, NJ, RICHA Toward Field-FrienWILLIAM M. HANN, LORRAINE H. KELLER, THOMAS W. SANDERS, BARRY WEINSTEIN, Rohm and Ha COD Reduction in NIGEL E BLAKE, MARK NEVILLE, AEA Technology, Didcot, Oxfordshire, U.K., MARK ISAACS, AEA Techno Removal of Ammoni HANS D. PFLUG, MARTIN B. HEIN, H. J. BETTENWORTH, ANDREAS HENKEL, Preussen Elektra AG, H UV/Oxidation Proce DANIEL J. TIERNEY, EG&G Florida, Inc., Kennedy Space Center, FL, GEORGE S. KOSAR, NASA Kenne Simultaneous Remo MARY H. KERR, Sargent & Lundy, Chicago, IL Simultaneous Remo JOSEPH A. DRAGO, D. A. FRUTH, L. Y. C. LEONG, Kennedy/Jenks Consultants, San Francisco, CA, G. F Use of New MembrKRISH PARTHASARATHY, Culligan Water Technologies, Inc. Northbrook, IL, FRANK VARNI, Kagome, U.S RO Operating CostROBERT P. ALLISON, Ionics, Watertown, MA RO Operating CostSERGIO GHERARDI, Culligan Italiana S.p.A., Bologna, Italy Pretreatment SysteA. F. ASCHOFF, Sargent & Lundy, Arlington Heights, IL Pretreatment SysteT. H. PIKE, Western Farmers Electric Cooperative, Ft. Towson, Oklahoma, EMERY LANGE, Drew Industria Regulatory Issues CAROL JONES LEDWIG, The Dow Chemical Co., Freeport, TX Regulatory Issues DANIEL M. CICERO, Nalco Chemical Company, Naperville, IL Predict and OptimiGARY REGGIANI, Betz Dearborn, Horsham, PA Operational SteamJAMES W. SMITH, Shell Oil Products, Houston, TX Operational SteamEDWARD BEARDWOOD, Drew Chemical Limited, an Affiliate of Ashland Chemical Company, Ajax, Ontario Operational SteamEDWARD S. BEARDWOOD, Drew Chemical Limited, an Affiliate of Ashland Chemical Company, Ajax, Onta Industrial Plant C ROBERT T. HOLLOWAY, Holloway Associates, Islington, Ontario, Canada Industrial Plant C PAUL G. SCHMIDT, DAVID E. SIMON, II, Cyrus Rice Water Consultants, Pittsburgh, PA Recovery of ReverLAWRENCE V. KRZESOWSKI, General Motors North America Operation Facilities, Detroit, MI, (ROBERTO Waste MinimizationDENNIS SHEA, Solutia Inc., Alvin, TX Waste MinimizationWOLFGANG HATER, Henkel Surface Technologies, Düsseldorf Germany, THOMAS SCHMITZ, Hydro Agri Microfiltration of LINDA V. DELLA CORNA, JAMES L. FILSON, U.S. Filter Corporation, Outsourcing WaterRICHARD J. BARTKOWSKI, U.S. Filter Operating Services, Inc., Pittsburgh, PA Strategic Alliance BRENT W. CHETTLE, Water & Energy Systems Technology, Inc., Anaheim, CA Comphensive Spe CARSON M. BARRY, Westvaco, Luke, MD Comphensive Spe DAVID E. SIMON II, Cyrus Rice Water Consultants, Pittsburgh, PA Comphensive Spe DAVID E. SIMON II, Cyrus Rice Water Consultants, Pittsburgh, PA A Comparision of HAROLD ARONOVITCH, Hungerford & Terry, Inc., Clayton, NJ A Comparision of CHRISTIAN BELTLE, L. & C. Steinmuller Gmbh, Gummersbach, Germany, GERHARD LISSON, InfraServe Key Components in SALLIE FISHER, Puricons, Inc., Malvern, PA Key Components in PHILIP W. FATULA, Bayer Corporation, Pittsburgh, PA Successful Exper GEORGE J. CRITS, Aqua-Zeolite Sciences, Inc. Successful Exper AMY H. LETTOFSKY, AMIN KIAMI, Rohm and Haas Company, Philadelphia, PA Suspended Solids CHARLES FRITZ, Black & Veatch, Kansas City, MO Suspended Solids DANIEL B. RICE, The Dow Chemical Company, Midland, MI, ANDRE MEDETE, Dow Deutschland Inc., Rh Beneficial Reuse DAVID H. BECK, ChemTreat Inc., Richmond, VA Beneficial Reuse CLIFFORD S. WILKINSON, P.E., Paulus, Sokolowski, and Sartor, Inc., Warren, NJ Development and P Ing.VICTOR MARTΊNEZ MORALES, Altos Hornos de México S.A. DE C. V, Monclova, Coahuila, Mexico Water Reuse for TeDR. LUCIANA COCCAGNA, Culligan Italiana S.P.A., Bologna, Italy Development of HiSAKAE KOSANDA, TATSUYA DEGUCHI, AKIRA MATSUMOTO, TAKESHI IZUMI, MASAHIRO HAGIWARA Sixteen Year ExperKWANG-KYU PARK, DO-YEONG WEON, JEON-SOO MOON, Korea Electric Power Research Institute, Ta

ARTHUR J. FREEDMAN, Arthur Freedman Associates, Inc., East Stroudsburg, Pennsylvania, GEORGE F. Successful OperatiNewark, New Jersey Controlling Microb CRAIG SCHINZER, 3M Facilities Engineering, St Paul, Minnesota Controlling Microb ABDULMOHSEN D. AL-MAJHOUNI, HOWARD R. RUSSER, Jr., ARIFE E. JAFFER, Saudi Aramco, Dhahra Start-Up and OperCRAIG W. MYERS, Nalco Chemical Company, Naperville, Illinois Start-Up and OperT. J. TVEDT, Jr., The Dow Chemical Company, Freeport, Texas, ROBERT STEWART, Dow Chemical Canad History and Strat WILLIAM T. BOYD, Associated Electric Cooperative, New Madrid, Missouri History and Strat GEORGE VERIB, Ohio Edison Company, Akron, Ohio Equilibrium PhosphTHOMAS J. WYSOCKI, PETER R. LOVALLO, Detroit Edison, Detroit, Michigan Equilibrium PhosphWILLIAM B. BORCK, Tennessee Valley Authority, Gallatin, Tennessee, GREG L. BARTLEY, Tennessee Val Solubility and Th FRANCES M. CUTLER, Southern California Edison, Paramount, California Solubility and Th PETER TREMAINE, SEAN QUINLAN, J. BRIDSON, Memorial University of Newfoundland, St. John’s New Ten Years of Equil KAL BOYD, TOM FITZSIMMONS, Basin Electric Power Cooperative, Stanton, North Dakota Ten Years of Equil JAN STODOLA, Ontario Hydro, Toronto, Ontario, Canada How to Specify a DOUGLAS P. GLANZ, Hydranautics, Lake Geneva, Wisconsin, JOHN HERRING, Hydranautics, Houston, T The Importance ofJAMES C. DROMGOOLE, Fort Bend Services, Inc., Stafford, Texas Ion Exchange Capac DANIEL B. RICE, The Dow Chemical Company, Midland, Michigan 450 PPM Silica SuDENNIS McBRIDE, Intel Corporation, Rio Rancho, New Mexico, DEB MUKHOPADHYAY, Independent Che A New Method of Pr DAVID C. AUERSWALD, Southern California Edison, Paramount, California A New Method of Pr JAMES A. BELL, Smith & Loveless, Inc., Lenexa, Kansas City, Dr. GRAEME K. PEARCE, Ph.D., Kalsep, Lt Membrane Separati JANE KUCERA, HARI GUPTA, Wheelabrator Water Technologies, Inc., Naperville, Illinois Membrane Separati JOHN E. DAILY, Metcalf & Eddy, Inc., Tampa, Florida Membrane and Elem LEE COMBS, Osmonics, Minnetonka, Minnesota Membrane and Elem JOHN PERLMAN, DAVID FABER, Fluid Systems Corporation, San Diego, California Sodium Chlorate ba SCOTT A. MARCKINI, Ashland Chemical Company, Drew Industrial Division, Boonton, New Jersey Feasibility of the ROBERTO RAMIREZ-DELGADO, JUAN PAREDON-DELGADO, Comision Federal de Electricidad-Laborat Identification of JOSEE CHAULT, L. D’ARSIE, BetzDearborn Inc., Mississauga, Ontario, Canada, P. M. BODKIN, BetzDearb Rapid Biocide SeleYANG XIAO QIN, Consultant Center of Environment Protection Technology of Power Industry, Nanjing, Chi Copper DepositionKEN LAYTON, Pacificorp, SaltWilmington, Lake City, Utah OTAKAR JONAS, Jonas, Inc, Delaware, LARRY DUPREE, Carolina Power & Light, Raleigh, N Copper DepositionOTAKAR Portage, Wisconsin, MARK STEVENS, Montana Power, Colstrip, Montana JONAS, Jonas, Inc, Wilmington, Delaware, LARRY DUPREE, Carolina Power & Light, Raleigh, N Copper DepositionPortage, Wisconsin, MARK STEVENS, Montana Power, Colstrip, Montana HOSCO: a Software F. SIGON, ENEL Spa/R&D Division – CRAM, Milano, Italy, R. MARTINI, CISE Innovative Technologies, Mila Steam Chemistry an ROBERT B. PETERSON, Salt River Project, Page, Arizona Steam Chemistry an OTAKAR JONAS, Jonas, Inc., Wilmington, Delaware, BARRY DOOLEY, EPRI, Palo Alto, California Steam Chemistry an OTAKAR JONAS, Jonas, Inc., Wilmington, Delaware, BARRY DOOLEY, EPRI, Palo Alto, California Problem-Solving TJ. E. HOOTS, Nalco Chemical Company, Naperville, Illinois, S. J. ARMITAGE, Nalco Chemical Company, N Troubleshooting StSTEPHEN JAMES ROBINSON, BetzBaltimore Dearborn,Gas Water Management Group, Horsham, Pennsylvania J. SHULDER, & Electric Company Comparison of Sil Baltimore Maryland Comparison of Sil D. P. NESBITT, PE, HydroChem Industrial Services, Inc., Houston, Texas Field Application WAYNE W. FRENIER, Hydrochem Industrial Services, Inc., Houston, Texas, STEPHEN P. DONNER, Cons Pressure Pulse ChC. WAYNE URION, Delmarva Power & Light Company, Wilmington, Delaware Pressure Pulse ChSAM HARVEY, Tennessee Valley Authority, PHILIP J. BATTAGLIA, Westinghouse Electric Corporation OPTIMIZING PERModerator; Sam Dilcer, Cyrus Rice Water Consultants, Bradford Woods, Pennsylvania OPTIMIZING PERModerator: Sam Dilcer, Cyrus Rice Water Consultants, Woods, Pennsylvania FRANK CROWSON, SETH PERKINSON,, Fuji AmericaBradford Inc. Here’s How RomatCharlotte, North Carolina Technical Study foREYES AVILA JESUS, MORALES HERNANDEZ LUIS, Instituto Mexicano del Petroleo, Division de Protecc Carbon-Add® in FrT WILLUWEIT, I. SCHUSTER, N. KURZ, S. WEIßGRÄBER, Soell-Ecological Processings Hof, Germany , P Chemical CleaningROBERT E. PALMER, The Detroit Edison Company, Detroit, Michigan Chemical CleaningG. S. LAWRENCE, Columbia Energy Center, Wisconsin Power & Light Company, Portage, Wisconsin Application of Robo JOHN M. PUGSLEY, Consultant, ARD Environmental, Inc. Water Management ROSS K. FULLER, CHRISTOPHER R. LEITZ, JOHN W. McCARTHY, Ashland Chemical Company, Drew In Twelve Years of Wa P. J. DU T. ROUX, Sastech Engineering Services, Secunda, Republic of South Africa, R. S. LUDLUM, Ionic

Activated Sodium B R. M. MOORE, Albemarle Corporation, Baton Rouge, Louisiana, W C. LOTZ, V. R. PERRY, Albemarle Corp The Use of Reclai PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, Colorado, KRIS HELM, West Basin Munic Corrosion ReductioJOSEPH A. WILLY, Sony Electronics Inc., Pittsburgh, Pennsylvania Corrosion ReductioDENNIS K. MORAN, City of Burbank, Plant Manager, Burbank, California, BRUCE A. JOHNSTON, PE, Bet Advanced Water Co ROBERT J. CUNNINGHAM, Chemisis, Inc., Mission Viejo, California Advanced Water Co M. C. GERAGHTY, R. A. LAHTI, Calgon Corporation, Pittsburgh, Pennsylvania, R. CHIMNEY, BP Oil Comp Improvement of PrSHIGERU OKUYAMA, NAGAO SUZUKI, Tokyo Electric Power Company, Tokyo, Japan, SEIICHI KAZAMA, Improved Precoat JOSEPH M. RAGOSTA, Ph.D., PETER YARNELL, Ph.D., Graver Chemical Company, Glasgow, Delaware Improved Water Che LEO F. RYAN, Finetech Incorporated, Mountain Lakes, New Jersey Improved Water Che W. F. MURRAY, E. BRENNAN, Virginia Power, Surry, Virginia, E. C. FRESE, Virginia Power, Glen Allen, Virg Application of th PHILIP D’ANGELO, JoDan Technologies, Ltd., Glen Mills, Pennsylvania Application of th TAKESHI IZUMI, YOUSHI OHMORI, MASAHIRO HAGIWARA, TAKAO INO, TATSUYA DEGUCHI, Ebara C Kinetics Testing i FRANCES M. CUTLER, Southern California Edison, Paramount, California Kinetics Testing i SALLIE FISHER, LYNETTE DAY, Puricons, Inc., Malvern, Pennsylvania Operation of the FiW. E. ALLMON, The Babcock & Wilcox Company, Research & Development Division, Alliance, Ohio Operation of the FiROY McBRAYER, P.E., JAMES GRIFFITH, P.E., Eco Waste Technologies, Inc., Austin, Texas Hydrothermal Oxida JAMES C. BELLOWS, Westinghouse Electric Corporation, Orlando, Florida Hydrothermal Oxida G. T. HONG, MODAR, INC., Natick, Massachusetts Supercritical WateDIGBY MACDONALD, Ph.D., Pennsylvania State University, University Park, Pennsylvania Supercritical WateMICHAEL MODELL, SIEGFRIED MAYR, Modell Environmental Corporation, Waltham, Massachusetts, AND Supercritical WateSERGUEI LVOV, Ph.D., Pennsylvania State University, University Park, Pennsylvania Supercritical WateKENNETH A. SMITH, JONATHAN G. HARRIS, JACK B. HOWARD, JEFFERSON W. TESTER, PETER GR On-Site GenerationWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, Virginia On-Site GenerationCHRISTOPHER MOSS, Capital Controls Limited, Sittingbourne, Kent, United Kingdom, GERALD CONNEL Impact of Cooling MICHAEL G. TRULEAR, ChemTreat, Inc., Glen Allen, Virginia Impact of Cooling JEFFREY F. KRAMER, FMC Corporation, Princeton, New Jersey Case History: Per ALAN PRYOR, Ozone Process Consultants, Sunnyvale, California Case History: Per DANIEL J. TIERNEY, ELLEN S. FEENEY, EG&G Florida, Inc., Kennedy Space Center, Florida, ROBERT A. Laboratory Test MeT. K. HAACK, Albright & Wilson Americas, Inc., Richmond, Virginia Laboratory Test MeG. H. CANULLO-VUNK, Ph.D., J. C. STEELHAMMER, Ph.D., JIM LUKANICH, Buckman Laboratories, Mem EDS+R/O Combinati MITSUGU ABE, YASUHIKO AOKI, YUJI HARAGUCHI, Nomura Micro Science Company, Ltd. Kanagawa, J Progress Report: ABRIAN P. HERNON, LI ZHANG, LINDA R. SIWAK, ERIK J. SCHOEPKE, Ionics, Incorporated, Watertown, M Tests of a Three- WAYNE BATES, Hydranautics, Rockton, Illinois Tests of a Three- WAYNE L. ADAMSON, BRIAN E. WEBER, DAVID J. NORDHAM, Naval Surface Warfare Center Carderock Carbon Dioxide a FRED WIESLER, Hoechst Celanese, Charlotte, North Carolina Carbon Dioxide a S. H. MACKLIN, Northeast Utilities, Waterford, Connecticut, W. E. HAAS, W. S. MILLER, Ecolochem, Inc., N Optimizing the PerGARY C. GANZI, U.S. Filter, Lowell, Massachusetts Optimizing the PerDAVID C. AUERSWALD, Southern California Edison, Paramount, California Optimizing the PerDAVID C. AUERSWALD, Southern California Edison, Paramount, California A Successful Disp ARIEL BARBOSA DE CARVALHO, Aco Minas Gerais S.A., Ouro Branco, MG, Brazil, CARLOS EDUARDO Waste Treatment of DAVID RADTKE, WAYNE FRENIER, HydroChem Industrial Services, Inc., Houston, Texas, ROBBIE RUSE Treatment of PoweWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc., Charlottesville, Virginia Treatment of PoweH. D. PFLUG, M. B. HEIN, H. J. BETTENWORTH, H. A. SYRING, PreussenElektra AG, Hanover, Germany Treatment of PoweH. D. PFLUG, M. B. HEIN, H. J. BETTENWORTH, H. A. SYRING, PreussenElektra AG, Hanover, Germany Design and Start- LAWRENCE GURNARI, The Graver Company, Union, New Jersey, THOMAS ROBERTS, Pennsylvania Ele Conversion of a C DANIEL B. RICE, The Dow Chemical Company, Midland, Michigan, ANDRE MEDETE, Dow Deutschland I Upflow Counter-C JAMES DOBOS, F. BRYAN POPP, EUGENE M. VEGSO, Aquatech International Corporation, Canonsburg, Co-Current Ion Ex JASON T. MIYAKE, JAMES D. FRUGE, Hoechst Celanese, Pampa, Texas New Life for Ion HENRY P. STOEBENAU, Rohm and Haas Company, Philadelphia, Pennsylvania Two Years’ OperatiROGER WARREN, Ph.D., Saskferco Products, Inc., Belle Plaine, Saskatchewan, Canada, DAVID HICKLIN 40 Years in the EvWALTER ERBSCHWENDNER, Bayer A.G., Leverkusen, Germany, PHILIP W. FATULA, NANCY C. MILLHO

Financial Justific GERALD P. GALL, Pennsylvania Electric Company, New Florence, Pennsylvania Financial Justific STEVE BARBER, W. E. STEVENS, HydroChem Industrial Services, Inc., Houston, Texas Cleaning of SteamWILLIAM STEVENS, HydroChem Industrial Services, Inc., Houston, Texas Cleaning of SteamDENNIS A. FREY, Babcock and Wilcox Company, Barberton, Ohio Monitoring of Sup STEPHEN J. SHULDER, Baltimore Gas & Electric, Baltimore, Maryland Monitoring of Sup OTAKAR JONES, RAVI MATHUR, Jonas, Inc., Wilmington, Delaware Monitoring of Sup OTAKAR JONAS, RAVI MATHUR, Jonas, Inc., Wilmington, Delaware Comparative EffectIAN LEWIS, Antigua Public Utilities Authority, Antigua, West Indies, JOHN DILZELL, Betz Water Manageme Coagulation/FloccuMOHAMMAD F. ALBELDAWI, Ph.D., Compass Environmental Inc., Hamilton, Ontario, Canada An Ideal Scale InhiRICHARD H. ASHCRAFT, ChemTreat, Inc., Richmond, Virginia An Ideal Scale InhiJ. S. GILL, Ph.D., Calgon Corporation, Pittsburgh, Pennsylvania Recovery of WaterGREG FUHR, Diversey Group, Pittsburgh, Pennsylvania Recovery of WaterBRIAN C. MULINIX, Goodyear Tire & Rubber Company, Lincoln, Nebraska, RONALD A. CAPPELLO, Chem Pilot Studies and E. V. MAUGHAN, Krohne Messtechnik GmbH & Co. KG, Duisburg, Germany, G. GERICKE, Eskom Techno Operating PracticeT. J. TVEDT, Jr., The Dow Chemical Company, Freeport, Texas, ROBERT T. HOLLOWAY, Nalco Canada, In Steam Purity RecoA. BANWEG, Nalco Chemical Company, Naperville, Illinois Steam Purity RecoJAMES C. BELLOWS, Westinghouse Electric Corporation, Orlando, Florida Phosphate Treatme JAN STODOLA, Ontario Hydro, Toronto, Ontario, Canada Phosphate Treatme R. B. DOOLEY, EPRI, Palo Alto, California, L. D. PAUL, B & W Research and Development Division, Allianc Biodegradable AlteROBERT J. ROSS, KIM C. LOW, ANNE MARIE ATENCIO, Donlar Corporation, Bedford, Park, Illinois, JAM Expert System in ALBERTO ALVAREZ-GALLEGOS, SUSANA SILVA-MARTINEZ, University of Southampton, Southampton, Scaling is a ProduROBERT J. FERGUSON, French Creek Software, Kimberton, Pennsylvania Scaling is a ProduDR. D. HAWTHORN, min-DEP Research, Ruddington, Nottingham, United Kingdom Scaling is a ProduDR. D. HAWTHORN, min-DEP Research, Ruddington, Nottingham, United Kingdom New Non-Phosphoru HAROLD A. CHAGNARD, The Dow Chemical Company, Plaquemine, Louisiana New Non-Phosphoru G. E. GEIGER, Betz Water Management Group, Horsham, Pennsylvania Evaluation of ExpeA. M. BADAWI, E. A. M. GAD, A. ABDELHAFIZ, Egyptian Petroleum Research Institute, Nasr City, Cairo, E Controlling Costly DENNIS MARTIN, ChemTreat, Inc., Richmond, Virginia Controlling Costly A. GRYSCAVAGE, P.E., Betz Water Management Group, Horsham, Pennsylvania, D. STONER, Princeville Poly(Ethylene OxidSTANLEY P. WALKER, Allied Colloids, Inc., Suffolk, Virginia Poly(Ethylene OxidAMY M. BOLLINGER, Union Carbide Corp., Bound Brook, New Jersey, Assuring Long-Term S. S. HEBERLING, Dionex Corporation, Sunnyvale, California Novel Monitoring aW. A. MITCHELL, L. M. KYE, G. A. SMALL, Grace Dearborn, Lake Zurich, Illinois, D. O. VEIL, Air Products Actives-Based Chem JOHN W. SIEGMUND, Sheppard T. Powell Associates, Baltimore, Maryland Actives-Based Chem J. E. HOOTS, S. J. ARMITAGE, E. W. EKIS, JR., Nalco Chemical Company, Naperville, Illinois, J. M. JOHN Monitoring and Con ROLAND A. LEATHRUM, Thomas M. Laronge, Inc., Newark, Delaware Monitoring and Con ROBERT RUMELFANGER, KEITH J. SALAMONY, Calgon Corporation, Pittsburgh, Pennsylvania Applications of N JAMES R. WILSON, The Purolite Company, Bala Cynwyd, Pennsylvania, CLAUDE GAUTHIER, The Puroli Prevention of ResiSHAN S. SUNDARAM, Ambi-Design, Inc., Rockford, Illinois, S. ARRINDALE, S. DE LANOI, KAREL P. VAN A Comparison of aGEORGE L. DIMOTSIS, FRANK McGARVEY, Sybron Chemicals Inc., Birmingham, New Jersey Operating ExperieF. X. McGARVEY, Sybron Chemicals Inc., Birmingham, New Jersey Operating ExperiePETER S. MEYERS, ResinTech Inc., Cherry Hill, New Jersey Case History: MicrRUSSELL W. LANE, Consultant, Champaign, Illinois Case History: MicrSTEPHEN A. WORTENDYKE, Capital Controls Company, Colmar, Pennsylvania Case History in th MICHAEL A. WINTERS, Amoco Corporation, Naperville, Illinois Case History in th FEDERICO ARIAS, H-O-H Systems, Inc., Katy, Texas, LINH V. TRAN, The Goodyear Tire and Rubber Com Development and T WAYNE W. FRENIER, HydroChem Industrial Services, Inc., Houston, Texas, DAVID LARSON, Larson Engi Cleaning of Oil Fi PETER S. MEYERS, ResinTech Inc., Cherry Hill, New Jersey Innovations in De MICHAEL O’BRIEN, Graver Water Division, Union, New Jersey, PAUL McNULTY, Public Service Electric & Developing the EPBRUCE LARKIN, LES WEBB, Black & Veatch, Kansas City, Missouri, BARRY DOOLEY, Electric Power Re Predicting Mixed RICHARD HETHERINGTON, Epicor, Inc., Linden, New Jersey

Predicting Mixed GREG BACHMAN, JULIE STERLING, DAVE MOYER, U.S. Filter/IWT, Rockford, Illinois Effects of Ethanol PHILIP J. D’ANGELO, JoDAN Technologies, Glen Mills, Pennsylvania Effects of Ethanol DIANA KEELING, Pacific Gas and Electric Company, Avila Beach, California, SALLIE FISHER, Puricons, In Optimization of C BILLY D. FELLERS, TU Electric, Glen Rose, Texas, KERMIT FORSHAGE, Buckman Laboratories, Denton, On-Line Monitorin R. J .A. TIPPETT, V. R. KUHN, Grace Dearborn, Lake Zurich, Illinois, J. RICHARDSON, Grace Dearborn, C Improvement of Cal ALBERTO ALVAREZ GALLEGOS, Instituto de Investigaciones Electricas, Cuernavaca, Mor., Mexico, SUSA Evaluation of HighWAYNE C. MICHELETTI, Wayne C. Micheletti, Inc. Charlottesville, Virginia Evaluation of HighJIM LUKANICH, Buckman Laboratories, Memphis, Tennessee Performance of a JAMES L. RABER, 3M Company, St. Paul, Minnesota Performance of a J. S. GILL, M. A. YORKE, S. L. HOLLINGSHAD, Calgon Corporation, Pittsburgh, Pennsylvania Improvements in WSCOTT CHURBOCK, Envirotrol, Inc., Sewickley, Pennsylvania Improvements in WRICHARD L. RANICH, DONALD E. FRESHCORN, Zinc Corporation of America, Monaca, Pennsylvania Advanced Treatmen DAN WESOLOWSKI, Westinghouse, Pittsburgh, Pennsylvania Advanced Treatmen LO TAN, Orange County Water District, Fountain Valley, California Case Study: Wastew KASHI BANERJEE, Chester Environmental, Moon Township, Pennsylvania Case Study: Wastew OSCAR QUINONES, Cydsa, El Salto, Jalisco, Mexico, RODI LUDLUM, Resources Conservation Company Wastewater Treatme JIM CARROLL, U.S. Filter, Warrendale, Pennsylvania Wastewater Treatme MEINT OLTHOF, DAVID D. MONIOT, Killam Associates, Warrendale, Pennsylvania, ZYGMUNT V. OSIECK On-Line Diagnos1sOTAKAR JONAS, Jonas, Inc., Wilmington, Delaware Optimum Chemistry B. DOOLEY, Electric Power Research Institute, Palo Alto, California, J. MATHEWS, Duke Power, Charlotte, On-Line MonitoringFRANCES M. CUTLER, Southern California Edison, Paramount, California On-Line MonitoringANNA DEAK-PHILLIPS, Duke Power Company’ Charlotte, North Carolina, RICHARD HUTTE, Sievers Instr High Temperature C EHARLES C. STAUFFER, Babcock & Wilcox Research, Alliance, Ohio High Temperature B EILLY D. FELLERS TU Electric Glen Rose, Texas, SUE A. HOBART, Adams & Hobart, Inc. Fremont, Califor Phosphate Treatme GEORGE J. VERIB, ALAN L. WADDINGHAM, Ohio Edison Company, Stow, Ohio Phosphate Treatme B. DOOLEY, Electric Power Research Institute, Palo Alto, California, S. PATERSON, Aptech Engineering Se Design and OperatiMARY H. KERR, Sargent & Lundy, Chicago, Illinois Design and OperatiQAZI HAIDER, DANNY MOORE, TU Electric, Glen Rose, Texas, PAUL W. GROSS, Monk Engineering, Inc Application of Ele THOMAS A. DAVIS, Consultant, Graver Water, Union, New Jersey Application of Ele BRIAN P. HERNON, R. HILDA ZANAPALIDOU, LI ZHANG, LINDA R. SIWAK, ERIK J. SCHOEPKE, Ionics, Application of Ele BRIAN P. HERNON, R. HILDA ZANAPALIDOU, LI ZHANG, LINDA R. SIWAK, ERIK J. SCHOEPKE, Ionics, Removal of Dissol JACK B. PRATT, H-O-H Systems, Inc., Katy, Texas Removal of Dissol YUJI HARAGUCHI, ISAMU SUGIYAMA, MASAHIKO KOGURE, Nomura Micro Science Co. Ltd., Atsugi Cit Development of a M RALPH G. LOMBARDO, HARRIS KRAINMAN, Ashland Chemical Company, Drew Industrial Division, Boon Constructed Wetlan ROBERT HEDIN, Hedin Environmental Co., Sewickley, Pennsylvania Constructed Wetlan ANDREW DZURIK, Florida State University, Tallahassee, Florida, DANUTA LESZCZYNSKA, Jackson State Benefits of Activa MICHAEL G. TRULEAR, Nalco Chemical Company, Naperville, Illinois Benefits of Activa ROBERT M. MOORE, Albemarle Corporation, Baton Rouge, Louisiana, WAYNE C. LOTZ, VERNON R. PE Crud Removal Char TAKESHI IZUMI, YOUSHI OHMORI, TAKAO INO, TATSUYA DEGUTI, Ebara Corporation, Tokyo, Japan, YO Dimethylamine Che MIKE ROOTHAM, Westinghouse Corporation, Madison, Pennsylvania Dimethylamine Che BILLY D. FELLERS, TU Electric, Glen Rose, Texas, DAVE S. SHENBERGER, Calgon Corporation, Pittsbur Development of Hig SEIICHI KAZAMA, MASAHITO KOBAYASHI, Hitachi Works, Hitachi, Ltd., Hitachi-shi, Japan, TAKAYOSHI T PWR Reactor CoolJGARY OHN KRISTENSEN, New York G. Power, Buchanan, New York Electric Corporation, Monroeville, Pennsylva J. CORPORA, GEORGE KONOPKA, Westinghouse PWR Reactor CoolAshford, Alabama Pre-Morpholated DWILLIAM H. CARLIN, Jr., Rohm and Haas Company, Philadelphia, Pennsylvania Pre-Morpholated DBILLY D. FELLERS, TU Electric, Glen Rose, Texas, DR. JOSEPH RAGOSTA, Graver Chemical Company, Ultrasoft Water Tr SHAN S. SUNDARAM, Ambi-Design, Inc., Rockford, lllinois Studies on the Eff GEORGE L. DIMOTSIS, FRANK X. McGARVEY, Sybron Chemicals Inc., Birmingham, New Jersey Makeup Water Trea MILTON P. CROSSEN, Aquatech International Corp., Canonsburg, Pennsylvania Makeup Water Trea WILLIAM S. MILLER, Ecolochem, Inc., Norfolk, Virginia Weak Acid Cation G. J. CRITS, Aqua Zeolite Sciences, Inc., Havertown, Pennsylvania

Weak Acid Cation HAROLD ARONOVITCH, Hungerford & Terry, Inc., Clayton, New Jersey, ROGER C. FORD, NRS Consultin Short Bed Deminera DR. JOSEPH RAGOSTA, Graver Chemical Company, Glasgow, Delaware Short Bed DeminerDOUG JACKSON, BRYCE HEARTWELL, TransAlta Energy Corporation, Mississauga, Ontario, Canada, M Ultrasperse™: A NSCOTT BOYETTE, KIM JOLLEY, Betz Water Management Group, Trevose, Pennsylvania, JERRY HENRY The Use of a Corr IAN EISNER, T. Y. CHEN, M. R. GODFREY, Nalco Chemical Company, Naperville, Illinois Avoiding Failures ALBERT D. OWENS, Cyrus Rice Consulting Group, Pittsburgh, Pennsylvania Avoiding Failures ROBERT J. CUNNINGHAM, Chemisis, Inc., Laguna Hills, California An Integrated Boi HAROLD CHAGNARD, The Dow Chemical Company, Plaquemine, Louisiana An Integrated Boi ROBERT BRADLEY, JOSEPH HOOLEY, Arrowhead Industrial Water, Inc., Lincolnshire, Illinois Oil Spill Protectio YUSUF MUSSALLI, Stone & Webster Environmental Technology & Services, Boston, Massachusetts, JOH Trace Metals Remov DAVID L. DRUMMONDS, RICHARD D. HORNBUCKLE, Southern Company Services, Inc., Birmingham, Al New Chelating Resi JUNJI FUKUDA, TAKAYUKI TASHIRO, TAKASHI KASHIWAGI, Mitsubishi Chemical Corp. Tokyo, Japan, M Automatic Control R o OBERT G. COOK, Jr., Modular Environmental Technologies, Inc., Pittsburgh, Pennsylvania Automatic Control F oRANK MORENSKI, Graver Water, Union, New Jersey Copper Reduction CHARLES b M. SUENKONIS, Rust Environment & Infrastructure, Mechanicsburg, Pennsylvania Copper Reduction CHARLES b H. FRITZ, Black & Veatch, Kansas City, Missouri, LAWRENCE R. LATTA, American Electric Pow Demineralizer Up David L. Drummonds, Southern Company Services, Inc., Birmingham, Alabama Upgrading Boiler KEN LAYTON, Pacificorp Salt Lake City, Utah, MICHAEL R. PACEK, CHRIS FOURNIER, U.S. Filter, Lowel Reverse Osmosis F PRANCES M. CUTLER, Southern California Edison, Paramount, California Reverse Osmosis H P ERMAN C. HAMANN, Ph.D., Arrowhead Industrial Water, Brecksville, Ohio, WILLIAM G. BUYOK, Chevro Operating ExperienSHARON S. WHIPPLE, The Dow Chemical Company, Larkin Laboratory, Midland, Michigan Operating ExperienJ. DAVID WILLERSON, Georgia Power Company, Atlanta, Georgia, PETER E. DOWN, Mitco Water Labora Upgrading of Boil BRUCE L. COULTER, U.S. Filter, Rockford, Illinois Upgrading of Boil JOHN J. KIM, ROBBIN M. JOLLY, DAVID E. BRINDLE, ANNA-DEAK PHILLIPS, Duke Power Company, Ch The Open RecirculROBERT STRANDBERG, Ogden Martin Systems of Lancaster, Inc., Marietta, Pennsylvania, TERRY D. Mc Eight Years of Ze BRIAN HARRINGTON, Bristol Energy, Bristol, New Hampshire, JEFF KISTY, MARCUS VASKA, Ashland C Waste Water Recycl KEVIN CARLSON, Husky Oil Operations Ltd., Lloydminster, Saskatchewan, Canada, W. HUGH GOODMAN Low Cost Zero DisJAMES E. VAN WYK, Niro Inc., Columbia, Maryland, WILLIAM RUNYON, Crane-Cochrane Environmental Water ConservatioDENNIS J. SHERREN, DAN STONE, Enron Power Corporation, Houston, Texas, PETER A. BROWN, Calg Progress Report oJOHN F. GAREY, Bridger Scientific, Sandwich, Massachusetts, DEBORAH WIEBE, Marine Biocontrol Corp Automatic Control YONG H. KIM, PHIL KISER, Stranco, Inc., Bradley, Illinois New High Resolutio WILLIAM E. GARRETT, Jr., Alabama Power Company, Birmingham, Alabama New High Resolutio JOHN E. ORTH, Metal Samples Company, Munford, Alabama Chlorine Residual ROGER L. STRAND, Stranco, Inc., Bradley, Illinois Chlorine Residual WAYNE B. HUEBNER, Wallace & Tiernan, Belleville, New Jersey Membrane FiltratioDR. MOHAMMAD ALBELDAWI, K. M. SIMMS, S. A. ZAIDI, Wastewater Technology Centre, Rockliffe Rese Study on MinimizaDAISUKE KOJIMA, KUNIO SAEKI, HIDEYUKI MORIYAMA, Sendai Nuclear Power Station, Kyusyu Electric Improved Instrumen TOM DUPNIK, Sybron Chemical, Inc., Birmingham, New Jersey Improved Instrumen JOHN W. KAAKINEN, Chemetek, Portland, Oregon, AMOS J. COLEMAN, USA Tank Automotive Command Reverse Osmosis GREG BARTLEY, Tennessee Valley Authority, Chattanooga, Tennessee Reverse Osmosis DAVID H. PAUL, David H. Paul, Inc., Farmington, New Mexico Feed System Devel LEROY E. RUGG, PETER A. BROWN, Calgon Corporation, Pittsburgh, Pennsylvania, THOMAS C. KUECH Chlorine Dioxide- LINDA RUSZNAK, GEORGE MINCAR, NANCY SMOLIK, Ashland Chemical – Drew Division, Boonton, New Biocidal Efficacy PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, Colorado Biocidal Efficacy RALPH KAJDASZ, Chemlink Division of BPCI a Baker Hughes Company, Houston, Texas A Simplified Meth J. W. SIEGMUND, Sheppard T. Powell Associate, Baltimore, Maryland A Simplified Meth JAMES BENKE, City Public Service, San Antonio, Texas The Stability of Az M. T. COSCHIGANO, W. S. GO, Drew Industrial Division, Ashland Chemical, Inc., Boonton, New Jersey New Corrosion InhiK. FIX, L. SCERBO, Drew Industrial Division, Ashland Chemical Inc., Boonton, New Jersey Microprocessor-Bas GEORGE C. SOLYMOSI, Capital Controls Company, Inc., Colmar, Pennsylvania, DAVID LOWTHER, Ontar Recent ExperienceWILLIAM M. HANN, S. TABB ROBERTSON, JUDY H. BARDSLEY, Rohm and Haas Company, Philadelphia

Evaluation of RedoWALTER ZABBAN, Chester Environmental, Pittsburgh, Pennsylvania Evaluation of RedoYONG H. KIM, Stranco, Inc., Bradley, Illinois Desalination PlantJANUSZ SIKORA, Debiensko Coal Mine, Katowice, Poland, KRZYSZTOF, SZYNDLER, Energotechnika, K Disposal of MembrLOU OBRADOVICH, Eichleay Engineers, Pittsburgh, Pennsylvania Disposal of MembrROBERT L. HAMILTON, Hamilton Engineering, Inc., Denver, Colorado, SUSAN L. COULTER, U.S. Filter/IW Ceramic Microfilte BRIAN D. BURRIS, JEFFERY J. PETERS, U.S. Filter/IWT, Rockford, Illinois Treatment of SecoJ. D. DARJI, P. TEMPLE BALLARD, Infilco Degremont, Inc., Richmond, Virginia Water Recycling Sy HIDEAKI KUROKAWA, TOSHIO SAWA, Energy Research Laboratory, Hitachi Ltd., Hitachi-shi, Ibaraki, Jap Emerging Filter T ASHOK JIM MATHEWS, Charlotte, Carolina PASRICHA, JOHNNorth WILSON, PSE&G, Hancocks Bridge, New Jersey, Emerging Filter T PHIL D’ANGELO, EPRI, Philadelphia, Pennsylvania BWR Condensate PETER Pol MEYERS, PWT Americas, City of Industry, California BWR Condensate JOSEPH Pol GIANNELLI, Finetech, Inc., Mountain Lakes, New Jersey, PAM COOPER, ROB HILLMAN, Oyster Anion Resin UnderSUE HOBART, Adams & Hobart, Fremont, California Anion Resin UnderDOMINIC R. D’ANGELO, Susquehanna Steam Electric Station, Berwick, Pennsylvania, DAVID J. MORGAN Anion Resin UnderDOMINIC R. D’ANGELO, Susquehanna Steam Electric Station, Berwick, Pennsylvania, DAVID J. MORGAN Recent Improvemen RONALD G. BYERS, The Dow Chemical Company, Midland, Michigan Recent Improvemen JOSEPH M. RAGOSTA, Graver Chemical, Union, New Jersey Industrial Water JAMES P. McINTYRE, Betz Industrial, Trevose, Pennsylvania Optimise Investmen SATISH N. CHILEKAR, Ion Exchange (India) Ltd., Bombay, Maharashtra, India Day-To-Day Operati LAWRENCE GURNARI, Graver Water Union, New Jersey, ROBERT STANDBERG, Graver Water, Bainbrid Design of a Zero DALBERT Cyrus Rice Consulting Group, Pittsburgh, Pennsylvania, TORRY TVEDT, Jr., Dow Ch RICHARDD.J.OWENS, STRITTMATTER, MARY KAY KAUFMAN, Nalco Chemical Company, Re-Use of ReclaimNaperville, Illinois Steam Blow CleaniJIM POOLE, Powell & Associates, Baltimore, Maryland Steam Blow CleaniCHRISTOPHER BLOCH, Dowell Industrial Services, Kingwood, Texas, FREDRICK MACK, Jr., Duke Power Chemical CleaningGEORGE VERIB, Ohio Edison, Stow, Ohio Chemical CleaningDENNIS A. FREY, Babcock & Wilcox Co., Barberton, Ohio MIC Met the Master ROBERT E. TATNALL, MIC Associates, Inc., Chadds Ford, Pennsylvania MIC Met the Master J. I. BENNETCH, Virginia Power, Richmond, Virginia, C. T. SZYMKE, R. L. CLARK, Calgon Corporation, Pit Choosing and Oper DONALD JOHNSON, Nalco Chemical Company, Naperville, Illinois Choosing and Oper WILLIAM McGRANE, TriOx, Dublin, California, ALAN SMITH, Calgon Corporation, Pittsburgh, Pennsylvania New Condensate Fi AL REVERE, Pacific Gas & Electric, San Francisco, California New Condensate Fi JOHN J. PEARROW, South Carolina Electric & Gas Company, Columbia, South Carolina, THOMAS W. PO Resin Capacity unPHILIP W. RENOUF, Waterscience International Pty. Ltd, Sydney, Australia Resin Capacity unDIANA KEELING, BRUCE TRIPP, Pacific Gas & Electric Co., Avila Beach, California, SALLIE FISHER, Pur Resin Capacity unDIANA KEELING, BRUCE TRIPP, Pacific Gas & Electric Co., Avila Beach, California, SALLIE FISHER, Pur None JOHN A. S. McGLENNON, ERM – New England Inc., Boston, Massachusetts ESCA: an Expert Sy FABIO SIGON, ENEL, Milano, Italy, LUISELLA RICCARDI, CISE Innovative Technology, Segrate, Italy Corrosion Product OTAKAR JONAS, Jonas, Inc., Wilmington, Delaware Corrosion Product S. G. SAWOCHKA, M. E. CLOUSE, NWT Corporation, San Jose, California, B. LARSON, T. ARCHBOLD, M Corrosion Product S. G. SAWOCHKA, M. E. CLOUSE, NWT Corporation, San Jose, California, B. LARSON, T. ARCHBOLD, M Conversion of a HiPRENTISS B. CARTER, Central Illinois Light Company, Peoria, Illinois, K. ANTHONY SELBY, Puckorius & All Amines are No RICK KRICHTEN, Betz Incorporated, Trevose, Pennsylvania All Amines are No WILLIAM TRACE, Calgon Corporation, Pittsburgh, Pennsylvania, CRAIG SZYMKE, Calgon Corporation, Ba Evaluation of the JAMES A. MATTHEWS, Duke Power Company, Charlotte, North Carolina Evaluation of the P. PINACCI, S. BONATO, G. BUZZANCA, CISE, Segrate, Italy, P. V. SCOLARI, R. STEFANONI, ENEL, Pia Field Study – OzonRICHARD C. SCHWARZ, C.L. Schwarz & Company, Pineville, Pennsylvania A Computer-Control RICHARD H. ASHCRAFT, ChemTreat, Inc., Ashland, Virginia Ozone in Cooling W WALTER ZABBAN, Consultant, Pittsburgh, Pennsylvania Ozone in Cooling W ARTHUR J. FREEDMAN, Thomas M. Laronge, Inc., East Stroudsburg, Pennsylvania, THOMAS M. LARON Cleaning the WaterROGER V. LONG, Stone & Webster Engineering Corp. Denver Colorado Cleaning the WaterMICHAEL SICINSKI, Pure Air, Allentown, Pennsylvania, DAVID LAVALLE, Northern Indiana Public Service

Cleaning the WaterMICHAEL SICINSKI, Pure Air, Allentown, Pennsylvania, DAVID LAVALLE, Northern Indiana Public Service Case Study: Wastew TED FOSBERG, Resources Conservation Company, Bellevue, Washington, BILL SWEET, Millar Western P A Practical Guide tJAMES 0. ROBINSON, Betz Industrial, Trevose, Pennsylvania, A. W. FYNSK, Consultant, Wilmington, Dela SPE – Enhanced Coo CARROLL W. STEELE, Jr., Amoco Chemical Company, Wando, South Carolina, MICHAEL A. WINTERS, A Troubleshooting I V. R. DAVIES, Rohm and Haas Company, Rosemont, Illinois Current Reverse ODAVID PAUL, David H. Paul, Inc., Farmington, New Mexico The Purpose and PJ. R. WEBB, A. BANWEG, Nalco Chemical Company, Naperville, Illinois Water Treatment Co TUWON CHANG, SHIN-EE KANG, Samsung Advanced Institute of Technology, SHIM SANG-HEA, Nalco K Investigation of a YASUHIRO NOHATA, Hakuto Co., Ltd., Tokyo, Japan, HIROSHI TAGUCHI, Mie University, Mie, Japan Study of CorrosionK. K. NANDWANA, D. K. SHAH, R. N. PHATAK, G. MARIMUTHU, V. C. MALSHE, Ion Exchange (India) Lim The Impact of ChloJOHN W. SIEGMUND, Sheppard T. Powell Associates, Baltimore, Maryland The Impact of ChloREIMER HOLM, DAVID BERG, Miles, Inc., Pittsburgh, Pennsylvania, FRANK LU, DON JOHNSON, Nalco C The Impact of ChloREIMER HOLM, DAVID BERG, Miles, Inc., Pittsburgh, Pennsylvania, FRANK LU, DON JOHNSON, Nalco C Molybdate-Based S J. FRED WILKES, Consultant, Titusville, Florida, FLAVIO BIANCHI, MESSIAS C. AMARAL, REGINA M. C. Current Pretreatm FRANK MORENSKI, Graver Water, Union, New Jersey Operational ExperiDAVID C. AUERSWALD, Southern California Edison, Paramount, California RO Systems TroubPAUL L. PARISE, Aquagenics, Inc., Londonderry, New Hampshire RO Concentrate Us CHRISTOPHER SOULE, (ROBERT BRADLEY), Arrowhead Industrial Water, Lincolnshire, Illinois, DAN STO An Investigation o J. IRVING, J. A. DALE, Purolite International Limited, Pontyclun, Wales Sulfate Sloughage:SALLIE FISHER, EUGENE BURKE, GERARD OTTEN, Puricons, Inc., Malvern, Pennsylvania Investigation of H D. M. BLOOM, Nalco Chemical Company, Naperville, Illinois, J. O. ROBINSON, Betz Industrial, Trevose, Pe New Non-Hazardous HERMAN HAMANN, MARK BERTLER, Arrowhead Industrial Water, Inc., Lincolnshire, Illinois, RON MADDE Diagnosing and Tro MIKE WINTERS, Amoco Research, Naperville, Illinois Diagnosing and Tro IAN E. EISNER, Nalco Chemical Company, Naperville, Illinois HTP-2, a New IronTORRY TVEDT, Dow Chemical Company, Freeport, Texas HTP-2, a New IronSCOTT M. BOYETTE, Betz Industrial, Trevose, Pennsylvania, FREDERICK A. ELLIOT, IMC Fertilizer, Sterl New RO Pretreatmen BRUCE L. COULTER, WAYNE BATES, ROBERT MARAVICH, Illinois Water Treatment, Inc., Rockford, Illino Reverse Osmosis R OOBERT M. QUINN, RQ Associates Inc., Teaneck, New Jersey Makeup Water Treat PETER CARTWRIGHT, Cartwright Consulting Company, Minneapolis, Minnesota Makeup Water Treat GARY COKER, TOM WILLIAMSON, Entergy Operations, Inc., Port Gibson, Mississippi, KEITH SIMS, LI ZH Lime Treatment vs.WILLIAM J. CONLON, Camp, Dresser & McKee, Inc., Ontario, California Lime Treatment chúng tôi LIBERATORE, JEFF CAMPBELL, Desalination Systems, Inc., Escondido, California Lime Treatment chúng tôi LIBERATORE, JEFF CAMPBELL, Desalination Systems, Inc., Escondido, California Study on the DeterSTUART McCLELLAN, Dow Chemical Company, West Palm Beach, Florida Study on the DeterYUICHI YOKOMIZO, Japan Organo Co., Ltd., Tokyo, Japan Study on the DeterYUICHI YOKOMIZO, Japan Organo Co., Ltd., Tokyo, Japan Cloth1ng a Naked ALROY U ASCHOFF, Sargent & Lundy Engineers, Chicago, Illinois Cloth1ng a Naked LESTER U C. WEBB, Black & Veatch, Kansas City, Missouri, RONALD W. TOMLIN, Mcintosh Power Plant, L Cloth1ng a Naked LESTER U C. WEBB, Black & Veatch, Kansas City, Missouri, RONALD W. TOMLIN, Mcintosh Power Plant, L Ammonia Form Oper MIKE SADLER, NEI Thompson Kennicott, West Midland, England Ammonia Form Oper FRANK McCARTHY, Moneypoint Power Station, Kilrush, County Clare, Ireland Impact of a Novel CLARENCE D. COLLEY, Boeing Aerospace, Huntsville, Alabama Impact of a Novel BILL KRATZER, Star Enterprise, Port Arthur, Texas, STEVEN COKER, Dow Chemical U.S.A., Freeport, Tex Operating ExperieMICHAEL GOTTLIEB, ResinTech, Inc., Cherry Hill, New Jersey, KEN FLEGLE, Energy Systems, Omaha, N EPRI Oxygenated BUD F HERRE, Pennsylvania Power & Light, Allentown, Pennsylvania EPRI Oxygenated S. F RONNIE PATE, Georgia Power Company, Atlanta, Georgia, CALVIN E. TAYLOR, RANDY C. TURNER, T EPRI Oxygenated S. F RONNIE PATE, Georgia Power Company, Atlanta, Georgia, CALVIN E. TAYLOR, RANDY C. TURNER, T TU Electric Tradin TOM GILCHRIST, Colorado-Ute Electric, Craig, Colorado TU Electric Tradin MIKE WADLINGTON, MARLIN EARLY, CHARLIE HENRY, TU Electric, Dallas, Texas Oxygenated WaterBOB PALMER, T. J. WYSOCKI, Detroit Edison, Detroit, Michigan Oxygenated WaterGEORGE J. VERIB, Ohio Edison/CMAC, Stow, Ohio, J. E. NEIDHARDT, Ohio Edison, Stratton, Ohio, W. E

Oxygenated Treatme JAMES A. MATTHEWS, Duke Power Company, Charlotte, North Carolina Oxygenated Treatme BARRY DOOLEY, EPRI, Palo Alto, California, B. LARKIN, L. WEBB, Black & Veatch, Kansas City, Missouri Oxygenated Treatme BARRY DOOLEY, EPRI, Palo Alto, California, B. LARKIN, L. WEBB, Black & Veatch, Kansas City, Missouri Pre-Treatment of RWILLIAM L. TRACE, Calgon Corporation, Pittsburgh, Pennsylvania Pre-Treatment of RS. K. CHATTERJEE, (V. N. SAHAKARI), Zuari Agro Chemicals Limited, Zuarinagar, Goa, India 1940’s Power PlantMIKE DALTON, ChemLink, Allison Park, Pennsylvania 1940’s Power PlantLEYON O. BRESTEL, Aquatec Chemical International, Inc., Montrose, Colorado, DENNIS COUSINO, Mid 1940’s Power PlantLEYON O. BRESTEL, Aquatec Chemical International, Inc., Montrose, Colorado, DENNIS COUSINO, Mid The Design Method ROBERT HART, Conoco, Ponca City, Oklahoma, ROBERT O’CONNELL, Graver Water, Union, New Jersey The Use of Anion EMICHAEL C. GOTTLIEB, ResinTech, Cherry Hill, New Jersey The Use of Anion EJAMES M. SYMONS, University of Houston, Houston, Texas, PAUL L-K FU, CH2M Hill, Santa Ana, Californ The Use of Anion EJAMES M. SYMONS, University of Houston, Houston, Texas, PAUL L-K FU, CH2M Hill, Santa Ana, Californ Polyacrylamide Bas JASBIR S. GILL, Calgon Corporation, Pittsburgh, Pennsylvania Polyacrylamide Bas KATSUHIKO MOMOZAKI, MAYUMI KIRA, YASUSHI MURANO, MASARU OKAMOTO, FUMIO KAWAMUR Practical Applicati KENNETH R. WEISS, THOMAS R. GUENTHER, Black & Veatch, Overland Park, Kansas, CLIDE M. FORT The Effects of CooPAUL PUCKORIUS, Puckorius & Associates, Evergreen, Colorado The Effects of CooE. HOBSON, G. A. FITCHETT, National Power PLC, Harrogate, United Kingdom Condenser CoolingMARTIN W. PATE, Calgon Corporation, Friendswood, Texas, BILLY G. MARTIN, Houston Lighting and Pow Cooling Water ScaBILLY FELLERS, CHARLES HENRY, Texas Utilities Generating Company, Glen Rose, Texas Cooling Water ScaWILFRED OLSON, BRIAN VESTBY, SaskPower, Boundary Dam Power Station, Saskatchewan, Canada Cooling Water ScaWILFRED OLSON, BRIAN VESTBY, SaskPower, Boundary Dam Power Station, Saskatchewan, Canada Optimize Design oPRABHAT KUMAR SINHA, ALEXANDER V. SUPERFIN, RONALD P. BOYD, Bechtel Corporation, Gaithers Conquering UniqueALBERT D. OWENS, Cyrus Rice Consulting Group, Pittsburgh, Pennsylvania, TORRY TVEDT, Jr., Dow Ch Alcan Sebree WateBOB ROSAIN, CH2M Hill, Bellevue, Washington Alcan Sebree WateMORRIS LEE, Alcan Ingot, Henderson, Kentucky, LOIS NEIL, Calgon Corporation, Evansville, Indiana PPQ Water – The NMICHAEL A. SADLER, ROY BOLTON, E. K. BULLAS, Thompson Kennicott, Rolls Royce Nuclear Engineer Analysis of NonvolNAOHITO UETAKE, HIDEAKI KUROKAWA, TOSHIO SAWA, Energy Research Laboratory, Hitachi, Ltd., Ib Kinetic Evaluation PETER S. MEYERS, L*A Water Treatment Corporation, City of Industry, California Kinetic Evaluation RONALD N. WHITE, BRIAN D. BURNS, THOMAS K. MALLMANN, Illinois Water Treatment, Inc., Rockford, None MARK S. SINGEL, Pennsylvania Lieutenant Governor Crud Removal Char TAKESHI IZUMI, HIDEO KAWAZU, MASAHIRO HAGIWARA, KAZUMI MAEHARA, Ebara Corporation, Kan Use of Elevated p J. MICHAEL HOSEA, Bio-Recovery Systems, Inc., Las Cruces, New Mexico Use of Elevated p WAYNE CICCARELLI, Westinghouse Nuclear Services Division, Pittsburgh, Pennsylvania, JOEL McELRAT Use of Elevated p WAYNE CICCARELLI, Westinghouse Nuclear Services Division, Pittsburgh, Pennsylvania, JOEL McELRAT Deionized Water foM. B. YELIGAR, Permutit Company, Inc., Warren, New Jersey Deionized Water foPETER MEYERS, L*A Water Treatment Corporation, City of Industry, California Recent Improvemen CARL C. SCHEERER, Central Illinois Public Service Company, Springfield, Illinois Recent Improvemen MICHAEL J. O’BRIEN, The Graver Company, Union, New Jersey The Optimization FRANCIS CUTLER, Southern California Edison Company, Paramount, California The Optimization J. IRVING, Purolite International Limited, Pontyclun, Wales Turn-Key ChemicalEDWARD W. EKIS, Jr., Nalco Chemical Company, Naperville, Illinois, JOSEPH A. KOWALEWSKI, KENNET In-Place Chemical DENNIS JONES, Monongahela Power Company, Willow Island, West Virginia, RALPH E. MICKEL, Epicor, Corrosion, CleaninANGELA ERVIN, KATHLEEN HARDY, Naval Research Laboratory, Washington, DC Neutral pH Process ARTHUR J. FREEDMAN, Thomas M. Laronge, Inc., East Stroudsburg, Pennsylvania Neutral pH Process KURT D. HEINZ, Dearborn Division, W.R. Grace & Company, Lake Zurich, Illinois, J. A. GRAY, Dearborn Ch Copper Problems aW. B. WILLSEY, W.B. Willsey Associates Inc., Cherry Hill, New Jersey Copper Problems aTHOMAS FITZSIMMONS, Basin Electric Power Cooperative, Bismarck, North Dakota, BRIAN LARSON, Ba Computerized Ion ROBERT J. FERGUSON, French Creek Software, Inc., Malvern, Pennsylvania Water Treatment wiGERALD F. CONNELL, BLAIR JONES, Capital Controls Company, Inc.Wolverhampton, Colmar, Pennsylvania ROY BOLTON, CEDRICW. MARSON, NEI Thompson Kennicott, Ettingshall, West Midlands, Maintaining Effici Sugar Land, Texas Case History – Inv PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, Colorado, TAMMY O. BRICE, Gulf States

Electron MicroprobD. HARTWICK, Dearborn Canada, Mississauga, Ontario, Canada, J. RICHARDSON, D. LITTLE, Dearborn Practical ApproachJ. FRED WILKES, Consulting Chemical Engineer, Titusville, Florida, RENATO ARAUJO DA SILVA, Aquatec An EnvironmentallyANON An EnvironmentallyDAVE KARLOVICH, JOE KOSTYAL, Betz Industrial, Trevose, Pennsylvania, LAURA WEBER, GPU Three M An EnvironmentallyDAVE KARLOVICH, JOE KOSTYAL, Betz Industrial, Trevose, Pennsylvania, LAURA WEBER, GPU Three M Cooling Tower BiofCATHERINE BOLIN, Duke Power Company, Clover, South Carolina, GARY WARD, Duke Power Company, Strategies for the RICHARD W. LUTEY, Buckman Laboratories International, Memphis, Tennessee, PAMELA J. ALLISON, Bu Why Morpholine JOHN E. KRISTENSEN, New York Power Authority, Buchanan, New York Powdered Ion Exch BRUCE L. LIBUTTI, KURT BOZENMAYER, Graver Chemical, Division of the Graver Company, Union, New The Impact of a P M. A. SADLER, Consultant, Bristol, England The Impact of a P WILLIAM L. TRACE, JOHN D. ZUPANOVICH, JOHN W. BISH, Calgon Corporation, Pittsburgh, Pennsylvan Life Prediction o PHILIP J. D’ANGELO, Philadelphia Electric Company, Wayne, Pennsylvania Life Prediction o ICHIRO INAMI, Toshiba Corporation, Kawasaki, Japan, YOSHITAKE MORIKAWA, KATSUJI MAEDA, Toshi Life Prediction o ICHIRO INAMI, Toshiba Corporation, Kawasaki, Japan, YOSHITAKE MORIKAWA, KATSUJI MAEDA, Toshi Approach to ChemiJ. J. BURCHILL, E. J. FULLER, Drew Division of Ashland Chemical, Boonton, New Jersey An Experimental StJAMES C. BELLOWS, Westinghouse Electric Corporation, Orlando, Florida An Experimental StJ. M. SIMONSON, DONALD A. PALMER, Oak Ridge National Laboratory, Oak Ridge, Tennessee An Experimental StJ. M. SIMONSON, DONALD A. PALMER, Oak Ridge National Laboratory, Oak Ridge, Tennessee Global Treatment fHAROLD CHAGNARD, The Dow Chemical Company, Plaquemine, Louisiana History Case – GloCLAUDIA RIBEIRO, SAMUEL SID, Copesul-Companhia Petroquimica Do Sul, Triunfo, RS, Brasil Experience of ReaJOHN M. RIDDLE, Halliburton NUS Environmental Corporation, Pittsburgh, Pennsylvania Experience of ReaKATSUJI MAEDA, MASAHIRO NAKAMURA, Toshiba Corporation, Yokohama, Japan, KATSUMI NAGASAW Experience of ReaKATSUJI MAEDA, MASAHIRO NAKAMURA, Toshiba Corporation, Yokohama, Japan, KATSUMI NAGASAW Tapered Mixing ImYONG H. KIM, CARL L. BRAZELTON, Stranco, Inc., Bradley, Illinois Selective Ion ExchF. X. McGARVEY, D. TAMAKI, Sybron Chemicals, Inc., Birmingham, New Jersey Water Purification HIDEAKI KUROKAWA, TOSHIO SAWA, Energy Research Laboratory, Hitachi Ltd. Ibaraki, Japan, NOBUAT Toxics Reduction WALTER ZABBAN, The Chester Engineers, Pittsburgh, Pennsylvania Toxics Reduction DANIEL C. FINN, THOMAS J. O’TOOLE, DENNIS R. SHAFER, The Chester Engineers, Pittsburgh, Pennsy The Treatment andEDGAR G. PAULSON, Edgar G. Paulson, P.E., Richmond , Virginia The Treatment andMAREK K. MIERZEJEWSKI, Infilco Degremont Inc., Richmond, Virginia The Treatment andMAREK K. MIERZEJEWSKI, Infilco Degremont Inc., Richmond, Virginia Advances in the UERIC V. MAUGHAN, Krohne (Pty) Ltd., Halfway House, South Africa, GERHARD GERICKE, GERRIT W. LO Iodine-Bias Analyze EARL L. HENN, Calgon Corporation, Pittsburgh, Pennsylvania Iodine-Bias Analyze WAYNE B. HUEBNER, Wallace & Tiernan, Belleville, New Jersey The Use of a ContDAVID F. PENSENSTADLER, Halliburton NUS Environmental Corp., Pittsburgh, Pennsylvania The Use of a ContKAJ D. RONDUM, WINSTON S. GO, Drew Industrial Division, Ashland Chemical, Inc., Boonton, New Jerse The Use of a ContKAJ D. RONDUM, WINSTON S. GO, Drew Industrial Division, Ashland Chemical, Inc., Boonton, New Jerse New Electrode PairMARTIN S. FRANT, CHARLES S. BAER, Orion Research, Inc., Boston, Massachusetts Carbon Dioxide and JAN STODOLA, Ontario Hydro, Toronto, Ontario, Canada Carbon Dioxide and ALBERT BURSIK, Grosskraftwerk Mannheim AG, Mannheim, Germany Carbon Dioxide and ALBERT BURSIK, Grosskraftwerk Mannheim AG, Mannheim, Germany Laboratory Scale M JAMES K. RICE, James K. Rice, Chartered, Olney, Maryland Laboratory Scale M OTAKAR JONAS, Jonas, Inc., Wilmington, Delaware Laboratory Scale M A. C. McDONALD, D. J. KOTWICA, S. B. PRUETT, B. L. TRACY, Betz Laboratories, Inc., The Woodlands, Boiler Pressure D GEORGE J. VERIB, Ohio Edison, Stow, Ohio Boiler Pressure D F. SIGON, ENEL/CRTN, Milano Italy, G. QUADRI, ENEL/Construction Department, Piacenza, Italy Boiler Pressure D F. SIGON, ENEL/CRTN, Milano Italy, G. QUADRI, ENEL/Construction Department, Piacenza, Italy Future Direction o DONALD D. GOLDSTROHM, Salt River Project, Saint Johns, Arizona Future Direction o A. F. ASCHOFF, Sargent & Lundy, Chicago, Illinois, R. B. DOOLEY, Electric Power Research Institute, Palo Remote MonitoringCHARLES M. CUMMINGS, Fluid Systems Corporation, San Diego, California Remote MonitoringWILLIAM S. MILLER,, Ecolochem, Inc., Norfolk, Virginia, BASIL L. BELSCHES, ED BRENNAN, Virginia Po

Remote MonitoringWILLIAM S. MILLER, Ecolochem, Inc., Norfolk, Virginia, BASIL L. BELSCHES, ED BRENNAN, Virginia Po Benefits of ReplacCHARLIE HENRY, T.U. Electric, Dallas, Texas, STEVE PARKER, T.U., Electric – Stryker Creek SES, Jackso Triple Membrane MTIM BASHFORD, Hewlett-Packard, Boise, Idaho Triple Membrane MHENRY C. VALCOUR, Jr., Ionics, Incorporated, Watertown, Massachusetts Desalting Brackis MICHAEL L. WISDOM, Destec Energy, Inc., Houston, Texas, PHIL V. QUALLS, West Texas Utilities Co., Ve Operating ExperienRICK LESAN, Hydranautics, San Diego, California Operating ExperienDAVID B. MALKMUS, South Carolina Electric & Gas Company, Jenkinsville, South Carolina, JAMES C. CH Zinc Based CorrosiJEFFREY STEWART, Pennsylvania Power Company, Shippingport, Pennsylvania Surface Analytical REIMER HOLM, DAVID A. BERG, Mobay Corporation, Pittsburgh, Pennsylvania, JOHANNES EICKMANS, New Pitting CorrosDAN P. VANDERPOOL, SUSAN P. REY, Calgon Corporation, Pittsburgh, Pennsylvania The Synthesis of OWANG ZU-MO, CHEN KANG, East China University of Chemical Technology, Shanghai, People’s Republic ECLSS Water Proce CLARENCE D. COLLEY, Boeing Defense and Space Group, Huntsville, Alabama Design and Start-UJULIUS ISAAC, JOSE P. LOZADA, Ebasco Services Inc., New York, New York Ammonia Regenerat DAVID C. AUERSWALD, Southern California Edison, Paramount, California Ammonia Regenerat JOSEPH F. SELANN, DAVID BROMLEY Engineering (1983) Ltd., Edmonton, Alberta, Canada, DILIP P. DE First Year’s Opera KENNETH H. FREDERICK, Ion Exchange Associates, Inc., Reading, Pennsylvania First Year’s Opera GEORGE C. FLYNN, The Permutit Company, Inc., Warren, New Jersey, T. K. SPAULDING, PSI Energy Inc None RITA SCHMIDT SUDMAN, Water Education Foundation, Sacramento, California A Performance DriHOWARD KLEE Jr., Amoco Oil Company, Whiting, Illinois, MICHAEL A. WINTERS, Amoco Research Cent Chemical ChangesKAROL i DAUCIK, Skaerbaekvaerket I/S, Fredericia, Denmark Problems in the UsMICHAEL W. VERA, Mt. Poso Operating Company, Bakersfield, California High-Rate Sub-Micr DANIEL J. KELLY, ALAIN BLAIS, Sonitec, Inc., Holyoke, Massachusetts Continuous On-LinMIKE E. ROGERS, ANANTH VENKATRAMAN, DAVE BRYENTON, Syncrude Canada Ltd., Fort McMurray, An ASTM D2777-86 Dr. ROBERT J. FAUST, Calgon Corporation, Pittsburgh, Pennsylvania An ASTM D2777-86 BEN C. EDMONDSON, Inquiry Computer Systems, Ltd., San Louis Obispo, California, L. W. STANTON, Ca Steam Cycle Passiv J. A. KELLY, C. M. HWA, J. C. FAN, Dearborn Division, W.R. Grace & Company, Lake Zurich, Illinois, K. L. R A Unique EmpiricalEVERETT J. FULLER, Consultant, Drew Industrial Division, Boonton, New Jersey, DOUGLAS B. DEWITT-D Cycle Chemistry anF. SIGON, C. ZAGANO, ENEL/DSR-CRTN, Milano, Italy, G. QUADRI, ENEL/DCO-ULP, Piacenza, Italy Influence of Disso SHIRO TAYA, KATSUHIKO MOMOZAKI, Kurita Water Industries, Tokyo, Japan Improved Diluter MFRANCIS M. CUTLER, Southern California Edison, Paramount, California Improved Diluter MC. RICHARD NOLAN, DAVID F. PENSENSTADLER, NUS Corporation, Pittsburgh, Pennsylvania Automation of TotaJAN STODOLA, Ontario Hydro, Toronto, Ontario, Canada Automation of TotaMASATAKA SUDA, Kyushu Electric Power Co., Ltd., Tokyo, Japan, YASUTO FUTAGOISHI, Nikkiso Compa Automation of TotaMASATAKA SUDA, Kyushu Electric Power Co., Ltd., Tokyo, Japan, YASUTO FUTAGOISHI, Nikkiso Compa Installation of a WILLIAM GREENAWAY, Bechtel Savanah River, Inc., North Augusta, South Carolina Installation of a NORMAN SEVRIN, HARRY MILLER, Public Service Electric & Gas Company, Hancocks Bridge, New Jerse Project Discovery TORRY J. TVEDT, Dow Chemical Company, Freeport, Texas Project Discovery JANE E. ANTOINE, GARY C. COKER, STEVEN D. LEE, G. O. SMITH, Entergy Operations, Port Gibson, M Project Discovery-JANE E. ANTOINE, GARY C. COKER, STEVEN D. LEE, G. O. SMITH, Entergy Operations, Port Gibson, M Microprocessor BaDENNIS SHEA, Monsanto Chemical Company, Alvin, Texas Microprocessor BaDAN MORRIS, Morr Control, Inc., Muskogee, Oklahoma, JOANNE KUCHINSKI, Drew Industrial Division, B CHRISTINE STUART, Nalco Chemical Company, Naperville, Illinois Application of a S BYRON PERRIGO, RAYMOND M. POST, Betz Industrial, Trevose, Pennsylvania, THOMAS CLAY, Betz Eq The Impact of ComARTHUR J. FREEDMAN, Thomas M. Laronge, Inc., Califon, New Jersey The Impact of ComSIDNEY T. COSTA, MEI H. HWANG, CHARLES J. McCLOSKEY, Calgon Corporation, Pittsburgh, Pennsylv The Impact of ComSIDNEY T. COSTA, MEI H. HWANG, CHARLES J. McCLOSKEY, Calgon Corporation, Pittsburgh, Pennsyl Corrosion Rate and JOE LUX, Consultant, Massillon, Ohio Corrosion Rate and PIETRO PINACCI, MARCO FERRARI, G. BUZZANCO, CISE Technologie Innovative Spa., Segrate, Italy, P Makeup TreatmentK.f ANTHONY SELBY, Puckorius & Associates, Inc., Evergreen, Colorado Makeup TreatmentKEVIN f J. SHIELDS, DAVID A. CLINE, Jr., Sheppard T. Powell Associates, Baltimore, Maryland, R. BARRY Makeup TreatmentKEVIN f J. SHIELDS, DAVID A. CLINE, Jr., Sheppard T. Powell Associates, Baltimore, Maryland, R. BARRY

Combined Water TrJAMES A. MATTHEWS, Duke Power Company, Charlotte, North Carolina Combined Water TrG. QUADRI, P. V. SCOLARI, ENEL-DCO-ULP, Piancenza, Italy, G. P. SCALARI, ENEL-DPT, Pisa, Italy, F. S Combined Water TrG. QUADRI, P. V. SCOLARI, ENEL-DCO-ULP, Piancenza, Italy, G. P. SCALARI, ENEL-DPT, Pisa, Italy, F. S International WateALBERT BURSIK, Grosskraftwork Mannheim AG, Federal Republic of Germany International WateOTAKAR JONAS, Jonas, Inc., Wilmington, Delaware, BARRY DOOLEY, Electric Power Research Institute, International WateOTAKAR JONAS, Jonas, Inc., Wilmington, Delaware, BARRY DOOLEY, Electric Power Research Institute, New High Velocity BURNETT SCHNEIDER, RICHARD A. RIDDLE, Aqua-Chem, Inc., Milwaukee, Wisconsin A New Microfiltra RANDOLPH L. TRUBY, UOP Fluid Systems Corporation, San Diego, California A New Microfiltra G. R. GROVES, Epoc Water, Inc., Fresno, California Ultrafiltration wi PETER DOWN, Romicon, Woburn, Massachusetts Ultrafiltration wi RICHARD E. IDE, DANIEL L. COMSTOCK, Desalination Systems, Inc., Escondido, California Approaches to Tw WILLIAM J. CONLON, Stone & Webster Technology Services, Fort Lauderdale, Florida Approaches to Tw LEE COMB, ANITA TUCH, Osmonics, Inc., Minnetonka, Minnesota A Report on the UsNORIYUKI V. C. MALSHE, B. B.TAKAO SHAHI,BABU, S. N. CHILEKAR, S. S. SHIRODKAR, Ion MAEDA, ExchangeSACHIO (India) Ltd., SASAKI, SHINICHI YAMAGUCHI, MATSUJI DOL,Bombay, ToshibaInd Co Operational ExperiTokyo, Japan Field PerformanceF. X. McGARVEY, Sybron Chemicals, Inc., Birmingham, New Jersey, M. MOLDOFSKY, Sybron Chemicals, Optical Sizing vs. GERARD OTTEN, SALLIE FISHER, Puricons, Inc., Malvern, Pennsylvania Selective SorbentsSHELTON. A. DIAS, Ontario Hydro Research Division, Toronto, Ontario, Canada, BABU R. NOTT, Electric P Control of Iron andWILLIAM M. HANN, S. T. ROBERTSON, Rohm and Haas Company, Spring House, Pennsylvania Successful BrominERNEST SANGER, Indianapolis Power & Light, Indianapolis, Indiana, GERALD F. CONNELL, Capital Cont Effectively Disper J. MIKE BROWN, Betz Laboratories, Inc., The Woodlands, Texas, JIM POLITO, Betz industrial, Cincinnati, Laboratory Methods BRUCE W. VIGON, DAVID P. EVERS, Battelle Memorial Institute, Columbus, Ohio, WAYNE C. MICHELET A Portable Test FaWAYNE C. MICHELETTI, Electric Power Research Institute, Palo Alto, California, CHARLES D. HARDY, He Controlling Sludg BOSCO HO, Calgon Corporation, Pittsburgh, Pennsylvania Controlling Sludg JAMES B. CARLING, Milton Roy Company, Ivyland, Pennsylvania, KHAI TRAN, Ashbrook-Simon-Hartley, H Treatment of McIntWALTER ZABBAN, Consultant, Pittsburgh, Pennsylvania Treatment of McIntCHARLES D. GARING, McIntosh Power Plant, Lakeland, Florida, LESTER C. WEBB, KENNETH R. WEISS Treatment of McIntCHARLES D. GARING, McIntosh Power Plant, Lakeland, Florida, LESTER C. WEBB, KENNETH R. WEISS An On-Site Chemica ROBERT CHIESA, RMT Inc., Madison, Wisconsin An On-Site Chemica STANLEY B. McCONNELL, Dowell Schlumberger, Tulsa, Oklahoma, JAMES RUCK, Dowell Schlumberger, An On-Site Chemica STANLEY B. McCONNELL, Dowell Schlumberger, Tulsa, Oklahoma, JAMES RUCK, Dowell Schlumberger, Treatment of a HazROBERT M. ROSAIN, CH2M Hill, Bellevue, Washington Treatment of a HazRANDALL K. DRAZBA, Floyde Browne Associates, Inc., Marion, Ohio, GARY P. FRITSCH, Zimpro/Passava Comparison of Ope CLARENCE COLLEY, Boeing, Huntsville, Alabama Comparison of Ope MICHAEL J. KASZYSKI, KENNETH L. FULFORD, AT&T Microelectronics, Allentown, Pennsylvania, MICHA Impact of Particle BRIAN HOFFMAN, Rohm and Haas Company, Philadelphia, Pennsylvania Impact of Particle DANIEL B. RICE, The Dow Chemical Company, Midland, Michigan Impact of Particle DANIEL B. RICE, The Dow Chemical Company, Midland, Michigan Development of a R JERRY GUTER, Boyle Engineering, Bakersfield, California Development of a R L. S. GOLDEN, J. IRVING, Purolite International Limited, Pontyclun, Wales Development of a R L. S. GOLDEN, J. IRVING, Purolite International Limited, Pontyclun, Wales Major Advances ofPAUL PUCKORIUS, Puckorius & Associates, Evergreen, Colorado Major Advances ofQINAI BAO, Jinling Petrochemical Corporation, Nanjing, Jiangsu, China Economics and Per TOM LARONGE, Thomas M. Laronge, Inc., Vancouver, Washington Economics and Per ALAN PRYOR, National Water Management Corp., San Jose, California, MICHAEL BUKAY, Michael Bukay Macrofouling ContrJOHN F. GAREY, Marine Biocontrol, Sandwich, Massachusetts Macrofouling ContrMICHAEL G. TRULEAR, EDWARD W. EKIS, Nalco Chemical Company, Naperville, Illinois, GEORGE ELLI Current Cooling Wa ALAN M. YEOMAN, Ciba-Geigy Ardsley, New York Current Cooling Wa JOHN P. TERRY, CRYSTAL W. YATES, Buckman Laboratories, Memphis, Tennessee Ten Years ExperienJOHN UNGAR, Dianex Systems, Inc., Lockport, New York Ten Years ExperienZHANG SHUCHUN, Tianjin Petrochemical Corp., Tianjin, China, JOHN C. KONEN, Rohm and Haas, Philad

Is Total Capacity F. X. McGARVEY, M. E. CHILDS, R. GONZALEZ, Sybron Chemicals, Inc., Birmingham, New Jersey Is Total Capacity F. X. McGARVEY, M. E. CHILDS, R. GONZALEZ, Sybron Chemicals, Inc., Birmingham, New Jersey Are all Type I Str STEPHEN C. FOOR, Illinois Water Treatment Company, Rockford, Illinois Are all Type I Str SALLIE FISHER, GERARD OTTEN, Puricons, Inc., Malvern, Pennsylvania Are all Type I Str SALLIE FISHER, GERARD OTTEN, Puricons, Inc., Malvern, Pennsylvania Ion Exchange, PasDr. ROBERT KUNIN, Consultant, Yardley, Pennsylvania Removal of OrganiMICHAEL J. O’BRIEN, The Graver Company, Union, New Jersey Removal of OrganiABU NURMOHAMED, Partec Lavalin, Inc., Calgary, Alberta, Canada, ROGER COWLES, Syncrude Canad Removal of OrganiABU NURMOHAMED, Partec Lavalin, Inc., Calgary, Alberta, Canada, ROGER COWLES, Syncrude Canad Case History of a PETER CARTWRIGHT, Cartwright Consulting, Minneapolis, Minnesota Case History of a A. ARSENEAUX, L. STONER, Air Products and Chemicals, Inc., New Orleans, Louisiana, S. WHIPPLE, Do Computer SimulatiGIANNI O. CHIERUZZI, The Chester Engineers, Pittsburgh, Pennsylvania Computer SimulatiJING-YEA YANG, SHI-TAO YEH, Roy F. Weston, Inc., West Chester, Pennsylvania Removal and TreatG. KENT PETERSON, Illinois Water Treatment Company, Rockford, Illinois Removal and TreatTHOMAS F. KLUMPP, The Dow Chemical Company, Midland, Michigan, SAMUEL S. TAWNEY, D. ERIC ST Removal and TreatTHOMAS F. KLUMPP, The Dow Chemical Company, Midland, Michigan, SAMUEL S. TAWNEY, D. ERIC ST Separation and DeFRANCES M. CUTLER, Southern California Edison Company, Paramount, California Separation and DeDANIEL L. CAMPBELL, ROBERT JOYCE, JOHN STILLIAN, STEVE CARSON, Dionex Corporation, Sunny Separation and DeDANIEL L. CAMPBELL, ROBERT JOYCE, JOHN STILLIAN, STEVE CARSON, Dionex Corporation, Sunny Monitoring and CoROBERT MUEHLENKAMP, PAUL SCHUMACHER, Wisconsin Electric Power Company, Milwaukee, Wisco Monitoring and CoWILLIAM L. TRACE, Calgon Corporation, Pittsburgh, Pennsylvania, KEITH H. KEHRER, Delmarva Power, The Direct Measure W. E. ALLMON, C. C. STAUFFER, The Babcock & Wilcox Company, Alliance, Ohio The Direct Measure JEFFREY SCOT HOVIS, Orion Research Incorporated, Boston, Massachusetts Industrial Water Q JAMES K. RICE, Consulting Engineer, Olney, Maryland Erosion Corrosion FRANK J. WITT, U.S. Nuclear Regulatory Commission, Washington, DC Erosion Corrosion NORBERT HENZEL, DENNIS EGAN, Bechtel-KWU Alliance, Gaithersburg, Maryland Erosion Corrosion NORBERT HENZEL, DENNIS EGAN, Bechtel-KWU Alliance, Gaithersburg, Maryland Field Operating ExMANFRED NOACK, Olin Corporation, Cheshire, Connecticut, Field Operating ExKAJ D. RONDUM, EVERETT J. FULLER, DOUGLAS B. DEWITT-DICK, Drew Industrial Division, Ashland C Status of CorrosionR. B. POND, Jr., Baltimore Gas & Electric Company, Baltimore, Maryland Status of CorrosionR. W. PATTERSON, JAN STODOLA, D. SIDEY, D. McNABB, Ontario Hydro, Toronto, Ontario, Canada, A. Cycle Chemistry Re DAVID N. FRENCH, David N. French, Inc., Metallurgists, Northborough, Massachusetts Cycle Chemistry Re BARRY DOOLEY, Electric Power Research Institute, Palo Alto, California The Waste Water C PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, Colorado The Waste Water C WILLIAM G. SANDERSON, RICHARD L. LANCASTER, Zurn/Nepco, Woodinville, Washington Boiler Chemical C STANLEY B. McCONNELL, Dowell Schlumberger Incorporated, Tulsa, Oklahoma Boiler Chemical C THOMAS A. LOTT, WAYNE C. MICHELETTI, Electric Power Research Institute, Palo Alto, California, LARR Boiler Chemical C THOMAS A. LOTT, WAYNE C. MICHELETTI, Electric Power Research Institute, Palo Alto, California, LARR Operation of SEM MARK MANZIONE, Brown & Caldwell Consulting Engineers, Walnut Creek, Operation of SEM PRABHAT KUMAR SINHA, LAWRENCE J. GASPER, Bechtel Corporation, Gaithersburg, Maryland, GREG Operation of SEM PRABHAT KUMAR SINHA, LAWRENCE J. GASPER, Bechtel Corporation, Gaithersburg, Maryland, GREG The Impact of WatWINSTON CHOW, Electric Power Research Institute, Palo Alto, California The Impact of WatROBERT M. ROSAIN, CH2M Hill, Bellevue, Washington How to Inspect IndALLEN E. FEITZIN, Airco Industrial Gases, Murray Hill, New Jersey How to Inspect IndJAMES L. WILLA, Willa, Inc., St. Louis, Missouri Manganese/Aminoph JOSEPH S. ROTI, Drew Industrial Division, Boonton, New Jersey Manganese/Aminoph T. J. TVEDT, Jr., C. A. JONES, J. G. GRIERSON, The Dow Chemical Company, Freeport, Texas, L. JONES Cooling Water TechM. A. WINTERS, Amoco Corporation, Naperville, Illinois Cooling Water TechJACK V. MATSON, University of Houston, Houston, Texas Cooling Water TechJACK V. MATSON, University of Houston, Houston, Texas Fifty Years of Co E. A. SAVINELLI, Savinelli Associates, Wilton, Connecticut, A. J. FREEDMAN, T. M. LARONGE, Thomas M

Removal of CopperKENNETH F. LAYTON, Utah Power and Light Company, Salt Lake City, Utah Removal of CopperS. T. ARRINGTON, G. W. BRADLEY, Halliburton Services, Duncan, Oklahoma Removal of CopperS. T. ARRINGTON, G. W. BRADLEY, Halliburton Services, Duncan, Oklahoma Service Water IronPETER L. TREMONT, Peter L. Tremont & Associates, Houston, Texas Service Water IronJ. E. ANTOINE, G. O. SMITH, System Energy Resources, Inc., Port Gibson, Mississippi, E. W. EKIS, Nalco Service Water IronJ. E. ANTOINE, G. O. SMITH, System Energy Resources, Inc., Port Gibson, Mississippi, E. W. EKIS, Nalco Evaluation of Lo ROY HANGO, IBM Corporation, Essex Junction, Vermont Evaluation of Lo ROBERT M. QUINN, R Q Associates, Inc., Teaneck, New Jersey, CONNIE MORTON, City of Sarasota Wat Past, Present and PETER S. CARTWRIGHT, Cartwright Consulting Co., Minneapolis, Minnesota The History of the J. FRED WILKES, Consultant, Titusville, Florida Low Pressure SteaM. ROTTOLI, CISE, Innovative Technology, S.P.A., Milan, Italy, F. SIGON, ENEL, Thermal and Nuclear Res Advances in ContinWAYNE B. HUEBNER, Wallace & Tiernan Division, Pennwalt Corporation, Belleville, New Jersey On-Line Analysis S. P. ULBRICHT, M. D. MORETTI, The Babcock & Wilcox Company, Alliance, Ohio, R. BARRY DOOLEY, E Regeneration of AnC. J. ROMERO, Champlin Refining Company, Corpus Christi, Texas, S. D. COKER, The Dow Chemical Co Factors Influenci DON BRODIE, The Purolite Company/Microlite, Bala Cynwyd, Pennsylvania, JOSEPH BORGQUIST, CIND Pilot Scale EvaluaR. J. H. COWLES, Syncrude Canada Ltd., Edmonton, Alberta, Canada Experiences in Applying Statistical Process Control Techniques to Managing Industrial JENNY Water THOM, Treatment MARK Programs GIUSTO, Drew Industrial Division, Ashland Chemical Company, Boonton, New Jerse Real Time Control ROBERT J. FERGUSON, ChemLink, Malvern, Pennsylvania, OMAR CODINA, WAYNE RULE, TU Electric Application of Mic KENNETH B. MANCINI, Betz Equipment Systems, Horsham, Pennsylvania, ERNEST G. TAYLOR, Betz Ind Feedwater Iron ReB. C. WHITE, J. W. DAVIS, Carolina Power and Light Company, Southport, North Carolina Ultrapure Water TrJOSE LUIS DEL ARCO, Graver Espanola, SA, Bilbao, Spain, JOSE FERNANDEZ BENLLOCH, Foster Wh Selection and TreaM. I. MORRIS, J. L. KASTEN, T. M. GILLIAM, Waste Management Technology Center, Oak Ridge National Water and Wastewat O. ELMER MITCHELL, Rust International Corporation, Birmingham, Alabama The Use of Ceramic ROBERT MARAVICH, GLEN P. SUNDSTROM, WAYNE T. BATES, Illinois Water Treatment Company, Rock Biofouling Control SUNAO IKUTA, Mitsubishi Gas Chemical Company, Inc., Tokyo Japan, KUNIO NISHIMURA , TORU YASUN Predict Fouling, P J. FRED WILKES, Aquatec Quimica S.A., TitusvilLe, Florida, FLAVIO BIANCHI, MESSIAS C. AMARAL, Aqu Performance and M PATRICK SULLIVAN, ALAN YEOMAN, Ciba-Geigy Corporation, Ardsley, New York Corrosion Inhibito MIKE A. WINTERS, Amoco Corporation, Naperville, Illinois Corrosion Inhibito ALLEN E. FELTZIN, Airco Industrial Gases, Murray Hill, New Jersey, ROBERT DISTEFANO, Syntec Corpor Comparison of OxyPETER TURNER, San Diego State University, San Diego, California Comparison of OxyP. D. SCHUMACHER, R. D. MUEHLENKAMP, K. A. KOEHLER, Wisconsin Electric Power Company, Milwa On-Line ChemistryDALE M. SOPOCY, Sargent & Lundy Engineers, Chicago, Illinois On-Line ChemistryJAMES C. BELLOWS, KAREN L. WEAVER, Westinghouse Electric Corporation, Orlando, Florida, JAMES On-Line ChemistryJAMES C. BELLOWS, KAREN L. WEAVER, Westinghouse Electric Corporation, Orlando, Florida, JAMES Phosphate HideoutA. BANWEG, Babcock & Wilcox Company, Alliance, Ohio Phosphate HideoutA. C. McDONALD, B. L. TRACY, Betz Laboratories, Inc., The Woodlands, Texas Phosphate HideoutA. C. McDONALD, B. L. TRACY, Betz Laboratories, Inc., The Woodlands, Texas Free vs. Captive AlT. J. TVEDT, Jr., The Dow Chemical Company, Freeport, Texas Free vs. Captive AlS. T. COSTA, J. M. PACE, R. D. TRUMBETTA, Calgon Corporation, Pittsburgh, Pennsylvania Free vs. Captive AlS. T. COSTA, J. M. PACE, R. D. TRUMBETTA, Calgon Corporation, Pittsburgh, Pennsylvania An On-Site ChemicJOHN W. SCHUMANN, City of Los Angeles Department of Water and Power, Los Angeles, California An On-Site ChemicM. L. SAMUELSON, S. B. McCONNELL, E. F. HOY, Dowell Schlumberger, Tulsa, Oklahoma An On-Site ChemicM. L. SAMUELSON, S. B. McCONNELL, E. F. HOY, Dowell Schlumberger, Tulsa, Oklahoma FRG Wastewater Tre WALTER ZABBAN, The Chester Engineers, Pittsburgh, Pennsylvania FRG Wastewater Tre ALBERT BURSIK, Grosskraftwerk Mannheim AG, Mannheim, Federal Republic of Germany, ERICH DETER FRG Wastewater Tre ALBERT BURSIK, Grosskraftwerk Mannheim AG, Mannheim, Federal Republic of Germany, ERICH DETER New Lower Cost Met DONALD H. NEWMAN, Schneider Engineers, Bridgeville, Pennsylvania New Lower Cost Met RONALD L. KOLBASH, American Electric Power, Lancaster, Ohio, DONALD BUDEIT, Turbojett Internationa Removal of Trace DAVID E. SIMON, II, Cyrus Rice Consulting Group, Pittsburgh Pennsylvania Removal of Trace MARK A. MANZIONE, DOUGLAS T. MERRILL, Brown and Caldwell, Pleasant Hill, California, MARY E. Mc Removal of Trace MARK A. MANZIONE, DOUGLAS T. MERRILL, Brown and Caldwell, Pleasant Hill, California, MARY E. Mc

Leachables vs. Io J. R. STAHLBUSH, R. M. STROM, The Dow Chemical Company, Midland, Leachables vs. Io SALLIE A. FISHER, GERARD OTTEN, Puricons, Inc., Malvern, Pennsylvania Leachables vs. Io SALLIE A. FISHER, GERARD OTTEN, Puricons, Inc., Malvern, Pennsylvania Evaluation of a UniFRANCES M. CUTLER, Southern California Edison, Paramount, California Evaluation of a UniGORDON P. USITALO, THOMAS J. P. WYSOCKI, Detroit Edison, Detroit, Michigan, DANIEL B. RICE, The Groundwater Resto HARI B. GUPTA, Culligan International, Northbrook, Illinois Groundwater Resto GLENN J. CATCHPOLE, Uranerz U.S.A., Inc., Casper, Wyoming, MARK MOXLEY, Wyoming Department o Groundwater Resto GLENN J. CATCHPOLE, Uranerz U.S.A., Inc., Casper, Wyoming, MARK MOXLEY, Wyoming Department o Reducing Chlorides ROBERT H. LANGER, VERNON W. JONES, Northeast Utilities, Hartford, Connecticut Reducing Chlorides ROBERT DVORIN, SAIC, Paramus, New Jersey, JOHN KRISTENSEN, New York, Power Authority, Buchan Reducing Chlorides ROBERT DVORIN, SAIC, Paramus, New Jersey, JOHN KRISTENSEN, New York, Power Authority, Buchan TOC Removal fromTHOMAS A. DAVIS, Graver Water Division of the Graver Company, Union, New Jersey TOC Removal fromWILLIAM E. KATZ, FREDERICK G. CLAY, Ionics, Incorporated, Watertown, Massachusetts TOC Removal fromWILLIAM E. KATZ, FREDERICK G. CLAY, Ionics, Incorporated, Watertown, Massachusetts A Novel SynergistiALAN L. SMITH, Calgon Corporation, Pittsburgh, Pennsylvania A Novel SynergistiE. E. WILLIAMS, B. KNOX-HOLMES, M. F. DIPROSE, Biofouling and Corrosion Control, Ltd., Sheffield, En A Novel SynergistiE. E. WILLIAMS, B. KNOX-HOLMES, M. F. DIPROSE, Biofouling and Corrosion Control, Ltd., Sheffield, En Quantitation of Le JAMES C. FEELEY, Pathogen Control Associates, Tucker, Georgia Quantitation of Le RICHARD W. GILPIN, ADELE M. KAPLAN, EDWIN F. GOLDSTEIN, Arc Water Treatment Company, Philad Quantitation of Le RICHARD W. GILPIN, ADELE M. KAPLAN, EDWIN F. GOLDSTEIN, Arc Water Treatment Company, Philad A Case History of BILLY D. FELLERS, Texas Utilities, Glen Rose, Texas A Case History of J. W. THORNTON, D. S. MORELEN, Virginia Power, Yorktown, Virginia A Case History of J. W. THORNTON, D. S. MORELEN, Virginia Power, Yorktown, Virginia Dioxine: TreatmenDIETER VON DER MARK, Schmidding-Werke, Cologne, West Germany, Dr. HANS H. RUMP, Institut Frese Wastewater TreatmRANDOLPH W. RAKOCZYNSKI, Waste Resource Associates, Inc., Niagara Falls, New York Destruction of TOCPETER S. MEYERS, L*A Water Treatment Corporation, City of Industry, California Electrodeionizatio JONATHAN WOOD, YAIR EGOZY, GARY C. GANZI, Millipore Corporation, Bedford, Massachusetts Condensate Polishe F. X. McGARVEY, Sybron Chemicals Inc., Birmingham, New Jersey Condensate Polishe ROBERT H. LANGER, Northeast Utilities Service Company, Berlin, Connecticut, THOMAS H. BURNS, Nor An Overview of th FRANCES M. CUTLER, Southern California Edison Company, Paramount, California An Overview of th GEORGE PLUME, CYRIL MACNEIL, New Brunswick Electric Power Commission, Lepau, New Brunswic KLM’s Optimized BWILLIAM E. KATZ, Ionics, Inc., Watertown, Massachusetts KLM’s Optimized BDONALD A. SCHUELKE, Northern States Power Company, Welch, Minnesota, B. GEORGE KNIAZEWYCZ Use of RO Membran HERMANN W. POHLAND, E.I. duPont de Nemours & Company, Inc., “Permasep” Products, Wilmington, D Use of RO Membran CHARLES M. CUMMINGS, UOP Fluid Systems, San Diego, California, ALAN B. RIEDINGER, Consultant, Containerized Tre DEAN SPATZ, Osmonics, Inc., Minnetonka, Minnesota Containerized Tre LUTHER B. JONES, Virginia Electric Power Company, Richmond, Virginia, JOHN F. BOSSLER, Arrowhead Seawater ReverseKURT O F. FRANK, FilmTec Corporation, San Diego, California Seawater ReverseEDWARD O L. DUBOST, CONSTANTINE T. POLIDOROFF, Pacific Gas and Electric Company, San Francisco Design and OperatA. C. McDONALD, G. J. PATEK, Betz Laboratories, Inc., The Woodlands, Texas Deterioration of C KAROL DAUCIK, I/S Skaerbaekvaeket, Fredericia, Denmark, KATE WIECK-HANSEN, I/S Nordkraft, Aalbor Simple and Rapid E MARL L. HENN, Calgon Corporation, Pittsburgh, Pennsylvania An Unusual Case St DOUGLAS B. DEWITT-DICK, Drew Industrial Div., Boonton, New Jersey Oxygen Scavenging BRUCE L. LIBUTTI, ROBERT KUNIN (Consultant), Graver Chemical Division of the Graver Company, Unio A Safe – Low Tempe EDGAR F. HOY, Dowell Schlumberger, Tulsa, Oklahoma Bromine ChemistryBILLY D. FELLERS, Texas Utilities Electric, Dallas, Texas Bromine ChemistryJEFFERY C. CONLEY, EDWARD H. PUZIG, Great Lakes Chemical Corporation, West Lafayette, Indiana, J Water Treatment i C. D. SCHROEDER, Consultant, Acton, Massachusetts Water Treatment i KEN W. ROWLAND, CHRISTOPHER H. NORTON, Southern California Gas Company, Los Angeles, Califo Role of Polymers THOMAS M. LARONGE, Thomas M. Laronge Inc., Vancouver, Washington Role of Polymers G. A. CRUCIL, J. R. MACDONALD, E. B. SMYK, Nalco Chemical Company, Naperville, Illinois

Practical Air Wash DAN M. THOMPSON, Culligan Industrial Water Treatment Division, Chattanooga, Tennessee Practical Air Wash JOHN C. PETERSON, Wright Chemical Corporation, Greensboro, North Carolina An Evaluation of thTHOMAS M. BACHEY, Cincinnati Gas & Electric, New Richmond, Ohio, ELLEN ADAMSKI, Illinois Water Tr Particle Size Mea L. S. GOLDEN, Purolite International Ltd., Pontyclun, S. Wales, United Kingdom New Technology inPETER MICHAEL LANGE, FRIEDRICH B. MARTINOLA, HANS KARL SOEST, Bayer AG, Leverkusen, Ger A Three Bed Demin STEVE BRUNTLETT, V. P. MURPHY, International Paper, Mansfield, Louisianna, , BRUCE BARTON, Calgo Weak Acid Resin fMICHAEL O’BRIEN, ELI SALEM, Graver Water Division, The Graver Company, Union, New Jersey, BRUCE On-Site GenerationTHOMAS A. DAVIS, TERRANCE LATERRA, Graver Water Division of The Graver Company, Union, New Je Ozone Treatment oA. RONY JOEL, SHAUN S. PIERSON, Capital Controls Company, Colmar, Pennsylvania Start-Up of a Coal H. A. (HANK) MANN, Stanley Consultants, Inc., Muscatine, Iowa, JIM VOLK, Niagara Mohawk Power Corp An Innovative SoluWILLIAM J. MERZ, Calgon Carbon Corporation, Pittsburgh, Pennsylvania An Innovative SoluROBERT L. SOLOMON, DANIEL J. PETERSON, Resources Conservation Co., Bellevue, Washington Wastewater Manage MIRA T. JUNUSZ, Westinghouse Electric Corporation, Resource Energy Systems Division, Pittsburgh, Penn Wastewater Manage PRABHAT KUMAR SINHA, NANCY D. ZEIGLER, LAWRENCE J. GASPER, Bechtel Civil, Inc., Gaithersbur Field Study: Elect JAMES L. RABER, 3M Company, St. Paul, Minnesota Field Study: Elect GARY CAPLAN, FRED STEGMAYER, Bird Archer, Inc., Cobourg, Ontario, Canada Study of the ChemiGORDON L. KEY, Combustion Engineering, Inc., Windsor Connecticut Study of the ChemiROCKY H. THOMPSON, Florida Power Corporation, Crystal River, Florida, LARRY S. LAMANNA, The Bab Deaerator Cracking M. A. WINTERS, Amoco Corporation, Naperville, Illinois Deaerator Cracking J. A. KELLY, Dearborn Division, W.R. Grace & Company, Lake Zurich, Illinois, C. E. GUZI, The Proctor & Ga Contaminated BoilT. J. TVEDT, Jr., Dow Chemical Company, Freeport, Texas Contaminated BoilEDGAR E. WATANABE, Aquatec Quimica, S.A., Sao Paulo, Brasil, JAMES N. TANIS, Aquatec Quimica, S.A The State of the A CECIL W. WAGES, Georgia Power Company, Atlanta, Georgia The State of the A M. D. ROSEN, A. F. ASCHOFF, Sargent & Lundy, Chicago, Illinois, J. A. BARTZ, Electric Power Research In Chemical CleaningJOSEPH M. PARENT, Florida Power & Light Co., Indiantown, Florida, JOSE P. LOZADA, JULIUS ISAAC, E Corrosion PreventiJAMES C. BELLOWS, Westinghouse Electric Corporation, Orlando, Florida Corrosion PreventiTHOMAS H. PIKE, Western Farmers Electric Cooperative, Hugo, Oklahoma Cooling Water Tre JERRY PRINTZ, Stearns-Roger Division, United Engineers & Constructors, Inc., Denver, Colorado Cooling Water Tre AL THIEME, Clark Oil and Refining Corporation, Blue Island, Illinois, HAROLD MOFFAT, JOHN CHIAPPET Comparison of New FREDERICK J. POCOCK, The Babcock & Wilcox Company, Alliance, Ohio Comparison of New DAVID J. NICHOLSON, Electricity Corporation of New Zealand, Ltd., Wellington, New Zealand, NATHANIE EPRI Interim Conse ALBERT BURSIK, Grosskraftwerk Mannheim, Mannheim, Federal Republic of Germany Deoxygenation in aELI SALEM, The Graver Company, Union, New Jersey Deoxygenation in aT. A. HOOK, W. L. PEARL, NWT Corporation, San Jose, California, J. SCHEIBEL, Electric Power Research Chemical Transport J. BROWN, Ontario Hydro Research Division, Toronto, Ontario, Canada Chemical Transport OTAKAR JONAS, Jonas, Inc. Consultants, Wilmington, Delaware, BARRY C. SYRETT, Electric Power Res Targeted ChlorinatW. CHOW, Electric Power Research Institute, Palo Alto, California, Y. G. MUSSALLI, Stone & Webster Engi Galvanized CoolinH. RAYMOND TOOL, Branchemco, Inc., Jacksonville, Florida On-Site Pilot PlantR. C. SCHWARZ, Betz Industrial, Trevose, Pennsylvania Field Testing of a SAMUEL E. SHULL, FRANCIS J. HIMPLER, Lonza, Inc., Williamsport, Pennsylvania, PAUL R. PUCKORIU Molybdate-Based C KENNETH F. SOEDER, JOSEPH S. ROTI, Drew Industrial Division, Boonton, New Jersey Study on Attrition C. WAYNE URION, Delmarva Power & Light Company, Newark, Delaware Study on Attrition HIDEO KAWAZU, MASAHIRO HAGIWARA, TAKESHI IZUMI, Ebara Corporation, Tokyo, Japan, KANROKU Prediction and IdeSALLIE A. FISHER, Puricons, Inc., Malvern, Pennsylvania Prediction and IdeJ. R. STAHLBUSH, R. M. STROM, R. G. BYERS, J. B. HENRY, N. E. SKELLY, The Dow Chemical Compan Thermal DegradatiMICHAEL C. GOTTLIEB, ResinTech Inc., Cherry Hill, New Jersey Thermal DegradatiF. X. McGARVEY, E. W. HAUSER, B. BACHS, J. STELLITANO, Sybron Chemicals Inc., Birmingham, New J Advanced Ion ExchGEORGE J. CRITS, Cochrane Environmental Systems, King of Prussia, Pennsylvania Advanced Ion ExchGREGORY R. ALLAN, Nuclear Fluids, Inc., Redmond, Washington Ion ChromatographDAVID BERG, DAN VANDERPOOL, Mobay Corporation, Pittsburgh, Pennsylvania, DICK RUBIN, Dionex C A Self-Cleaning D JIM DARTEZ, Royce Instrument Corporation, New Orleans, Louisiana

Microchemistry of DONALD L. GIBBON, RICHARD G. VARSANIK, GEORGE C. SIMON, Calgon Corporation, Pittsburgh, Pen Experiences with aROGER A. LEWIS, Southern Company Services, Birmingham, Alabama, JUD RICHARDSON, Alabama Po A PC Based Conden JACK K. SCHMOTZER, Babcock & Wilcox, Research & Development Division A PC Based Conden BARRY W. VIAL, Duke Power Company, Charlotte, North Carolina Water Quality DataJAMES A. MATHEWS, Duke Power Company, Charlotte, North Carolina Water Quality DataDAVID J. SONNTAG, LEONARD J. TRUDEAU, Detroit Edison, Detroit, Michigan Expert Systems App ROBERT L. HAMILTON, Stearns-Roger Division, United Engineers & Constructors, Denver Colorado Expert Systems App D. M. SOPOCY, A. R. GLAZER, J. A. MONTANUS, Sargent & Lundy, Chicago, Illinois Measurement of Sod HANS ZEHNDER, Polymetron AG, Switzerland Experiences with aKURT HOCHMÜLLER, BASF AG, Ludwigshafen/Rhein, Federal Republic of Germany Field Experiences F. X. McGARVEY, M. MOLDOFSKY, Sybron Chemicals, Inc., Birmingham, New Jersey, DAVID HAYEK, LE Anion Exchange Res J. T. McNULTY, M. EUMANN, C. A. BEVAN, V. C. T. TAN, Rohm and Haas Company, Spring House, Penns The Thermal DegraGEORGE SIMON, UNC Nuclear Industries, Richland, Washington New Condensate M WILLIAM E. BORNAK, Betz Laboratories, Inc., Trevose, Pennsylvania Operating ExperieWALTER G. FLINT, Kewaunee Nuclear Station, Kewaunee, Wisconsin, ROBERT J. McINTOSH, Anatel Inst Comparative InvestDAN VANDERPOOL, Mobay Corporation, Pittsburgh, Pennsylvania , H. J. ROTHER, Bayer AG, Krefeld, W Significant Silica GEORGE J. CRITS, Cochrane Environmental Systems, King of Prussia, Pennsylvania Significant Silica P. C. D. GEORGE SAMUEL, Madras Refineries Limited, Madras, Tamil Nadu, India Innovative Design GERALD F. CONNELL, Capital Controls Company, Inc., Colmar, Pennsylvania Innovative Design DAVID R. JONES, Brown and Caldwell Consulting Engineers, Pasadena, California, JOHN C. DETWEILER Identification and DOUGLAS B. DEWITT-DICK, Drew Industrial Division, Ashland Chemical Company, Div., Ashland Oil, Inc., Identification and MARVIN D. SILBERT, Marvin Silbert & Associates, Willowdale, Ontario, Canada, GABRIEL M. NICOLAIDE Utilizing SPC Tec WINSTON CHOW, Electric Power Research Institute, Palo Alto, California Utilizing SPC Tec WILLIAM W. T. CHISHOLM, D. BURELLA, Dofasco, Inc., Hamilton, Canada E. ALLMON, Babcock and Wilcox, A McDermott Company, Research and Development Division Advanced Real-Tim Alliance, Ohio Advanced Real-Tim SUSAN M. WOZNIAK, JOHN R. BALAVAGE, PHIL J. BATTAGLIA, Westinghouse Electric Corporation, Pitts Microcomputer CoROBERT M. ROSAIN, CH2M Hill, Bellevue, Washington Microcomputer CoDALE M. SOPOCY, ROBERT J. CHIESA, N. M. KASS, Sargent & Lundy, Chicago, Illinois, W. C. MICHELE Temperature EffecBRIAN A. DEMPSEY, The Pennsylvania State University, University Park, Pennsylvania, LINDLE WILLNOW Rapid Evaluation of MICHAEL F. COUGHLIN, GARY CAPLAN, Bird Archer, Inc., Cobourg, Canada Kinetic Analysis o WILLIAM F. McCOY, JAMES E. RIDGE, EDWARD S. LASHEN, Rohm and Haas Company, Spring House, Rapid Biocide SeleRICHARD A. CLARK, Buckman Laboratories, Memphis, Tennessee Corrosion Inhibiti BENNETT P, BOFFARDI, Calgon Corporation, Pittsburgh, Pennsylvania Corrosion Inhibiti PAUL D. SCHUMACHER, Wisconsin Electric Power Company, Milwaukee, Wisconsin, ROY A. JOHNSON, An Improved Method GEORGE F. HAYS, JENNY THOM, Drew Industrial Division, Boonton, New Jersey Controlling Corros RUSSELL W. LANE, Water Treatment Consultant, Champaign, Illinois Controlling Corros CHRIS A. BISSETT, West Texas Utilities Company, Abilene, Texas, JAMES F. HARRISON, Calgon Corpora Field Experiences PAUL R. PUCKORIUS, Puckorius & Associates, Inc., Evergreen, Colorado Field Experiences ROBERT D. MOSS, Tennessee Valley Authority, Chattanooga, Tennessee, STEPHEN P. GAUTNEY, Tenne The Physical Stre MALCOLM BALL, RICHARD RONAYNE HARRIES, WALTER WILLIAM PICKERING, Central Electricity, Ge Hollow Fiber Filtr WILLIAM G. LIGHT, Fluid Systems Division, UOP, Inc., San Diego, California Hollow Fiber Filtr ROBERT L. HARKINS, HPD Incorporated, Naperville, Illinois, JAMES F. SMALL, Union Electric Company, F Colloidal Silica Rem ROBERT DVORIN, New York Power Authority, White Plains, New York Colloidal Silica Rem JAMES A. McCAW, Jr., New York State Electric & Gas Corporation, Binghamton, New York The Decision to ReJAMES L. McNUTT, Camp Dresser & McKee Inc., Dallas, Texas The Decision to ReMARK C. ADAMS, Kansas City Power & Light Company, Kansas City, Missouri, AUSTIN F. McCORMACK, EDR Retrofit at M JOHN M, PUGSLEY, Florida Power & Light Company, Miami, Florida EDR Retrofit at M PRABHAT KUMAR SINHA, Bechtel Civil, Inc., Gaithersburg, Maryland, KISHORE T. GOKHALE, Bechtel Ea Monitoring of MorpWESLEY L. PEARL, S. G. SAWOCHKA, NWT Corporation, San Jose, California Monitoring of MorpJOHN M. RIDDLE, US Operating Services Corporation, Pittsburgh, Pennsylvania, THOMAS O. PASSELL, Review of Boiler Wa DOUGLAS OLIVER, East Kentucky Power Cooperative, Inc., Winchester, Kentucky

Review of Boiler Wa J. STODOLA, Ontario Hydro, Toronto, Canada Eight Years of Mo FREDERICK J. POCOCK, The Babcock & Wilcox Company, Research and Development Division, Alliance, Eight Years of Mo ALBERT BURSIK, Grosskraftwerk Mannheim AG, Mannheim, Federal Republic of Germany Standby Boiler ProJESSE S. BEECHER, Consultant, Drew Industrial Division, Ashland Chemical Co., Standby Boiler ProEDGAR EIHACHI WATANABE, NELSON GASTALDO, Aquatec Quimica S. A., Sao Paulo, Brasil, JAMES N Corrosion Product JAMES BROWN, Ontario Hydro Research Division, Ontario, Canada Corrosion Product T. A. HOOK, W. L. PEARL, S. G. SAWOCHKA, NWT Corporation, San Jose, California, J. PUGSLEY, Florid Retrofitting of Co ISHAI OLIKER, FRANK SILAGHY, Burns & Roe Company, Oradell, New Jersey Retrofitting of Co ROBERT E. PALMER, MICHAEL J. BENNETT, Detroit Edison, Detroit, Michigan EPRI RP2712-3 Moni D. M. SOPOCY, A. F. ASCHOFF, Sargent & Lundy, Chicago, Illinois, R. B. DOOLEY, Electric Power Resear Alternative ChemicJAMES O. ROBINSON, Betz Laboratories, Trevose, Pennsylvania Alternative ChemicLINDA L. SCHNEIDER, DONOVAN C. HUTCHENS, Omaha Public Power District, Omaha, Nebraska “Extractables” in SALLIE FISHER, GERARD OTTEN, Puricons, Inc., Berwyn, Pennsylvania Powdered Weakly BRUCE A L. LIBUTTI, ROBERT D. FORMAN, ROBERT KUNIN, Graver Chemical Company, Union, New Jers Ion Exchange Resin FRIEDRICH B. MARTINOLA, HEIKO D. HOFFMANN, Bayer AG, Leverkusen West Germany, Field Experience wDAVID J. HAYEK, LEONARD W. TARPLEY, Corpus Christi Petrochemical Company, Corpus Christi, Texas Demineralization RKENNETH J. KALLFISCH, Ebasco Services, Inc., New York, New York, NICHOLAS T. ESPOSITO, Jersey C The Regeneration R. RAITER, Ayalon Water Conditioning Company, Ltd., Haifa, Israel Side-By-Side Comp YOSHI TAKAHASHI, Dohrman Division, Xertex, Santa Clara, California Side-By-Side Comp SUE A. HOBART, Electric Power Research Institute, Palo Alto, California, MICHAEL R. MILLER, GERARD Characterization vsYOUNG LEE, Sargent & Lundy Engineers, Chicago, Illinois Characterization vsJOSEPH F. SELANN, DAVID BROMLEY Engineering, Ltd., Edmonton, Canada, MICHAEL E. ROGERS, EM On-Line Analysis oJUDITH A. RAWA, Calgon Corporation, Pittsburgh, Pennsylvania On-Line Analysis oGARY D. BURNS, RICHARD C. NOLAN , JOHN E. CRUTCHFIELD, NUS Operating Services Corporation, Industry-Wide Sur OTAKAR JONAS, Jonas, Inc., Wilmington, Delaware Industry-Wide Sur W. A. BYERS, J. E. RICHARDS, Westinghouse Electric Corporation, Pittsburgh, Pennsylvania, S. A. HOBA TOC Removal fromMARJORIE BALAZS, Balazs Analytical Lab., Mountain View, California TOC Removal fromWILLIAM S. MILLER, Ecolochem, Inc., Norfolk, Virginia, VERNON W. JONES, ROBERT H. LANGER, North Design and Impleme PETER S. MEYERS, L*A Water Treatment Corporation, a Member of the Portals Group, City of Industry, Ca Design and Impleme ROBERT A. HART, SALLY A. THOMAS, Conoco, Inc., Ponca City, Oklahoma Ion Exchange ResFRANK X. McGARVEY, Sybron Chemicals Inc, Birmingham, New Jersey Ion Exchange ResBABU R. NOTT, DUANE A. CHARTIER, DUNCAN H. BARBER, Ontario Hydro, Research Division, Toronto, Water Deionizatio PETER WATTS, Ecodyne, Ltd., Oakville, Ontario, Canada Water Deionizatio CRAIG J. BROWN, CATHERINE J. FLETCHER, Eco-Tec Limited, Pickering, Ontario, Canada Water Quality ProtROY H. REUTER, ICAIR – Life Systems, Inc, Cleveland, Ohio Water Quality ProtROY W. H. WILKES, pHox Systems, Ltd., Shefford, Beds., England The Application of R. J. SIKORA, Resources Conservation Company, Bellevue, Washington The Application of JAMES D. LYNCH, HPD Incorporated, Naperville, Illinois, TAKUO KAWAHARA, Asahi Glass Company, Ltd. Renovation of DiluJAMES MAVIS, CH2M Hill, Bellevue, Washington Renovation of DiluJOHN M. BEGOVICH, CLIFTON H. BROWN, JOHN F. VILLIERS-FISHER, Oak Ridge National Laboratory, Video Inspection ROCKY H. THOMPSON, Florida Power Corporation Crystal River, Florida, LARRY S. LAMANNA, Babcock Enhanced Foam Cle HOWARD E. STEIMAN, Boston Edison Company, Boston, Massachusetts Enhanced Foam Cle EDGAR F. HOY, Dowell Schlumberger, Tulsa, Oklahoma Experience with a PAUL COHEN, Consultant, Pittsburgh, Pennsylvania Experience with a CARL J. CAPPABIANCA, PAUL F. PELOSI, JAMES H. NULTY, Drew Industrial Division, Ashland Chemical Treatment and Temp PAUL E. FOSTER, Hercules Incorporated, Wilmington, Delaware Water Treatment Fa ROGER V. LONG, Stone & Webster Engineering Corporation, Denver, Colorado Water Treatment Fa JERRY PRINTZ, Stearns Catalytic Corporation, Denver, Colorado Hydrogeological InHENRY C. HUNT, Bennett & Williams, Inc., Columbus, Ohio Partial Demineral FRIEDRICH MARTINOLA, Bayer. AG, West Germany Partial Demineral WOLFGANG H. HOELL, Karlsruhe Nuclear Research Center, federal Republic of Germany

In-Ground Removal WILLIAM R. GREENAWAY, NUS Corporation, Gaithersburg, Maryland In-Ground Removal DAVID F. EDSON, Ground Water Associates, Inc., Arlington, Massachusetts Foamed Solvents fJAMES S. POOLE, Sheppard T. Powell & Associates, Baltimore, Maryland Foamed Solvents fGARY W. BRADLEY, GARY D. ARNOLD, Halliburton Services, Duncan, Oklahoma Water ConditioningJESSE BEECHER, P.E., Consultant, Drew Industrial Division, Ashland Chemical Co., Boonton, New Jersey Water ConditioningEDGARD EIHACHI WATANABE, MESSIAS CANDIDO AMARAL, Aquatec Quimica S.A., Sao Paulo, Brazil Retrofitting the W J. D. McWILLIAM, Betz Inc., Kanata, Ontario, Canada Retrofitting the W MICHAEL E. ROGERS, Syncrude Canada, Ltd., Alberta, Canada Application of Hig WILLIAM R. KASSEN, NWT Corporation, San Jose, California, J. M. BURGER, Empire State Electric Energ Acid Free Scale CD. P. LOGAN, H. E. NEHUS, A. L. SMITH, Calgon Corporation, Pittsburgh, Pennsylvania A Progress ReportTHEODORE PICKERING, BETTY J. WHITE, Florida Power & Light Company, Fort Myers, Florida Unique Double PasGREGORY A. PITTNER, Arrowhead Industrial Water, Los Angeles, California, RICHARD R. LEVANDER, A Compatibility of D AMANDA MEITZ, The Mogul Corporation, Chagrin Falls, Ohio Compatibility of D RICHARD W. WALTER, JR., JACK F. MILLS, ATTILA G. RELENYI, The Dow Chemical Company, Midland, Cooling Water Tre KENNETH J. KOZELSKI, E. I. Du Pont de Nemours & Co., Inc., Camden, South Carolina Cooling Water Tre BLAIR JONES, Capital Controls Company, Inc., Colmar, Pennsylvania, GERALD SLIFER, Public Service E Evaluation of Alte RODNEY H. SERGENT, Great Lakes Chemical Corporation, West Lafayette, Indiana Evaluation of Alte ROBERT CHIESA, Sargent & Lundy, Chicago, Illinois, DENNIS GEARY, Wisconsin Power & Light Compan Initial Operating ROBERT S. BERNSTEIN, Ebasco Services, Inc., New York, New York, STEVEN F. DEMARCO, New York S Fluid Composition OLEH WERES, LEON TSAO, Lawrence Berkeley Laboratory, University of California, Berkeley, California PWR Secondary Sys P. J. BATTAGLIA, M. W. ROOTHAM, M. J. WOOTTEN, Westinghouse Electric Corporation, Pittsburgh, Pen Condenser Air InleSCOTT BOND, Union Electric, Callaway Nuclear Plant, Fulton, Missouri, ALEXANDER D. MACARTHUR, W Silica Removal in TIMOTHY W. TROFE, MILTON L. OWEN, Radian Corporation, Austin, Texas, WAYNE C. MICHELETTI, Ele The Investigation JOHN W. SIEGMUND, Sheppard D. Powell Associates, Baltimore, Maryland The Investigation LEROY V. BALDWIN, ELLEN S. FEENEY, EG&G Florida, Inc., Kennedy Space Center, Florida, RICK BLAC Cooling Water Prod BRIAN R. OHLER, Public Service Company of New Mexico, Farmington, New Mexico, WILLIAM A. HOLLIN On Stream Iron ReJ. FRED WILKES, Consulting Chemical Engineer, Titusville, Florida On Stream Iron ReGARY G. ENGSTROM, VINCENT KUHN, Dearborn Chemical Company, Lake Zurich, Illinois Zero Blowdown byTHOMAS M. LARONGE, Thomas M. Laronge, Inc., Vancouver, Washington Zero Blowdown byBOB L. PROCTOR, CHESTER M. MALEWSKI, Sierra Pacific Power Company, Reno, Nevada Parametric Study JOHN E. CADOTTE, FilmTec Corporation, Minneapolis, Minnesota Parametric Study YAW C. YANG, J. M. DICKSON, McMaster University, Ontario, Canada ED: Polarity ReverF. X. McGARVEY, Sybron Chemicals Inc., Birmingham, New Jersey ED: Polarity ReverPAUL A. FOSTER, MICHAEL J. ROWE, JOHN B. FARRAR, Portals Water Treatment, Isleworth, United Kin Commercial Produc CLARENCE D. COLLEY, Hewlett Packard Company, Palo, Alto, Calitornia Commercial Produc WILLIAM E. KATZ, FREDERICK G. CLAY, Ionics, Incorporated, Watertown, Massachusetts ASTM Power PlantW. E. ALLMON, S. J. POTTERTON, The Babcock & Wilcox Company, Alliance, Ohio, S. I. LERMAN, Conso Water Chemistry SCURT BLAIR, Tennessee Valley Authority, Chattanooga, Tennessee, JIM MULLENIX, JOHNNY BARKER, T Evaluation and AppOTAKAR JONAS, Jonas, Inc., Wilmington, Delaware, R. W. BRADSHAW, Combustion Engineering, Winds Analysis of Ethyle KATHLEEN A.S. HARDY, JOHN C. COOPER, Naval Research Laboratory, Washington, D.C., THOMAS K. The Use of an InorAa. BARKATT, K. A. MICHAEL, W. SOUSANPOUR, Al. BARKATT, L. M. PENAFIEL, B. P. MACEDO, The C Effects of Coagul L. M. MAY, W. M. CARLSON, Nalco Chemical Company, Naperville, Illinois Radwaste Ion-Exch N. P. JACOB, E. MORGAN, J. M. STORTON, J. F. KRAMER, Babcock & Wilcox, Lynchburg, Virginia, J. P. K Progress in the ReLIGAYA S. HARTJEN, Graver Water Division, Union, New Jersey, WILLIAM M. CAIN, Carolina Power & Lig A Progress ReportMICHAEL O’BRIEN, ROBERT KUNIN, GEORGE C. FLYNN, Graver Water Division Union, New Jersey Evolution of a Wa DAVID H. PAUL, JIM PETRIE, Public Service Company of New Mexico, Waterflow, New Mexico Evolution of a Wa G. L. WILSON, J. C. NOWAK, T. M. O’NEAIL, Resources Conservation Co., Bellevue, Washington Water-Related InteWILLIAM P. GROBMYER, Burns & McDonnell, Kansas City, Missouri Water-Related IntePHILIP A. BARKER, Nalco Chemical Company, Oak Brook, Illinois, THOMAS FITZSIMMONS, Basin Electr EPRI’s Interim ConF. J. POCOCK, Babcock & Wilcox, Research and Development Division, Alliance, Ohio EPRI’s Interim ConDALE M. SOPOCY, A. F. ASCHOFF, Sargent & Lundy Engineers, Chicago, Illinois, R. B. DOOLEY, Electric

Vanadium RemovalDAVID CLINE, Sheppard T. Powell & Associates, Baltimore, Maryland Vanadium RemovalHANS W. SCHICK, Central Hudson Gas & Electric Corporation, Poughkeepsie, New York, ROBERT M. RO A New GKM Approa JAMES D. SHIVERS, Water Treatment Corporation. Member of the Portals Group, City of Industry, Californ A New GKM Approa ALBERT BURSIK, Grosskraftwerk Mannheim AG, Mannheim, West Germany Condensate Polishe JAMES SHIELDS, Belco Pollution Control, Parsippany, New Jersey Condensate Polishe JOHN E. KRISTENSEN, HARRY G. HARTJEN, Ebasco Services, Incorporated, Lyndhurst, New Jersey The Tripol Condens LOUIS F. WIRTH, Consultant, Midland, Michigan The Tripol Condens MALCOLM BALL, ROBERT J. BURROWS, Central Electricity Generating Board, Nottingham, England Operating Experienc OTAKAR JONAS, Jonas, Inc., Wilmington, Delaware Operating Experienc MICHAEL G. CASHIN, Central Illinois Public Service Co., Springfield, Illinois Evaluation of a ReM. J. WOOTTEN, Westinghouse Electric Corporation, Pittsburgh, Pennsylvania Evaluation of a ReG. L. WARD, Duke Power Company, Charlotte, North Carolina Gel vs. Macroporou SALLIE FISHER, Puricons, Inc., Berwyn, Pennsylvania Gel vs. Macroporou T. K. COPOLO, Carolina Power & Light Co., Roxboro, North Carolina, H. A. SCHLESINGER, Gibbs & Hill, I The Multistep SystIRVING M. ABRAMS, Ph.D., Consultant, Los Altos, California The Multistep SystFRIEDRICH B. MARTINOLA, GERFRIED WUTTE, Bayer AG, Leverkusen, West Germany Experiences with V. R. DAVIES, Rohm and Haas Company, Philadelphia, Pennsylvania Experiences with GEORGE P. BAKER, Cleveland Electric Illuminating Co., Cleveland, Ohio, GEORGE J. CRITS, Cochrane E Evaluation of WeakRICHARD HETHERINGTON, Epicor, Inc., Linden, New Jersey Evaluation of WeakJAMES A. MATHEWS, Duke Power Company, Charlotte, North Carolina Applications of Ad ERICH W. TIEPEL, Resource Technologies Group, Inc., Denver, Colorado, JACOB SHORR, Memtek Corpo When Hazardous W CHRISTOPHER R. RYAN, Geo-Con, Inc., Pittsburgh, Pennsylvania What Really Happen VIJAY K. PURI, Calgon Corporation, Pittsburgh, Pennsylvania What Really Happen SALLIE FISHER, GERARD OTTEN, Puricons, Inc. Berwyn, Pennsylvania Removal of OrganSUE A. HOBART, Electric Power Research Institute, Palo Alto, California Removal of OrganS. L. WISER, Virginia Electric and Power Company, Richmond, Virginia, Y. H. LEE, C. R. STROH, Sargent Lessons Learned fF. X. McGARVEY, Sybron Chemicals Inc., Birmingham, New Jersey Lessons Learned fWILLIAM E. MOORE, Monsanto Chocolate Bayou, Alvin, Texas, PETER L. TREMONT, Monsanto Central E Reduction of oxyg ANDREW J. HARHAY, Rochester Gas and Electric Corporation Ontario, New York Reduction of oxyg S. GREGORY DE SILVA, Westinghouse Electric Corporation, Pittsburgh, Pennsylvania, ALEXANDER J. S Performance StudyCHRIS MISTRY, JERRY GILILLAND, ANDY KOUTALIDES, Hewlett Packard, CID, Cupertino, California, RO 50 Years in Separ C. CALMON, Consultant, Princeton, New Jersey 50 Years in Separ J. H. SMITH, Portals Water Treatment Ltd., Isleworth, United Kingdom, P. W. RENOUF, The Permutit Comp The WestinghouseEWING EVANS, City of Austin, Austin, Texas The WestinghouseWILLIAM M. HICKAM, P. K. LEE, W. E. SNYDER, Westinghouse R&D Center, Pittsburgh, Pennsylvania An Artificial Intel JAMES K. RICE, PE, Consulting Engineer, Olney, Maryland An Artificial Intel JAMES C. BELLOWS, Westinghouse Steam Turbine-Generator Division, Orlando, Florida An Artificial Intel JAMES C. BELLOWS, Westinghouse Steam Turbine-Generator Division, Orlando, Florida Application of Inn GIL K. DHAWAN, Applied Membranes, San Diego, California Application of Inn PETER S. CARTWRIGHT, C3 International, Inc., Roseville, Minnesota Cationic Polyelect WILLIAM L. McCULLEN, Rohm and Haas Research Laboratories, Spring House, Pennsylvania Cationic Polyelect DAVID I. DEVORE, Betz Laboratories, Inc., Trevose, Pennsylvania Reduction of Conde BENJAMIN NONAKA, DALE LIAO, Department of Water and Power, City of Los Angeles, Long Beach, Cali “Extractables” in SALLIE A. FISHER, GERARD OTTEN, Puricons, Inc., Berwyn, Pennsylvania Biocidal Efficacy RICHARD A. CLARK, Buckman Laboratories, Inc., Memphis, Tennessee Biocidal Efficacy RALPH J. KAJDASZ, R. V. EINSTMAN, Dearborn Chemical Company, Nashville, Tennessee, LINDA YOUN Pilot Plant Study t KEN CHEN, KEN L. OGLE, ROBERT D. MOSS,Tennessee Valley Authority, Chattanooga, Tennessee Trace Element Rem DOUGLAS T. MERRILL, Brown and Caldwell, Walnut Creek, California, WINSTON CHOW, Electric Power R Electrodialysis Re EDWARD P. GEISHECKER, Ionics Incorporated, Watertown, Massachusetts Electrodialysis Re MIKE MANSOURI, Amoco Chemicals Corporation, Alvin, Texas Efficacy and Deco AMANDA K. MEITZ, Mogul Corporation, Chagrin Falls, Ohio

Biocontrol – Key toKENNETH J. KOZELSKI, E.I. du Pont, Camden, South Carolina Biocontrol – Key toJACK L. FREUND, JR. Midwest Solvents Company, Inc., Atchison, Kansas , JAMES M. BOOKER JR., WIL Design and OperatBOB ROSAIN, CHET MORTON, CH2M Hill, Bellevue, Washington Evaluation of Uni FRANCES M. CUTLER, Southern California Edison, Paramount, California Evaluation of Uni JACK B. PRENTISS, Central Power and Light Company, Corpus Christi, Texas, MICHAEL D. MAYNE, Dow Progress in the D JOHN F. LONGO, Graver Chemical Division of the Graver Company, Union, New Jersey Field Experiences J. FRED WILKES, Consulting Chemical Engineer, Titusville, Florida Field Experiences I. R. GIBSON, Industrial Water Management, Pty., Johannesburg, South Africa, C. M. HWA, Dearborn Chem A Program for ReviJOHN C. PETERSON, Wright Chemical Corporation, Greensboro, North Carolina A Program for ReviTIMOTHY E. KEISTER, Brockway, Inc., Brockway, Pennsylvania Four Years OperatJIM W. SMITH, Shell Oil, Company, Norco, Louisiana, M. B. YELIGAR, Permutit Company, Inc., Paramus, N Hydroperm Lime So ROBERT FROESE, PETWA Canada Ltd., Calgary, Canada Hydroperm Lime So LORNE GRAMMS, Urban Systems, Ltd., Calgary, Alberta, DANIEL COMSTOCK, DOUGLAS HAGEN, Nep Design and OperatC. PALMER, Cotter Corporation, Lakewood, Colorado, A. HIMSLEY, J. A. BENNETT, Himsley Engineering L Sulfite Degradatio MARK BRAYDEN, HAROLD CHAGNARD, HAYES BARNETT, REUBEN BARBER, Dow Chemical, U.S.A., Effective CondensM. LINDA LIN, BRIAN V. JENKINS, Nalco Chemical Company, Oak Brook, Illinois Sewer Plant EfflueG. W. SCHWEITZER, Calgon Corporation, Pittsburgh, Pennsylvania Sewer Plant EfflueNORMAN D. FAHRER, ChemTreat, Inc., Ashland, Virginia, SHULER W. MASSEY, Vero Beach Utilities, Ver Process and ContrROGER A. GUTTSCHALL, Belco Pollution Control Corporation, Parsippany, New Jersey Process and ContrCHERYL O. MALHIET, Cajun Electric Power Cooperative, Baton Rouge, Louisiana, HENRY BYDALEK, Eb Colorimetric Determ DANIAL L. HARP, Hach Company, Loveland, Colorado VGB Guidelines onDR. ALBERT BURSIK, Grosskraftwerk Mannheim AG, Mannheim, Federal Republic of Germany Boiler and FeedwatK. TITTLE, Central Electricity Generating Board, Manchester, England Water Chemistry GWARREN J. BILANIN, ROBIN L. JONES, CHARLES S. WELTY, JR., Electric Power Research Institute, Pa PWR Secondary Wat C. S. WELTY, JR., S. J. GREEN, Electric Power Research Institute, Palo Alto, California Examples of GrounF. BARRY NEWMAN, GAI Consultants, Inc., Monroeville, Pennsylvania Examples of GrounCHRISTOPHER R. RYAN, Geo-Con, Incorporated, Pittsburgh, Pennsylvania Studies of Two-PhaF. J. POCOCK, N. J. MRAVICH, W. E. ALLMON, Babcock & Wilcox, Alliance Research Center, Research an Studies of Two-PhaM. J. FOUNTAIN, G. S. HARRISON, D. PENFOLD, J. C. GREENE, M. A. WALKER, Central Electricity Gen An Automatic Inst RAY BAUM, Craft Products Company, Inc., Pittsburgh, Pennsylvania An Automatic Inst DOUGLAS K. BENDER, C. M. FINLEY, Rohrback Cosasco Systems, Santa Fe Springs, California Makeup DemineraliGEORGE J. CRITS, Cochrane Environmental Systems, Crane Co., King of Prussia, Pennsylvania Makeup DemineraliHUGH H. HARPER, JR., DAVID L. DRUMMONDS, Southern Company Services, Inc., Birmingham, Alabam Computer Modeling RORY R. MUSIL, HUGO J. NIELSEN, Sargent & Lundy Engineers, Chicago, Illinois The Hydraulic PropSALLIE FISHER, Puricons, Inc., Berwyn, Pennsylvania The Hydraulic PropFRANK X. McGARVEY, E. W. HAUSER, B. KIEFER, Sybron Chemicals, Inc., Birmingham, New Jersey Magnetic Water Tre KENNETH W. BUSCH, MARIANNA A. BUSCH, DEBORAH PARKER, RALPH E. DARLING, JAMES L. M Qualification of A JAMES K. RICE, Consulting Engineer, Olney, Maryland, RAY F. MADDALONE, Energy Technology Division Computer-Aided Po DON GOLDSTROHM, Salt River Project, Coronado Generating Station, St. Johns, Arizona Computer-Aided Po LEYON O. BRESTEL, DUANE K. NELSON, Colorado-Ute Electric Association, Montrose, Colorado Suggested GuideliDAVID E. SIMON, II, NUS Corporation-PEC Division, Pittsburgh, Pennsylvania Suggested GuideliTHOMAS E. GALE, Ashland Petroleum Company, Ashland, Kentucky, JESSE BEECHER, Drew Chemical C A New Aminic VolatMARCELLO BARTOLETTI, R. Dona, S.P.A., Milan, Italy A New Dissolved HJOHN M. HALE, Orbisphere Laboratories, Geneva, Switzerland Analysis of Calciu GERHARD A. MEYER, ROBERT R. FRABLE, Dow Chemical, U.S.A., Midland, Michigan Field Experience wKENNETH J. KOZELSKI, E.I. du Pont de Nemours & Co., Inc., Camden, South Carolina A Rapid Method forLINDA YOUNG-BANDALA, Dearborn Chemical Company, Lake Zurich, Illinois, RALPH J. KAJDASZ, Dearb Organic Acids in W. ARTHUR BYERS, S. L. ANDERSON, W. M. HICKAM, Westinghouse Research & Development Center, Application of TO ROBERT J. JOYCE, YOSHIHIRO TAKAHASHI, Dohrmann Division of Xertex Corporation, Santa Clara, Ca Ion Exchange of GREGORY R. ALLAN, Consultant, Bellevue, Washington Potential of Weak HOWARD L. SIMPSON, Water Treatment Consultant, Cupertino, California, CHARLES A. SAUER, Duolite

The Effect of BeadSALLIE FISHER, GERARD OTTEN, Puricons, Incorporated, Berwyn, Pennsylvania Removal of Organic JOSEPH F. GIANNELLI, Graver Water and Graver Chemical Divisions of the Graver Company, Union, New Deep Bed Counterc JOHN B. FARRAR, JOSEPH H. SMITH, Portals Water Treatment, Ltd., Isleworth, Middlesex, United Kingdo Colloidal Silica R FRANK P. ROMA, New York Electric & Gas Company, Binghamton, New York, THOMAS M. ISERT, Gilbert Factors Affecting J. R. EMMETT, A. HEBBS, NEI Thompson Ltd., Kennicott Water Treatment, West Midlands, England Monitoring, ContaiWALTER ZABBAN, The Chester Engineers, Pittsburgh, Pennsylvania, Air Stripping of Or R. P. HELWICK, C. D. BLUMENSCHEIN, The Chester Engineers, Coraopolis, Pennsylvania, Synthetic Linings fCHARLES E. STAFF, Staff Industries, Incorporated, Upper Montclair, New Jersey Subsurface Cut-OffROBERT D. MUTCH, Jr., Wehran Engineering, Middleton, New York Groundwater MoniC. J. TOUHILL, A. P. PAJAK, A. J. SHUCKROW, Baker/TSA Performance of Air Block Countercurrent Regeneration for Demineralization of Division, Michael Baker, Jr., Inc., Beaver, Penn High TDS Surface MICHAEL Water D. MAYNE, Dow Chemical U.S.A., Freeport, Texas Performance of Air Block Countercurrent Regeneration for Demineralization of High TDS Surface SID Water WILSON, Champlin Petroleum Company,Corpus Christi, Texas, JOSEPH E. ZUBACK, Infilco Degremo Development of AcT. R. DILLMAN, Illinois Water Treatment Company, Rockford, Illinois Development of AcA. D. PRUNAC, E. ZAGANIARIS, Rohm & Haas European Laboratories, Valbonne, France Syncrude Canada IRVING L M. ABRAMS, Consultant, Los Altos, California Syncrude Canada ELIZABETH L WILKINS, BOYCE TUCKER, Syncrude Canada, Ltd., Fort McMurray, Alberta, Canada, DAVID Prevention of Iron RICHARD HETHERINGTON, Epicor, Incorporated, Linden, New Jersey Prevention of Iron L. R. GESS, Nalco Chemical Company, Oak Brook, Illinois, L. M. MAY, A. W. OBERHOFER, Nalco Chemic Computerization oJOHN SIEGMUND, Sheppard T. Powell Associates, Baltimore, Maryland Computerization oALVIN N. HEWING, RAY H. WARMKESSER, General Public Utilities Service Corporation, Reading, Pennsy A New Potable WaCHARLES R. MADDOX, Division of Water Hygiene, Texas Department of Health, Austin, Texas A New Potable WaRICHARD E. ROZELLE, Dow Chemical Company, Midland, Michigan Reviewing the UsaJ. FRED WILKES, Consulting Chemical Engineer, Titusville, Florida Reviewing the UsaROBERT S. REIMERS, ANN C. ANDERSON, LUANN WHITE, Tulane University, New Orleans, Louisiana Generic Reactor Co P. V. BALAKRISHNAN, Atomic Energy of Canada Limited, Chalk River Nuclear Laboratories, Chalk River, O Generic Reactor Co MICHAEL TROY, SAM KANG, Westinghouse Electric Corporation, Pittsburgh, Pennsylvania, MICHAEL D. N Diethylhydroxylamin MANFRED T. NOACK, Olin Corporation, New Haven, Connecticut Diethylhydroxylamin DIONISIO G. CUISIA, Dearborn Chemical Company, Lake Zurich, Illinois The Behavior of S WILLIAM T. LINDSAY, Lindsay and Associates, Hopkinton, Massachusetts The Behavior of S STEPHEN G. SAWOCHKA, WESLEY L. PEARL, NWT Corporation, San Jose, California, Improved Control oTHOMAS M. LARONGE, Thomas M. Laronge, Inc., Vancouver, Washington Improved Control oJ. A. KELLY, M. L. LIN, G. W. FLASCH, Nalco Chemical Company, Oak Brook, Illinois Dynamic DepositioW. E. ALLMON, The Babcock & Wilcox Company, Research and Development Division, Alliance, Ohio Dynamic DepositioOTAKAR JONAS, Jonas, Incorporated, Wilmington, Delaware, M. ROIDT, A. S. MANOCHA, Westinghouse San Miguel: a Case GEORGE CRITS, Cochrane Environmental Systems, King of Prussia, Pennsylvania San Miguel: a Case EMERY LANGE, Jr., San Miguel Electric Cooperative, Incorporated, Jourdanton, Texas, DOUGLAS S. JOH A Unique AdvanceJAMES i T. McNULTY, Rohm and Hans Research Laboratories, Spring House, Pennsylvania A Unique AdvanceELI i SALEM, Graver Water Division of The Graver Company, Union, New Jersey, CARL SCHEERER, Centr Condensate PolishMICHAEL W. ROOTHAM, Westinghouse Electric Corporation, Pittsburgh, Pennsylvania Condensate PolishDAVID P. SIEGWARTH, D. A. McNEA, NWT Corporation, San Jose, California, WILLIAM A. THORNTON, V Condensate PolishS. W. LURIE, Combustion Engineering Corp., Windsor, Connecticut Condensate PolishC. S. WELTY, Jr., Electric Power Research Institute, Palo Alto, CaliforniaC. C. STAUFFER, S. J. ELMIGER, A New CommercialANDREW J. SHARPE, Jr., Consultant, Polyelectrolyte Manufacture & Application, Midland, Georgia A New CommercialWILLIAM E. HAGSTRAND, The Lubrizol Corporation, Wickliffe, Ohio Theoretical and PrCHARLES EBERSOLE, PHILIP A. JARVIO, Philip A. Hunt Chemical Corporation, Lincoln, Rhode Island Application of a N TIMOTHY KEISTER, Brockway, Incorporated, Brockway, Pennsylvania Application of a N THOMAS W. ISAAC, ALEX J. KORVIN, ROBERT S. KLONOWSKI, Wright Chemical Corporation, Schiller P The Use of TriazinRICHARD A. CLARK, Buckman Laboratories, Inc., Memphis, Tennessee The Use of TriazinGARY R. TANGEN, Cooperative Power Association, Underwood, North Dakota, JIM CHAMBERS, Drew Ch Ozone in High PuriG. F. CONNELL, Capital Controls Company, Inc., Colmar, Pennsylvania Ozone in High PuriCARL NEBEL, PCI Ozone Corporation, West Caldwell, New Jersey

Characterization aROY O. BALL, ERM-North Central, Inc., Park Ridge, Illinois Characterization aKEN L. OGLE, KEN CHEN, Tennessee Valley Authority, Chattanooga, Tennessee, ROGER A. MINEAR, Un Water Management RUSSELL C. VANDENBERG, Resources Conservation Co. Water Management DALE M. SOPOCY, AL F. ASCHOFF, ROBERT J. CHIESA, Sargent & Lundy Engineers, Chicago, Illinois State-Of-The-Art oDR. ROBERT KUNIN, Consultant, Yardley, Pennsylvania State-Of-The-Art oWILLIAM E. KATZ, Ionics, Incorporated, Watertown, Massachusetts Ultrafiltration Aid VERITY C. SMITH, Vaponics, Incorporated, Plymouth, Massachusetts Ultrafiltration Aid RAY N. HARALSON, Western Electric Company, Indianapolis, Indiana, KENNETH E. JOHNDAHL, Osmoni Hydroperm™ Cross JAMES F D. MAVIS, CH2M Hill, Bellevue, Washington Hydroperm™ Cross DANIEL F L. COMSTOCK, ANDREW K. HSIUNG, JOSEPH A. MILLEN,Neptune Microfloc, Corvallis, Oregon Mechanism of ScalD AVID L. NavalD. Research Laboratory, Washington, DC JASBIR S.VENEZKY, GILL, CHERYL ANDERSON, RICHARD G. VARSANIK, Calgon Corporation Mechanism of ScalPittsburgh, Pennsylvania The Use of CorrosiARTHUR W. FYNSK, E.I. du Pont de Nemours & Co., Wilmington, Delaware The Use of CorrosiLEONARD FREEDMAN, Hercules, Incorporated, Wilmington, Delaware Removal of ChlorinCHARLES D. BLUMENSCHEIN, ROBERT HELWICK, the Chester Engineers, Pittsburgh, Pennsylvania Demineralization oF. X. McGARVEY, JOHN UNGAR, Sybron Chemical Division, Birmingham, New Jersey Demineralization oDAVID H. CAMERON, London Monenco Consultants Ltd., St. Catherines, Ontario, Canada, HUGH BORLA Comparison of AniJ. H. SMITH, Portals Water Treatment, Permutit-Boby Limited, Isleworth, Middlesex, United Kingdom Organic Cooling Wa SHARON W. AKIN, Zimmite Corporation, Cleveland, Ohio Problems Observed BRIAN J. HOFFMAN, Rohm and Haas Company, Springhouse, Pennsylvania Cleaning and PassPETER A. THOMAS, JESSE BEECHER, JOHN R. STINGER, Drew Chemical Corporation, Boonton, New J Improved Biologic JOHN A. CHRISTIANSEN, J. PAIGE STRALEY, ANNE L. KOPECKY, Sybron/Biochemical Birmingham, New Tri-Ammonex: NewGEORGE J. CRITS, Cochrane Environmental Systems, King of Prussia, Pennsylvania Anion Resin KinetiJAMES T. McNULTY, CLAUDE A. BEVAN, Rohm and Haas Company, Springhouse, Pennsylvania Recent Development DAVID SISKIND, Ecodyne Corporation, Union, New Jersey Treatment of a BleJ. T. ARONSON, Stearns-Roger Engineering Corporation, Denver, Colorado, G. MILLER, Public Service Co Replacing Multi-StEARL W. JOHANSEN, Neptune Microfloc, Corvallis, Oregon, SAMUEL PAREDES ZARATE, Federal de Ele Barium Fouling of RICARDO ALLEN, KAREN ATKINS, Potomac Electric Power Company, Washington, D.C. Trace Metal Analy ARTHUR W. FITCHETT, JOHN RIVIELLO, E. L. JOHNSON, Dionex Corporation, Sunnyvale, California Organic Leakage an ROBERT KUNIN – Consultant, Yardly, Pennsylvania Organic Leakage an KURT HOCHMULLER, BASF AG, Ludwigshafen/Rhein, Federal Republic of Germany Characterization aDAVID L. VENEZKY, Naval Research Laboratory, Washington, DC Characterization aWILLIAM F. MASLER, B. F. Goodrich, Avon Lake, Ohio Control of Second PAUL COHEN, Consultant, Pittsburgh, Pennsylvania Control of Second TAKAHISA HATTORI, HISASHI FUJII, Mitsubishi Heavy Industries, Takasago, Japan, S. H. PETERSON, D Control of Second Pittsburgh, Pennsylvania The Use of DisperALLEN BAUM, Westinghouse Electric Corporation, Pittsburgh, Pennsylvania The of DisperWILLIAM LECHNICK, NUS Corporation, Pittsburgh, Pennsylvania, J. M. McDOWELL, T. THOMAS, T. H. WhatUse Should a User of Cooling Water Chemicals Expect from a Chemical Supplier in the way of Services,P.etc. Wrap-Up & Floor Discussion ROBERT J.Summary FRANCO, Exxon Research and Engineering Company, Florham Park, New Jersey What Should a User SIDNEY SUSSMAN, Olin Water Services, Stamford, Connecticut Insuring the SucceDAVID R. SEXSMITH, Drew Chemical Corporation, Boonton, New Jersey How New ProductsGERALD W. SCHWEITZER, Calgon Corporation, Pittsburgh, Pennsylvania What Questions Sho DAVID C. AUERSWALD, Southern California Edison, Paramount, California Existing Cooling THOMAS C. BRESKE, E. I. du Pont de Nemours & Co., Wilmington, Delaware What Does a UserJAMES AXSOM, Sun Tech, Marcus Hook, Pennsylvania Specifications for KENNETH H. FREDERICK, GPU Nuclear Corporation, Reading, Pennsylvania Specifications for SALLIE A. FISHER, GERARD OTTEN, Puricons, Inc., Berwyn, Pennsylvania Development of “TTERRANCE LATERRA, Graver Water Division of Ecodyne, Union, New Jersey Development of “TAKIMITSU MIYAHARA, YUJI HARAGUCHI, Japan Organo Company Ltd., Tokyo, Japan Three Years of FieF. X. McGARVEY, Sybron Chemical Division, Birmingham, New Jersey Three Years of FieKENNETH J. KOZELSKI, E. I. du Pont de Nemours and Company, Inc., Camden, South Carolina

Co-Counter-CurrenJ. D. DARJI, Infilco Degremont Inc., Richmond Virginia Co-Counter-CurrenGEORGE J. CRITS, W. Y. McLNTIRE, R. D. ZIMMERMAN, Cochrane Environmental Systems, King of Prus Measurement of TrJOHN M. HALE, Orbisphere Laboratories, Geneva, Switzerland Development of a THOMAS C. ROGINSKI, Betz*Converse*Murdoch* Inc., Plymouth Meeting, Pennsylvania, HARRY B. GAYL Colloidal Silica Re ROBERT GUPTILL, Corvallis, Oregon, DANIEL L. COMSTOCK, Neptune Microfloc, Corvallis, Oregon, , SA Pre-Operational S ROBERT BAIRD, Halliburton Services, New Orleans, Louisiana ,JOHN A. TASSIN, Central Louisiana Electr Pilot Plant Testing BLAIR B. EMORY, UNC Nuclear Industries, Inc., Richland, Washington Simulation of PWRM. NOE, P. BESLU, G. FREJAVILLE, Commissariat a L’Energie Atomique, St. Paul lez Durance, France The Release of Cor DEREK H. LISTER, E. McALPINE, Atomic Energy of Canada, Ltd., Chalk River, Ontario, Canada, W. H. HO Removal of High LIRVING B. REMSEN, Consultant, Asbury Park, New Jersey Removal of High LV. A. IADAVAIA, E. J. CONNELLEY, Havens and Emerson, Inc., Saddle Brook, New Jersey Wastewater as a PR. C. SCHWARZ, Betz Laboratories, Inc., Trevose, Pennsylvania Wastewater as aProcedure PTERRENCE J. McMANUS, Betz*Converse*Murdoch* Inc., Plymouth Meeting, Pennsylvania A Modified Test for Predicting Polymer Performance in Belt Press Sludge MARTIN Dewatering M.Predicting SCHLESINGER, Corporation, A Modified Test Procedure for PolymerDravo Performance in Pittsburgh, Pennsylvania Belt Press Sludge DELMAR Dewatering H. PRAH, WILLIAM H. LINDENBERGER, , Nalco Chemical Company, Naperville, Illinois Combined AlkalineBARBARA L. CORMIER, U. S. Steel Corp., Pittsburgh, Pennsylvania Combined AlkalineLYNN A. KLEINVEHN, HARRIS E. DICKER, Aquatechnics, Inc., Naperville, Illinois Impact of the ProgR. G. BALMER, Exxon Research and Engineering Company, Florham Park, New Jersey Impact of the ProgANTHONY M. ARNONE, The Permutit Company, Paramus, New Jersey Controlling Clarif ANTHONY S. CANZONERI, L’eau Claire Systems, Kenner, Louisiana Controlling Clarif RICHARD P. CARDILE, STEPHEN D. CLARK, Nalco Chemical Company, Oak Brook, Illinois, JAMES J. AD An Alternative AppWAYNE C. MICHELETTI, Radian Corporation, Austin, Texas An Alternative AppD. A. LANGE-KENNEDY, E. L. MIHELIC, U.S. Steel Corporation, Pittsburgh, Pennsylvania, R. G. LUTHY, C Optimizing the Co DONALD BRENNEMAN, NUS Corporation, Pennsylvania Optimizing the Co RICHARD F. JACCARINO, Neptune Microfloc Company, Corvallis, Oregon PWR Secondary Wat JOHN A. MUNDIS, Electric Power Research Institute, Palo Alto, California Arresting of Stea LYNN K. MILLER, Salem Nuclear Generating Station, Public Service Electric and Gas, Hancock Bridge, Ne Florida Power and ALAN J. GOULD, Florida Power and Light Company, Miami, Florida PWR Steam Generat WILLIAM A. THORNTON, Virginia Electric and Power Company, Richmond, Virginia A Rational ApproacCHARLES J. SCHELL, Calgon Corporation, Pittsburgh, Pennsylvania A Rational ApproacWILLIAM G. CHARACKLIS, Montana State University, Bozeman, Montana Polymeric ChelatinDr. RODERICK A. CAMPBELL, Philip A. Hunt Chemical Corporation, Lincoln, Rhode Island Polymeric ChelatinALLEN E. FELTZIN, Hercules, Incorporated, Wilmington, Delaware The Use of SubtoxEDWARD F. KLEN, Northern Indiana Public Service Company, Hammond, Indiana The Use of SubToxPAUL J. KEMP, Midland Research, Laboratory, Inc., Lenexa, Kansas, NORMAN OKIMOTO, Hawaiian Elect The Use of SubtoxPAUL J. KEMP, Midland Research, Laboratory, Inc., Lenexa, Kansas, NORMAN OKIMOTO, Hawaiian Elect Operation and PerROBERT HELWICK, The Chester Engineers, Pittsburgh, Pennsylvania Operation and PerA. W. SIMON, M. E. TERRIL, B. A. BURKE, U.S. Steel Corporation, Pittsburgh, Pennsylvania Experiences with JOSEPH A. BATTAGLIA, Westinghouse Electric Corporation, Monroeville Nuclear Center, Pittsburgh, Penn Experiences with ALEX O. HOBBS, KENNETH FENNELL, Carolina Power and Light, Raleigh, North Carolina Spiral Ultrafiltra MARK F. HAYWARD, WILLIAM T. CHOATE, Abcor, Inc., Wilmington, Massachusetts Zero Cooling TowerEDGAR G. PAULSON, Consultant, Weston Connecticut Zero Cooling TowerPAUL R. PUCKORIUS, TEAGUE HARRIS, Puckorius & Dannenbaum, Evergreen, Colorado A Case History of JAMES D. MAVIS, CH2M Hill, Bellevue, Washington A Case History of NED P. SWANSON, Northwest Alloys, Addy, Washington, F. MARK SITTIG, Aluminum Company of Americ New Approach of Re LOUIS F. WIRTH, Consultant, Midland, Michigan New Approach of Re JAMES Y. CHEN, Belco Pollution Control, Parsippany, New Jersey, JAMES NICHOLS, Kansas City Power Two-Year Study onIRVING M. ABRAMS, Consultant, Los Altos, California Two-Year Study onDAVID C. AUERSWALD, FRANCES M. CUTLER, Southern California Edison, Paramount, California Comparison of TestE. D. NACE, Rohm & Haas Company, Spring House, Pennsylvania Neutralizing Retur D. G. CUISIA, J. W. RUDOLPH, C. M. HWA, E. A. TEHLE, Jr., Dearborn Chemical (U.S.), Lake Zurich, Illino

The Investigation JOHN A. KELLY, Nalco Chemical Company, Naperville, Illinois Evaluation of CalcTHOMAS A. BEINEKE, JOHN F. HALL, DAVID J. MORGAN, KATHY E. MARUGG, D. B. SCOTT, Combusti Performance of a H DOUGLAS N. RODGERS, HENRY H. ELLIOTT, General Electric Company, San Jose, California Achieving High ReTERRY M. O’NEAIL, OTTO KIRCHNER, WILLIAM J. DAY, Resources Conservation Company, Seattle, Wa The Graver-Elf/An ROBERT T. O’CONNELL, Graver Water Conditioning Company, Union, New Jersey Transport of Ionic JAMES K. RICE, Consulting Engineer, 0lney, Md. 20832 Transport of Ionic OTAKAR JONAS, Westinghouse Electric Corporation, Wilmington, Delaware Transport of Ionic OTAKAR JONAS, Westinghouse Electric Corporation, Wilmington, Delaware Corrosion Product WILLIAM R. GREENAWAY, NUS Corporation, Park West II, Cliff Mine Road, Pittsburgh, Pennsylvania 1527 Corrosion Product W. L. PEARL, S. G. SAWOCHKA, P. S. WALL, NWT Corporation, San Jose, California Steam Turbine EnvDAVID P. BOUR, NUS Corporation Steam Turbine EnvJ. ANELLO, J. C. BELLOWS, Westinghouse Electric Corporation, Philadelphia, Pennsylvania Liftbed- and RinseFRIEDRICH B. MARTINOLA, GUENTER SIEGERS, Bayer AG, Leverkusen, West Germany Sloughage of Organ SALLIE FISHER, GERARD OTTEN, Puricons, Inc., Berwyn, Pennsylvania Inorganic Ion Excha JASBIR S. GILL, Calgon Corporation, Pittsburgh, Pennsylvania Kinetic Performan N. J. RAY, M. BALL, A. COATES, Central Electricity Generating Board, Nottingham, England Effects of Caustic F. X. McGARVEY, S. M. ZIARKOWSKI, E. W. HAUSER, M. C. GOTTLIEB, Sybron/Chemical Division, Birmi Techniques for th R. J. GARTMANN, Transamerica Delaval, Florence, New Jersey Evaluation of Sulf DONALD R. BRENNEMAN, NUS Corporation, Park West II, Cliff Mine Road, Pittsburgh. PA 15275 Evaluation of Sulf INGRID E. WOERNER, ROBERT A. HOLZER, National Starch & Chemical Corporation, Bridgewater, New The Effect of Car D. J. GOLDSTEIN, R. E. HICKS, L. LIANG, Water Purification Associates, 238 Main Street, Cambridge, MA The Effect of Car RALPH D. BOROUGHS, JEAN E. McKEE, ROBERT D. MOSS, Tennessee Valley Authority, Chattanooga, T A Rapid Method foG. D. HANSEN A Rapid Method foPAUL G. BLYSTONE, PAUL E. LARSON, Hach Company, Loveland, Colorado Practical Applicat E. F. KLEN, Northern Indiana Public Service Company, Hammond, Indiana Practical Applicat ROBERT J. FERGUSON, M. J. SMAS, Apollo Technologies, Whippany, New Jersey Digichem™ 3000 Ch JUDITH RAWA, Calgon Corporation, Pittsburgh, Pennsylvania 15230 Digichem™ 3000 Ch RONALD A. PERREAULT, Ionics Incorporated, Watertown, Massachusetts Energy Losses in JAMES G. KNUDSEN, Oregon State University, Corvallis, Oregon 97331 Energy Losses in W. G. CHARACKLIS, NICHOLAS ZELVER, MUKESH H. TURAKHIA, FRANK L. ROE, Montana State Unive Experiences with CROBERT W. GRIFFIN, NUS Corporation, Park West II, Cliff Mine Road Pittsburgh, Pennsylvania 15275 Experiences with CPHILIP D. HILL, American Cyanamid Company, Bound Brook, New Jersey, PAUL F. PELOSI, Drew Chemic A Different ApproaI. M. ABRAMS, Duolite International, Inc., Redwood City, CA 94063 A Different ApproaRALPH E. MICKEL, Allegheny Power Service Corporation, Greensburg, Pennsylvania, GLENN R. HOLMES Operating ExperieR. HETHERINGTON, Epicor, Incorporated Operating ExperiePHILIP W. RENOUF, The Permutit Company of Australia Pty., Ltd., Sydney, N.S.W, Australia, J. H. SMITH, State-Of-The-Art L. F. WIRTH, Consultant, Midland, MI 48640 State-Of-The-Art STEPHEN J. ELMIGER, M. J. BELL, Babcock & Wilcox Company, Alliance, Ohio, C. S. WELTY, Electric Po 150 Million MegawLEONARD J. BOBICK, Consumers Power Company, Essexville, Michigan 150 Million MegawMIKE WADLINGTON, Texas Utilities Generating Company, Dallas, Texas, JOHN LONGO, Ecodyne/Graver 1981 INTERNATI Moderator: B. CHARLES MALLOY, Engineering Science Inc., Berwyn, Pennsylvania, Panelists: WILLIAM W Guidelines for Lay-DAVID E. SIMON, II, Cyrus WM. Rice Division, NUS Corporation, Pittsburgh, Pennsylvania Guidelines for Lay-JESSE BEECHER, Drew Chemical Corporation, Boonton, New Jersey, RUSSELL W. LANE, Consultant, Ch Guidelines for Lay-JESSE BEECHER, Drew Chemical Corporation, Boonton, New Jersey, RUSSELL W. LANE, Consultant, Ch Some Research Work PAUL COHEN, Consultant, Pittsburgh Some Research Work QIN JIN ZAO, HE HUI CHEN, Thermal Power Engineering Research Institute, Peking, Peoples Republic of Electromagnetic FilW. L. PEARL, S. G. SAWOCHKA, NWT Corporation, Jose, CA Electromagnetic FilJAMES BROWN, B. BZOVEY, D. J. R. DODD, J. Y. HARNOY, P. J. LEINONEN, Ontario Hydro Research D Continuous and GrANON Continuous and GrTHOMAS C. ROGINSKI, Betz*Converse*Murdoch, Inc., Plymouth Meeting, Pennsylvania, HARRY B. GAY Development of GrROBERT HELWICK, The Chester Engineers Inc. Coraopolis, PA.

Development of GrDANIEL THRELFALL, DONALD J. MESSINGER, MARK J. DOWIAK, Penn Environmental Consultants, P Rotary Kiln IncinerRICHARD P. MOFFA, Battelle-Columbus Laboratories, Columbus, Ohio 43201 Rotary Kiln IncinerTHOMAS RINKER, Environmental Elements Corporation, Baltimore, Maryland Disposal PropertieCHARLES L. SMITH, IU Conversion Systems, Inc., Horsham, PA 19044 Disposal PropertieWILLIAM C. WEBSTER, Webster & Associates, Norristown, Pennsylvania, JAMES R. DONNELLY, A/S Ni A Case History of JAMES H. POELLOT, D’Appolonia Consulting Engineers Inc., Pittsburgh, Pennsylvania A Case History of E. LEE PATTON, Union Carbide Corporation, Charleston, West Virginia Heating Boiler Tr CHARLES R. PETERS, Calgon Corporation, Pittsburgh, Pennsylvania Heating Boiler Tr R. KENT STULTZ, LEE G. HARROW, C. K. HUANG, D. C. HUTCHENS, G. H. HOEKFER, Omaha Public P The Thermal Hydrol ROBERT S. MITCHELL, Monsanto Company, St. Louis, Mo. 63166 The Thermal Hydrol R. BRIAN GOOD, JOSEPH B. AROTS, Hercules Incorporated, Wilmington, Delaware Boiler Scale Inhibi J. FRED WILKES, 143 Seventh Avenue LaGrange, Illinois Boiler Scale Inhibi LAWRENCE A. GAYLOR, SHERMAN J. SPRAGUE, The Mogul Corporation, Chagrin Falls, Ohio Deposit FormationF. H. SEELS, D. L. GIBBON, G. C. SIMON, Calgon Corporation, Pittsburgh, Pennsylvania, Operating ExperieROBERT M. QUINN, RQ Associates, P.0. Box 1205 Teaneck, N. J. 07666 Operating ExperieIVAN VERA, CADAFE, National Electric Company of Venezuela, Carabobo, Venezuela Operating ExperieTHOMAS M. LARONGE, Thomas M. Laronge, Inc., P.O. Box 4448 Vancouver, Washington Operating ExperieMICHAEL O’BRIEN, TERRANCE LATERRA, Ecodyne/Graver Water Division, Union, New Jersey Cooling Water UseJ. W. SIEGMUND, Sheppard T. Powell Associates, Baltimore, Maryland Cooling Water UseW. K. JING, S. H. CHIANG, University of Pittsburgh, Pittsburgh, Pennsylvania What a User ExpecJAMES A. BAUMBACH What a User ExpecROBERT J. FRANCO, Exxon Research & Engineering Company, Florham Park, New Jersey The Status of DeeMIKE MADDAGIRI, JAMES RIOS, Bechtel Power Corp., San Francisco, California The Status of DeeLOUIS F. WIRTH, Jr. Dow Chemical Company, Midland, Michigan The Status of DeeLOUIS F. WIRTH, Jr. Dow Chemical Company, Midland, Michigan Denitrification Bl S. R. TRESSIDER, Perolin-Bird Archer, Ltd., Cobourg, Ontario, Canada, Dr. C.T. CORKE, Department of En Optimization of SuP. VULLIEZ-SERMET, E. ZAGANIARIS, Rohm & Haas European Laboratories, Valbonne, France The Use of an AutoJ. S. CARDARELLI, G. J. KRALIK, A. T. KASHUBA,, Betz Laboratories, Inc., Trevose, Pennsylvania The Use of the UpfGEORGE GOYAK, WILLIAM O’DONNELL, PHILIP W. SPRAKER, MICHAEL J. TELEPCHAK, Sybron Corp A New Oxygen Scav D. M. BLOOM , L. R. GESS, Nalco Chemical Company, Oak Brook, Illinois The Use of Sulfon PETER E. GREENLIMB, D. ANTHONY CARTER, Dearborn Chemical Company, Lake Zurich, Illinois Alternative Dewat F. JANKOWSKI, W. A. PABST, IU Conversion Systems, Horsham, Pennsylvania The Use of Comput CHARLES J. SCHELL, Calgon Corporation, Pittsburgh, Pennsylvania Computerized Water DONALD A. JOHNSON, KENNETH E. FULKS, Nalco Chemical Company, Oak Brook, Illinois Calculations of CoWINSTON CHOW, Electric Power Research, Institute, Palo Alto, CaliforniaJ, JOHN T. ARONSON, Stearns Procedure for MeaS. J. ELMIGER, J. M. KIBLER, E. D. YOCHHEIM, Babcock & Wilcox, Alliance, Ohio, W. D. MILLS, Toledo E PROGRESS REPORT: GEORGE J. CRITS, D. R. ZIMMERMAN, J. P. PRATT, Cochrane Environmental Systems, King of Prussia, Pilot Scale EvaluaL. T. CIPLIJAUSKAS, B. J. FUHR, J. K. LIU, Syncrude Canada, Ltd., Edmonton, Alberta, Canada Continuous Ion ExALEX HIMSLEY, Himsley Engineering Ltd., Toronto, Ontario, Canada Standardization ofSALLIE FISHER, GERARD OTTEN, Puricons, Inc., Berwyn, Pennsylvania Power Boilers – C WILLIAM L. TRACE, JERRY L WALKER,Calgon Corporation, Pittsburgh, Pennsylvania Modeling the Effe J. LEIBOVITZ, S. G. SAWOCHKA, NWT Corporation, San Jose, California Computation of Che P. V. BALAKRISHNAN, Atomic Energy of Canada, Ltd., Chalk River, Ontario, Canada Calculations of S R. J. BAWDEN, R. GARNSEY, Central Electricity Research Laboratories, Leatherbead, Surrey, England, L. A Study of CorrosiW. S. LEEDY, Babcock & Wilcox, Alliance, Ohio A Study of CorrosiW. W. FRENIER, K. J. LOESCHER, Dowell Division, Dow Chemical Company, Tulsa, Oklahoma The Chemical CleaKENNETH D. SCHOOF, Central Illinois Public Service Co., Coffeen, Illinois The Chemical CleaH. DEWEY JOHNSON, MICHAEL B. LAWSON, Halliburton Services, Duncan, Oklahoma Preoperational Cl CARL E. WISE, Powell Division, The Dow Chemical Company, Jacksonville, Florida Preoperational Cl ARTHUR. W. FYNSK, RICHARD. T. HARRIS, E.I. du Pont de Nemours & Company, Wilmington, Delaware Preoperational Cl ARTHUR W. FYNSK, RICHARD T. HARRIS, E.I. du Pont de Nemours & Company, Wilmington, Delaware

Fluoride Cleaning JOHN T. DILLMAN, Halliburton Services, Pittsburgh, Pennsylvania Fluoride Cleaning EDWARD A. YORKGITIS, U.S. Steel Corporation, Pittsburgh, Pennsylvania Fluoride Cleaning EDWARD A. YORKGITIS, U.S. Steel Corporation, Pittsburgh, Pennsylvania Estimates of ThermCECIL M. CRISS, University of Miami, Coral Gables, Florida Chemical Equilibr R. C. MURRAY, Jr., JAMES W. COBBLE, San Diego State University, San Diego, California Estimation of ConcW. T. LINDSAY., Jr., Westinghouse Electric Corporation, Pittsburgh, Pennsylvania Evaluation of MinerJOSEPH G. CARROLL, Ciba-Geigy Corporation, Ardsley, New York Evaluation of MinerJASBIR S. GILL, Calgon Corporation, Pittsburgh, Pennsylvania The Effect of Polym GERALD D. HANSEN, VICTORIA D. POWELL, ARCO Performance Chemicals, Glenolden, Pennsylvania Corrosion Control A. S. KRISHER, Monsanto Company, St. Louis, Missouri Corrosion Control WILLIAM J. WARD,

rporation, Greenbelt, Maryland

TON, Electric Power Research Institute, Palo Alto, California

ectric Power Research Institute, Palo Alto, California

., Wilmington, Delaware

ent, City of Industry, California

sity, Stanford, California

Research Institute, Palo Alto, California

xchanger Consultant, Sunnyvale, California

ation/Ionac Division, Birmingham, New Jersey, WILLIAM S. MILLER, Ecolochem, Incorporated, Norfolk, Virginia

um Company, Houston, Texas,

Company, Lynchburg, Virginia,

Ontario, Canada

esearch Institute, Palo Alto California, W. C. MICHELETTI, Radian Corporation, Austin, Texas, R. V. LONG, Stone and Webster Engineering

urgh, Pennsylvania

ton Rouge, Louisiana, KENNETH L. ATWOOD, Halliburton Services, Gaithersburg, Maryland

n Company, Seattle, Washington,

HEW, Chemical Manufacturers Association, Washington, DC, JAMES E. KERRIGAN, AMAX Environmental Services, Inc., Golden, Colorad

Columbia, Maryland,

Company, Omaha, Nebraska

, Radian Company, Austin, Texas,

, City of Austin, Austin, Texas

er Research Institute, Palo Alto, California er Research Institute, Palo Alto, California

[M. McDOWELL, Carolina Power & Light, Raleigh, North Carolina]

erly, Massachusetts,

N, Bechtel National, Inc., San Francisco, California rnia, J. A. BAUMBACH, Nalco Chemical Company

cox Company, Barberton, Ohio

NTNER, Los Angeles Dept. of Water and Power, Los Angeles, Calif., N. C. MOHN, Combustion Engineering, Windsor, Connecticut

Japan, MYLES H. KLEPER, JOSEPH MARKIND, ROBERT L. GOLDSMITH , JAMES M. RYAN, Abcor, Inc., Wilmington, Massachusetts Japan, MYLES H. KLEPER, JOSEPH MARKIND, ROBERT L. GOLDSMITH , JAMES M. RYAN, Abcor, Inc., Wilmington, Massachusetts

ogy Consultants, Chicago, Illinois ogy Consultants, Chicago, Illinois

s, Inc., Cambridge, Mass. s, Inc., Cambridge, Mass.

FELDMAN, Westinghouse Research and Development Laboratories, Pittsburgh, Pennsylvania

Station, Duke Power Company, Seneca, South Carolina

SLAU, P. R. LAMBERT, Romicon, Inc., Philadelphia, Pennsylvania

&D, Paris, France)

WILMSHURST, Electric Power Research Institute, Charlotte, NC WILMSHURST, Electric Power Research Institute, Charlotte, NC

S, Ph.D., FAIC, Environmental Health Sevices, Tulane University, New Orleans, LA

MED YASSEN, University of Basrah, Iraq

are System, Pittsburgh, PA

gy, Tampa, FL; LNSP NAGGHAPPAN, ROBERT HELWICK, Veolia Water Systems, Pittsburgh, PA

n, AL; DAN ROBINETTE, Puckorius & Associates, Inc. Evergreen, CO n, AL; DAN ROBINETTE, Puckorius & Associates, Inc. Evergreen, CO

ne Light Co., Pittsburgh, PA.

A. OAD, G. I. OGUNDELE, R. CLAUDI, Ontario Technologies,, Toronto, Ontario, Canada

Puckorius & Associates, Evergreen, CO

ompany, HIDEKI TAKIGUCHI, TAKU OHIRA, The Japan Atomic Power Company, JAMES R. STAHLBUSH, The Dow Chemical Company

UKI, Tokyo Electric Power Company, Tokyo, Japan, AKIRA MATSUMOTO, JAMES R. STAHLBUSH, The Dow Chemical Company, Midland,

s, K. TITTLE, Consultant, Mawdesley Ormskirk, Lancaster, United Kingdom

a, LOU WIRTH, Midland, Michigan

AKIRA MATSUMOTO, JAMES R. STAHLBUSH, The Dow Chemical Company, Midland, Michigan

l, Walnut Creek, California, W. HUGH GOODMAN, Nalco Chemical Company, Naperville, Illinois

ter, Muenster, Germany ter, Muenster, Germany

Services, Brighton, Colorado

Electric Power Research Institute, Palo Alto, California

s, Austin, Texas

assachusetts

AWA, Tokyo Electric Power Company, Tokyo, Japan

KOBAYASHI, Pennsylvania Power and Light Company, Washingtonville, Pennsylvania, WAYNE J. MARTIN, Battelle Pacific Northwest Labo OBAYASHI, Pennsylvania Power and Light Company, Washingtonville, Pennsylvania, WAYNE J. MARTIN, Battelle Pacific Northwest Labora

Palo Alto, California Palo Alto, California

oard), Varobacka, Sweden oard), Varobacka, Sweden

stitute, Palo Alto, California stitute, Palo Alto, California

ornia, KAN LIN, Bechtel Power Corporation, San Francisco, California

RIS JR., GPU Nuclear Corporation, Reading, Pennsylvania,

nta, Georgia

Stone and Webster Engineering Corporation, Boston, Massachusetts

Services, Inc., Golden, Colorado, JACQUELINE RAMS, ASTM, WMO, Washington, DC

g, Windsor, Connecticut

., Wilmington, Massachusetts ., Wilmington, Massachusetts

The Dow Chemical Company

w Chemical Company, Midland, Michigan

, Battelle Pacific Northwest Laboratories, Richland, Washington Battelle Pacific Northwest Laboratories, Richland, Washington


【#4】Aes New York 2021 ” Spatial Audio Track Event Details

Wednesday, October 18, 9:00 am – 10:00 am (Rm 1E13)

Spatial Audio: SA01 – Crash Course in 3D Audio

Presenter:

Nuno Fonseca, Polytechnic Institute of Leiria – Leiria, Portugal; Sound Particles

A little confused with all the new 3D formats out there? Although most 3D audio concepts already exist for decades, the interest in 3D audio has increased in recent years, with the new immersive formats for cinema or the rebirth of Virtual Reality (VR). This tutorial will psent the most common 3D audio concepts, formats, and technologies allowing you to finally understand buzzwords like Ambisonics/HOA, Binaural, HRTF/HRIR, channel-based audio, object-based audio, Dolby Atmos, Auro 3D, among others.

Wednesday, October 18, 9:30 am – 11:00 am (Rm 1E14)

Spatial Audio: SA02 – Riding the Faders: How to Stay Ahead of the Curve in Immersive Audio

VR, AR and XR have highlighted the importance of sound for “psence” in all content types from live TV to film to games. This repsents a unique opportunity for audio professionals. But making the transition to the new world of immersive realities is no easy task. Here you will learn from industry professionals who are creating new 3D audio workflows and growing professional networks while remaining firmly grounded in their craft. The Manhattan Producers’ Alliance is a New York/San Francisco-based membership organization comprised of engineers, composers and producers. Our focus is on nurturing personal creativity within the art and craft of music making.

AES Members can watch a video of this session for free.

Wednesday, October 18, 9:30 am – 10:30 am (Rm 1E06 – PMC Room)

Spatial Audio: SA19 – Sound Design in 3D for Dance Music

Presenter:

Lasse Nipkow, Silent Work LLC – Zurich, Switzerland

Producers of Dance Music pursue the target to put

nightclub audiences into trance.

3D audio includes a high

potential to increase sound impssions significantly and engage the emotions, as is the case in film sound. 3D audio allows to create spectacular spatiality and sound sources around the listener. There are however some challenges: Clubbers on the dance floor are not oriented in the same way as listener of classical music in the concert hall because they are dancing. PA systems of music clubs are nearly exclusively designed for mono and stereo reproduction. Therefore, the impact by changing to a 3D audio system is significant.

During the psentation, the possibilities of musical design in consideration of psychoacoustic phenomena will be described and demonstrated with example recordings and videos.

Wednesday, October 18, 11:00 am – 12:00 pm (Rm 1E06 – PMC Room)

Spatial Audio: SA16 – The Butterfly FX-Unintended Consequences of Mastering Tools

Presenter:

Piper Payne, Neato Mastering – San Francisco Bay Area, CA

Wednesday, October 18, 11:30 am – 12:30 pm (Rm 1E13)

Spatial Audio: SA03 – Spatial Music: Producing, Performing, and Publishing Music in 3D

Attendees will learn to produce immersive spatial music in an emerging market with familiar tools. Using examples of spatially mixed recordings from award winning artists, the tutorial will denote different approaches in various genres including EDM, Pop, and Jazz among others. Additional takeaways include various ways in which produced content can translate to a live performance setting while subsequently being released on various online platforms that support spatial formats.

Tutorial Will Cover:

* Basic Concepts in Spatial Audio (HRTFs & Ambisonics)

* Tools, Signal Flow, and Practical Implementation of 3D Audio technology

* Creative Tips and Best Practices when producing a track in 3D

AES Members can watch a video of this session for free.

Wednesday, October 18, 2:00 pm – 3:00 pm (Rm 1E06 – PMC Room)

Spatial Audio: SA04 – Spatial Audio for Multi-Tracked Recordings-Workflows, Phase Relationships, Equalization, Reverberation and Delivery Specs

VR and AR are making a big push onto the world’s stage with projected revenues to exceed $120B by 2021. Spatial audio is an integral part of this, albeit in Ambisonics, Dolby Atmos, or other formats. In this tutorial we will learn about the differences and similarities between spatial and stereo workflows, illustrated by real-world examples. We will discuss binaural rendering engines and its impact on phase coloration and frequency response, phase relationships of stereo and mono sources in the Ambisonic soundfield, and loudness management for different delivery platforms.

Wednesday, October 18, 4:00 pm – 5:00 pm (Rm 1E14)

Spatial Audio: SA05 – The State of the Art of Binaural Audio for Loudspeakers and Headphones

Presenter:

Edgar Choueiri, Princeton University – Princeton, NJ, USA

I will describe the challenges of binaural audio through headphones (BAH) and loudspeakers (BAL), recent solutions to these challenges, and the state of the art of binaural processing and content development tools. In particular I will describe BACCH 3D Sound processing, which relies on optimal crosstalk cancellation filters, head tracking and automatic inpidualization to deliver accurate 3D imaging from binaural audio. I will then describe the recently developed BACCH-HP headphones processing, which significantly enhances the robustness of 3D imaging and the ability to head-externalize binaural audio. I will use the powerful BACCH-dSP software, which allows designing BACCH filters for BAL and BAH, processing binaural audio, translational and rotational head tracking, and 3D mixing, to illustrate the talk and demonstrate the technologies.

AES Members can watch a video of this session for free.

Thursday, October 19, 9:00 am – 12:00 pm (Rm 1E11)

Paper Session: P06 – Spatial Audio-Part 1

Thursday, October 19, 9:30 am – 11:30 am (Rm 1E06 – PMC Room)

Spatial Audio: SA17 – PMC: A 9.1 Musical Experience

Presenters:

Morten Lindberg, 2L (Lindberg Lyd AS) – Oslo, Norway

Daniel Shores, Sono Luminus – Boyce, VA, USA; Shenandoah Conservatory Music Production and Recording Technology – Winchester, VA, USA

Thursday, October 19, 2:00 pm – 3:30 pm (Poster Area)

Poster: P11 – Spatial Audio

P11-1 Deep Neural Network Based HRTF Personalization Using Anthropometric MeasurementsChan Jun Chun, Korea Institute of Civil Engineering and Building Technology (KICT) – Goyang, Korea; Jung Min Moon, Gwangju Institute of Science and Technology (GIST) – Gwangju. Korea; Geon Woo Lee, Gwangju Institute of Science and Technology (GIST) – Gwangju. Korea; Nam Kyun Kim, Gwangju Institute of Science and Technology (GIST) – Gwangju, Korea; Hong Kook Kim, Gwangju Institute of Science and Tech (GIST) – Gwangju, Korea

A head-related transfer function (HRTF) is a very simple and powerful tool for producing spatial sound by filtering monaural sound. It repsents the effects of the head, body, and pinna as well as the pathway from a given source position to a listener’s ears. Unfortunately, while the characteristics of HRTF differ slightly from person to person, it is usual to use the HRIR that is averaged over all the subjects. In addition, it is difficult to measure inpidual HRTFs for all horizontal and vertical directions. Thus, this paper proposes a deep neural network (DNN)-based HRTF personalization method using anthropometric measurements. To this end, the CIPIC HRTF database, which is a public domain database of HRTF measurements, is analyzed to generate a DNN model for HRTF personalization. The input features for the DNN are taken as the anthropometric measurements, including the head, torso, and pinna information. Additionally, the output labels are taken as the head-related impulse response (HRIR) samples of a left ear. The performance of the proposed method is evaluated by computing the root-mean-square error (RMSE) and log-spectral distortion (LSD) between the referenced HRIR and the estimated one by the proposed method. Consequently, it is shown that the RMSE and LSD for the estimated HRIR are smaller than those of the HRIR averaged over all the subjects from the CIPIC HRTF database.

Convention Paper 9860 (Purchase now)

P11-2 The Upmix Method for 22.2 Multichannel Sound Using Phase Randomized Impulse ResponsesToru Kamekawa, Tokyo University of the Arts – Adachi-ku, Tokyo, Japan; Atsushi Marui, Tokyo University of the Arts – Tokyo, Japan

The upmix technique for 22.2 multichannel sound was studied using room impulse responses (RIRs) processed by phase randomized technique. From the result of the first experiment, the spatial impssion of proposed method was close to the original sound, but the timbre differed. In the second experiment we pided the RIRs at the moment when the diffuse reverberation tail begins (mixing time) by two kinds of time, namely fixed to 80 msec and different mixing times for each frequency band. From the result, the similarity of proposed methods and the original sound was improved, however, it is suggested that the similarity of the timbre depends on the sound sources and the suitable mixing time of RIRs.

Convention Paper 9861 (Purchase now)

P11-3 A 3D Sound Localization System Using Two Side Loudspeaker MatricesYoshihiko Sato, University of Aizu – Aizuwakamatsu-shi, Fukushima, Japan; , University of Aizu – Aizuwakamatsu City, Japan; , University of Aizu – Aizuwakamatsu City, Japan

We have proposed a new 3D sound reproduction system that consists of two side loudspeaker matrices each with four loudspeakers. The 3D sound images that applied to this system were created by the amplitude panning method and convolution of head-related transfer function (HRTF). In our past research we used the loudspeaker matrices arranged as a square shape, nevertheless the accuracy of sound image localization should be improved. We changed the shape of loudspeaker matrices from a square to a diamond by rotating 45 degrees to improve direction perception. As a result, we could be closer the localized sound images to intended directions than the square-shaped loudspeaker matrices by implementing the diamond-shaped loudspeaker matrices.

Convention Paper 9862 (Purchase now)

P11-4 Optimization of Interactive Binaural Processing François Salmon, CMAP – Ecole Polytechnique – Paris, France; Ecole nationale supérieure Louis-Lumière – Paris, France; Matthieu Aussal, CMAP – Ecole Polytechnique – Paris, France; Etienne Hendrickx, University of Brest – Paris, France; Jean-Christophe Messonnier, CNSMDP Conservatoire de Paris – Paris, France; Laurent Millot, ENS Louis-Lumière – Paris, France; Acte Institute (UMR 8218, CNRS/University Paris 1) – Paris, France

Several monitoring devices may be involved during a post-production. Given its lower cost and practical aspects, head-tracked binaural processing could be helpful for professionals to monitor spatialized audio contents. However, this technology provides significant spectral coloration in some sound incidences and suffers from its current comparison to a stereophonic signal reproduced through headphones. Therefore, different processing methods are proposed to optimize the binaural rendering and to find a new balance between externalization and timbral coloration. For this purpose, the alteration of the HRTF spectral cues in the frontal area only has been studied. In order to evaluate the accuracy of such treatments, listening tests were conducted. One HRTF processing method offered as much externalization as the original HRTFs while having a closer timbre quality to the original stereo signal.

Convention Paper 9863 (Purchase now)

P11-5 A Direct Comparison of Localization Performance When Using First, Third, and Fifth Ambisonics Order for Real Loudspeaker and Virtual Loudspeaker RenderingLewis Thresh, University of York – York, UK; Cal Armstrong, University of York – York, UK; Gavin Kearney, University of York – York, UK

Ambisonics is being used in applications such as virtual reality to render 3-dimensional sound fields over headphones through the use of virtual loudspeakers, the performance of which has pviously been assessed up to t