Evaluation of Expert Systems
Review articleOpen access
1989/01/01 Chapter DOI: 10.1016/B978-0-444-87321-7.50019-3
Publisher SummaryThis chapter discusses some of the essential aspects of expert system evaluation. The basis was the increasing use of expert systems in real applications, hence, the growing need to evaluate their functionality as a natural extension to the software engineering aspects of the design and development process. The reliability of an evaluation method can be established using techniques commonly known in other fields, whereas the validity of the method required more thorough considerations of the content validity and the construct validity. It was pointed out that whereas the validity can be reasonably well assessed by looking only at the results or output from the expert system, assessing the reliability also requires that the mechanisms of the system are studied in some detail. This should be done not only for the cases where something goes wrong, that is, as a part of a diagnosis, but also for normal conditions. Following that a number of frequently used evaluation methods were discussed using the classification established above. The methods include Turing's test, expert assessment, and statistical sampling, as well as some more specialized approaches.
Request full text