The twelve properties academic papers are using to validate Neural Networks and AI. I recently went to an AI conference, where many academics and industries presented their AI findings. One particular keynote highlighted a 2023 paper that introduced a set of properties and metrics for evaluating Neural Network filters and prototypes. This framework is gaining traction within the PhD community in Computer Science.
Currently, there is no universal metric for evaluating an AI, and most are case-by-case quantitative analyses or qualitative studies of how users perceive the outputs. However, the 2023 paper represents a significant step toward standardization.
These 12-co Properties are a list of comprehensible guidelines for your ML models. And with hopes that it gets adopted by more AI practitioners in the industry, the 12-co properties are listed below.
.png) |
| Image created by Microsoft Copilot |
There are 3 categories, and 12 sub-categories:
Content
Correctness: How faithful the explanation is with respect to the prediction model.
Evaluate prototype visualization with synthetic data or incremental deletion/addition of image patches.
Completeness: How much of the model behavior is described in the explanation.
Output-complete by design.
Evaluate human-output-completeness with simulatibility user studies.
Consistency: How deterministic and implementation-invariant the explanation method is.
Implementation invariance and nondeterminism.
Continuity: How continuous and generalizable the explanation is.
Stability for slight variation.
Contrastivity: How discriminative the explanation is with respect to other events.
Contrastive by design; can answer counterfactual questions.
Pragmatism and compactness for optimal contrastive explanation.
Target-sensitivity for location of prototypes.
Target-discriminativeness to evaluate prototypes.
Covariate Complexity: How complex the (interactions between) features in the explanation are.
Prototype homogeneity/purity with annotated data.
Perceived homogeneity with user studies (subjective).
Intruder detection for objective homogeneity evaluation.
Presentation
Compactness: The size of the explanation.
Number of prototypes (local & global).
Number of unique prototypes (redundancy).
Composition: The presentation and organization of the explanation.
Compare different explanation formats with the same content.
User study on how to present part-prototypes.
User study on classification format (e.g., linear layer or decision tree).
Confidence: The presence and accuracy of probability information in the explanation.
Reliability of classification confidence.
Reliability of explanation confidence.
Out-of-distribution detection confidence.
User
Context: How relevant the explanation is to the user.
User studies (lab and field).
Coherence: How accordant the explanation is with prior knowledge and beliefs.
Anecdotal evidence by visualizing reasoning with prototypes.
Alignment with domain knowledge from annotated data.
Subjective satisfaction with user studies.
Controllability: How interactive or controllable the explanation is.
GUI for interactive and personalized explanations.
Explanatory debugging to manipulate prototypes and model’s reasoning.
The paper further discusses these properties in detail and offers recommendations on how to evaluate them further. They cite Jacovi & Goldberg's paper, expanding on the notion that interpretable methods should be held to the same standards as post-hoc explanation methods.
Though the paper was initially intended to create an interpretable part-prototype learning method, many concepts extend beyond the prototyping stage to the production of AI use cases. The overarching themes in AI seem to be governance and compliance, reducing hallucinations, and increasing explainability. The 12-Co Properties are a great start on the metrics needed for those conversations.
Yorumlar
Yorum Gönder