Progress Report on Implementation of Evaluation Metrics and Protocols (v1)


This deliverable reports the progress of the second task of the second work package in the XAIface project (T2.2). It focuses on the methodology for efficient performance assessment, the design of efficient metrics and protocols, and the overall benchmarking process for face recognition systems This document summarizes the activities as follows. A new methodology of performance assessment is firstly proposed. It measures the impact of different types of influencing factors, which is in line with the content in T2.1, on the performance of deep face recognition systems. Then, a benchmarking process is defined to evaluate a face recognition system under two face recognition tasks. Four key elements of a benchmarking process are explained in detail, including two state-of-the-art reference deep face recognition pipelines, multiple face recognition databases, experimental protocols for the two face recognition tasks, and the performance metrics used to report the results. In the end, the benchmarking process is summarized with examples of key instantiations for each dimension.