Demystifying Face-Recognition with Locally Interpretable Boosted Features (LIBF)

Understanding and exposing decisions and wrong classifications of face-recognition systems is very important for social (and even legal) acceptance of high person-specific, biometric recognition techniques. In this paper we present first insights gained during the development of Locally Interpretable Boosted Features (LIBFs) – a set of well-located and easily interpretable image features by humans – for explaining the decisions of a state of the art face-recognition pipeline. In particular, we show initial results on applying an unlabelled, self-supervised embedding to specific regions of the face image for face verification tasks following the IARPA Janus Benchmark-B (IJB-B) and Benchmark-C (IJB-C) verification protocols. In addition we discuss some issues raised when applying explainable boosting machine (EBM), a recently developed, high performance explanation technique, to the features obtained