WNCG Seminar: Audio-Visual Quality Assessment Methodologies for Multimedia Applications

The great progress achieved by communications in the last twenty years can be attested by the amount of audio-visual multimedia services available nowadays, such as digital television and IP-based video transmission. The success of these kind of services relies on their trustworthiness and the delivered quality of experience. Therefore, the development of efficient real-time quality monitoring tools that can quantify the audio-visual experience (as perceived by the end user) is key to the success of any multimedia service or application. Although the research in audio and video quality assessment (tackled as individual modalities) is mature, there are still several issues that need to be addressed in the area of audio-visual quality. For instance, modeling how humans perceive audio and video signals (altogether) is a challenging task, especially when we consider the interaction in the perception of audio and video signals. In this talk, I will briefly discuss state-of-the-art research of audio-visual quality metrics. Then, I will present a no-reference pixel-based audio-visual quality metric, which uses autoencoders to generate a set of descriptive audio and video features. The method is based on the assumption that the autoencoders are able to produce a set of features that can describe the complex audio and video interactions. When tested on available datasets, the proposed method achieves a good performance. Finally, I will discuss some of the challenges faced by the area of audio-visual quality and potential research directions.