AVMiner - A Prototype of Audio-Visual Broadcast Transcription System

This paper focuses on the use of methods and algorithms from the area of speech processing and recognition and from the area of machine vision for designing of system for automatic audio-visual broadcast transcription. The resulting audio-visual system has been designed and created mainly for transcription of huge video databases with TV recordings in this work. The visual signal processing and recognition is usually several times computationally more demanding than audio signal processing and recognition. Therefore, all applied machine vision methods and algorithms were considered with respect to low computing time as well as the highest possible recognition rate. Our proposed broadcast transcription system was extended by several modules for visual signal segmentation, for TV channel identification, for face detection and identification and for Optical Character Recognition (OCR).

A system with graphical editor was created to display results from the prototype of audio-visual broadcast transcription system whose parts are described above. This system also allows to search on recognized TV recordings. At present, it is possible to search Key-words or short phrases in text from OCR and in text from audio signal transcription. At the same time, it is possible to search for identified speakers (people) from an acoustic or visual signal.


Source video data

Output from audio transcription
Output from visual transcription

Video - Result in AVMiner program