Following a successful identification, the client is allowed to query the server for the desired context-based metadata by specifying a target time interval. During playback, the client performs fingerprint extraction and transfers the fingerprints to the server which in turn performs the task of audio identification. The server implements a scheduler component which assigns a task ID to each incoming client request, hence facilitating multiple concurrent requests.

Our client-server implementation is Java -based using a C++ -library for fingerprint extraction, index creation and audio identification. In the following we discuss some issues on the implemented system. For the client application we implemented an enhanced audio player capable of fingerprint extraction, client-server communication, and synchronous display of lyrics during playback. As there are currently no general approaches to automatic PCM- lyrics synchronization available, this step is carried out manually. Sync files are created from a corresponding collection of lyrics for all of the songs contained in our audio collection. In our test setting the FPDB is created using a previously proposed audio fingerprinting technique. 2 shows an overview of a corresponding service for real-time delivery of context-based lyrics information. Note that when using a suitable synchronization protocol between client and server, a repeated fingerprint-based identification of the current playback position is not necessary. The server then retrieves all available data for that time interval from the sync file.

During subsequent playback, the client may request any further context-based information on the identified piece of audio by simply specifying a target time interval. For example, in a karaoke-like scenario one would prob- ably want to display entire lines of lyrics in advance and additionally highlight single words while they are actually sung. Note that the actual visualization depends on the particular type of application. During playback the client displays the information synchronously with the acoustic signal. Using a suitable communication protocol, the information is then transferred to the client. If successful, the server retrieves the context-based information from the sync file that is available for the current playback position. The extracted fingerprint is transmitted to the server, which then tries to identify the audio piece and current playback position. The user selects a particular audio piece for playback, which then undergoes a local fingerprint extraction at the current playback position. At runtime, we assume that a user is equipped with a client application which basically consists of an extended audio player. Both the FPDB and sync files are stored on the server. As a result we obtain a so-called sync file which allows us to access the available context-based information for each time interval of a particular piece of audio. Technically, the linking may be performed by specialized synchroniza- tion algorithms such as in the score-PCM scenario. Below, a corresponding illustration is given for a MIDI-PCM synchronization. The upper part of the figure illustrates the concept of score- PCM synchronization where a link between a symbolic note event and its corresponding physical realization is in- dicated by an arrow. 1 shows a score-, PCM-, and MIDI-version of the first mea- sures of J.S. This amounts to assigning a particular starting position and du- ration to each basic component, e.g., a single word in the lyrics scenario or a single note in the score scenario. The second part of preprocessing consists of linking the context-based information for each audio piece to the actual time-line of that piece.

Syncplay server status