Real-Time Audio
  • Platform
  • Framework / Engine
  • iOS
  • Android
  • macOS
  • Windows
  • Linux
  • Web
  • WeChat Mini Program

Media player

Update Time:2021-07-23 23:07

1 Introduction

The media player component of the ZEGO Express SDK provides the following abilities:

  • Play audio and video files.
  • Simultaneously stream out the audio/video data of the media file being played.

This document describes how to use the media player in your apps.

1.1 Use cases

You can use the media player for various use cases. The following are some typical examples:

  • Verify audio output device: Play test audio to verify that the audio playback device is working correctly.
  • Play background music: Play and stream audio to remote users so that you can add background music to your live stream.
  • Play video files: Play and stream video to remote users to provide a shared watching experience.

1.2 Supported media file formats

The media player supports MP3, MP4, FLV, WAV, AAC, M3U8, and other media file formats.

By default, the media player supports these media file formats: MP3, MP4, FLV, WAV, and AAC. If you need support for other formats, contact ZEGO Technical Support.

1.3 Supported protocols

The media player supports playing media resources from the internet over the HTTP or HTTPS protocol.

2 Sample code

For the sample code related to this feature, check out the source files in the /ZegoExpressExample/Mediaplayer directory in the ZEGO Express SDK sample code package.

For information about how to download and run the sample code, see Sample codes.

3 Prerequisites

Before you begin to use the media player in your project, make sure you complete the following steps:

  1. Create a project in the ZEGO Admin Console and get the AppID and AppSign of your project. For details, see ZEGO Admin Console - Project management.

  2. Integrate the ZEGO Express SDK into your project, and implement the basic real-time audio and video features with the SDK. For information about how to get started with the SDK, see Quick starts.

4 Implementation steps

4.1 Create media player instances

To create a media player instance, call the createMediaPlayer method of the engine (the ZegoExpressEngine instance you create when initializing the SDK).

A media player instance can only play one audio and video source at a time. You can create up to 4 media player instances to play multiple media resources at the same time. If there are already 4 media player instances running, calling this method will return null.

  • Function prototype:

    /**
     * Create a media player instance
     *
     * The ZegoExpressEngine class supports up to 4 media player instances. If there are already 4 media player instance running, calling this method returns `null`.
     * @return: The returned media player instance. 
     */
    public ZegoMediaPlayer createMediaPlayer();
  • Sample code:

    ZegoMediaplayer mediaplayer = mEngine.createMediaPlayer();
    if (mediaplayer != null) {
        Log.d(TAG, "createMediaPlayer sucess.");
    } else {
        Log.d(TAG, "createMediaPlayer failed.");
    }

4.2 Optional: Listen for and handle media player events

Set up an event handler for the media player Call the media player's [`setEventHandler`|_blank](/article/api?doc=express_video_sdk_API~Java~class~im-zego-zegoexpress-zego-media-player#set-event-handler) method to set up an event handler to listen for and handle various media player events, such as changes in the media player's current state, playback progress, and network status.
  • Function prototype:

    public abstract class IZegoMediaPlayerEventHandler {
    
        public void onMediaPlayerStateUpdate(ZegoMediaPlayer mediaPlayer, ZegoMediaPlayerState state, int errorCode){
    
        }
    
        public void onMediaPlayerNetworkEvent(ZegoMediaPlayer mediaPlayer, ZegoMediaPlayerNetworkEvent networkEvent){
    
        }
    
        public void onMediaPlayerPlayingProgress(ZegoMediaPlayer mediaPlayer, long millisecond){
    
        }
    
    }
  • Sample code:

    public class MyIZegoMediaPlayerEventHandler extends IZegoMediaPlayerEventHandler {
    
        private static final String TAG = "MyIZegoExpressMediaplay";
    
        @Override
        public void onMediaPlayerStateUpdate(ZegoMediaPlayer mediaPlayer, ZegoMediaPlayerState state, int errorCode) {
            // This event callback is triggered on the UI thread. You can make corresponding UI changes here. For example, to refresh the Play button.
            Log.d(TAG, "onMediaPlayerStateUpdate: state = " + state.value() + ", errorCode = " + errorCode + ", zegoExpressMediaplayer = " + mediaPlayer);
        }
    
        @Override
        public void onMediaPlayerNetworkEvent(ZegoMediaPlayer mediaPlayer, ZegoMediaPlayerNetworkEvent networkEvent) {
            // This event callback is triggered on the UI thread. You can make corresponding UI changes here. For example, to prompt the user about the current network condition.
            Log.d(TAG, "onMediaPlayerNetworkEvent: networkEvent = " + networkEvent.value() + ", zegoExpressMediaplayer = " + mediaPlayer);
        }
    
        @Override
        public void onMediaPlayerPlayingProgress(ZegoMediaPlayer mediaPlayer, long millisecond) {
            // This event callback is triggered on the UI thread. Developers can make corresponding UI changes here. For example, to refresh the playback progress bar.
            Log.d(TAG, "onMediaPlayerPlayingProgress: millisecond = " + millisecond + ", zegoExpressMediaplayer = " + mediaPlayer);
        }
    }
    
    mediaplayer.setEventHandler(new MyIZegoMediaPlayerEventHandler());

4.3 Load a media resource

Before you play a media resource, call the media player's loadResource method to load the resource first. According to the actual situation, you can pass the absolute path of a local resource or the URL (for example, http://your.domain.com/your-movie.mp4) of a network resource to the path parameter. To obtain the result of the resource loading, pass in a callback parameter.

If you are playing the media files loaded before, call the stop method to stop playing first, and then call the loadResource method to load the media resoucre; Otherwise, the loading operation will fail.

  • Function prototype:

    /**
     * Load a media resource
     *
     * @param path: The absolute path to a local resource or the URL of a network resource.
     * @param callback: The callback to report the result of loading the media resource.
     */
    public void loadResource(String path, IZegoMediaPlayerLoadResourceCallback callback);
  • Sample code:

    zegoMediaplayer.loadResource("sourcePath", new IZegoMediaPlayerLoadResourceCallback() {
        @Override
        public void onLoadResourceCallback(int errorcode) {
            // This event callback is called on the UI thread. Developers can make corresponding UI changes here.
            if(errorCode == 0){
                Log.d(TAG, "onLoadResourceCallback: success");
            } else {
                Log.d(TAG, "onLoadResourceCallback: errorcode = " + errorcode);
            }
        }
    });

4.4 Control the playback

4.4.1 Control the playback status

mediaplayer_state

Call the media player's methods start, pause, resume, and stop to control the playback.

When you execute these operations, the internal state of the media player changes accordingly, and the onMediaPlayerStateUpdate callback will be triggered.

You can also call the media player's getCurrentState method to get the current state of the media player at any time.

If you set enableRepeat to true, the media player will automatically repeat playing the loaded media resource.

  • Function prototype:

    /**
     * Whether to repeat playing the media resource
     *
     * @param enable: The repeat flag, defaults to false.
     */
    public void enableRepeat(boolean enable);
    
    /**
     * Start the playback (the media resource needs to be loaded first)
     *
     */
    public void start();
    
    /**
     * Pause the playback
     *
     */
    public void pause();
    
    /**
     * Resume the playback
     *
     */
    public void resume();
    
    /**
     * Stop the playback
     *
     */
    public void stop();
  • Sample code:

    mediaplayer.enableRepeat(true);
    mediaplayer.start();
    mediaplayer.pause();
    mediaplayer.resume();
    mediaplayer.stop();

4.4.2 Control the playback progress

The playback progress can be obtained through the callback onMediaPlayerPlayingProgress, which is triggered at an interval of 1,000 ms by default. You can change the callback interval by calling the media player's setProgressInterval.

You can also call the media player's getCurrentProgress method to get the current playback progress.

To adjust the playback progress, call the media player's seekTo method with the target time point.

  • Function prototype:

    /**
     * Set the playback progress callback interval
     *
     * The playback progress callback interval controls the frequency at which the callback [onMediaPlayerPlayingProgress] is called. The default callback interval is 1,000 ms. If it is set to 0, the callback is stopped. 
     * @param millisecond: The playback progress callback interval in milliseconds.
     */
    public void setProgressInterval(long millisecond);
    
    /**
     * Get the current playback progress
     *
     */
    public long getCurrentProgress();
  • Sample code:

    mediaplayer.setProgressInterval(2000);
    long progress = mediaplayer.getCurrentProgress();
    mediaplayer.seekTo(mediaplayer.getTotalDuration() / 2, new IZegoMediaPlayerSeekToCallback(){
        @Override
        public void onSeekToTimeCallback(int errorcode) {
            // This callback is triggered on the UI thread. You can make corresponding UI changes here.
            if(errorCode == 0){
                Log.d(TAG, "onSeekToTimeCallback: success");
            } else {
                Log.d(TAG, "onSeekToTimeCallback: errorcode = " + errorcode);
            }
        }
    });

4.4.3 Control the audio

To get the local playback volume, call the media player's getPlayVolume method.

To set the local playback volume, call the media player's setPlayVolume method.

To get the volume of the audio being streamed out, call the media player's getPublishVolume method.

To set the volume of the audio being streamed out, call the media player's setPublishVolume method.

To mix the audio of the media resource being played into the stream being published, call the media player's enableAux method.

To mute or unmute the local playback, call the media player's muteLocal method. This doesn't affect the audio being published out simultaneously.

int playVolume = mediaplayer.getPlayVolume();
mediaplayer.setPlayVolume(playVolume / 2);
mediaplayer.muteLocal(true);
mediaplayer.enableAux(true);

If you want to obtain the audio data of the media file being played, do the following:

  1. Call the media player's setAudioHandler method to set up an audio event callback handler for the media player.
  2. Listen for the onAudioFrame callback to obtain the audio frame data the media player sends out.
  • Function prototype:

    /**
     * Set up an audio event callback handler for the media player.
     *
     * By setting up this handler, you can obtain the audio frame data of the media file being played through the callback onAudioFrame.
     * @param handler: The audio event handler of the media player.
     */
    public void setAudioHandler(IZegoMediaPlayerAudioHandler handler)
  • Sample code:

    // Set up an audio event callback handler for the media player.
    mediaplayer.setAudioHandler(new IZegoMediaPlayerAudioHandler() {
        @Override
        public void onAudioFrame(ZegoMediaPlayer mediaPlayer, ByteBuffer data, int dataLength, ZegoAudioFrameParam param) {
            // Process the audio frame data the media player sends out. For example, you can save the audio data to a local file, perform sound effect processing, etc.
            Log.d(TAG, "onAudioFrame: buffer = " + data + ", zegoAudioFrame = " + dataLength);
        }
    });

4.4.4 Control the video

When playing a video resource, you can call the media player's setPlayerCanvas method to set the view for displaying the video.

  • Function prototype:

    /**
     * Set up the view for displaying the video
     *
     * @param canvas: The canvas object for rendering the video
     */
    public void setPlayerCanvas(ZegoCanvas canvas);
  • Sample code:

    // textureView is an instance of android.view.TextureView that is attached to the UI layout
    mediaplayer.setPlayerCanvas(new ZegoCanvas(textureView));

If you want to obtain the video data of the media file being played, do the following:

  1. Call the media player's setVideoHandler method to set up a video event callback handler for the media player.
  2. Listen for the onVideoFrame callback to obtain the video frame data the media player sends out.
  • Function prototype:

    /**
     * Set up a video event callback handler for the media player.
     *
     * By setting up this handler, you can obtain the video data of the media file being played through the callback onVideoFrame.
     * @param handler: The video event handler of the media player.
     * @param format: The video frame format of the video data.
     */
    public void setVideoHandler(IZegoMediaPlayerVideoHandler handler, ZegoVideoFrameFormat format);
  • Sample code:

    // Set up a video event callback handler for the media player
    mediaplayer.setVideoHandler(new IZegoMediaPlayerVideoHandler(){
        @Override
        public void onVideoFrame(ZegoMediaPlayer mediaPlayer, ByteBuffer[] data, int[] dataLength, ZegoVideoFrameParam param) {
            // Process the video frame data the media player sends out. For example, you can save the video data to a local file, perform video layer blending, etc.
            Log.d(TAG, "onVideoFrame");
        }
    }, ZegoVideoFrameFormat.Unknown); // Generally, the second parameter is set to the current platform's default video frame format.

4.4.5 Stream out the video being played

To stream out the video that the media player is currently playing, do the following:

  1. Call the media player's setVideoHandler method to set up an event handler for the media player.
  2. Listen for the onVideoFrame callback to obtain the video frame data the media player sends out.
  3. Follow the instructions in Custom video capture to capture and stream out the obtained video frame data.

When customizing data collection, it is recommended that developers define a flag bit:

  • When the onStart callback is triggered, set the flag bit to True, which means you can start sending the captured video frame data to the SDK.
  • When the onStop callback is triggered, set the flag bit to False, which means you need to stop sending the captured video frame data to the SDK.
  1. Check the flag bit in the callback method onVideoFrame: If it is True, which means the onStart callback has been triggered, call the engine's sendCustomVideoCaptureRawData method to send the obtained video frame data to the SDK.

  2. Call the engine's startPublishingStream method to start streaming. For more information, see Start publishing streams.

4.4.6 Use the voice changer

In online karaoke use cases, if you need to change the pitch of the accompaniment, you can call the media player’s setVoiceChangerParam method to do so. By changing the value of the ZegoVoiceChangerParam parameter, you can raise or lower the pitch as needed. The value range of this parameter is [-8.0, 8.0]. As you increase the value, the pitch gets higher. The default value is 0.0, which means the voice changer is turned off.

ZegoVoiceChangerParam param = new ZegoVoiceChangerParam()
param.pitch = 8.0f; /** Change an adult male voice into a child's voice. */
param.pitch = 4.0f; /** Change an adult male voice into an adult female voice. */
param.pitch = 6.0f; /** Change an adult female voice into a child's voice. */
param.pitch = -3.0f; /** Change an adult female voice into an adult male voice. */
mMediaplayer.setVoiceChangerParam(ZegoMediaPlayerAudioChannel.ALL, voiceChangerParam);

4.5 Destroy the media player instance

When a media player instance is no longer required, destroy it in time to release the occupied resources.

  • Function prototype:

    /**
     * Destroy a media player instance
     *
     */
    public void destroyMediaPlayer();
  • Sample code:

    mediaplayer.destroyMediaPlayer();
    // When calling this method, you must manually release the object references used at the business layer to avoid memory leak.
    mediaplayer = null;

5 API reference

Method Description
createMediaPlayer Creates a media player instance.
destroyMediaPlayer Destroys a media player instance.
setVoiceChangerParam Sets the voice changer parameters.

6 FAQ

  1. How to switch media resources during playback?

    First, call the media player's stop method, and then call the media player's loadResource method to load the new resource.