How to Integrate Active Speaker in Android(Kotlin) Video Chat App?

Introduction

Imagine an active video conference with multiple participants. In group calls, it is important to identify the active speaker. By integrating Active Speaker Indication feature into your Android (Kotlin) video chat application, you can enhance the user experience by providing visual cues that highlight who is currently speaking.

This guide will cover everything through the step-by-step process of implementing active speaker signaling in your application using the VideoSDK. Available through VideoSDK, it transforms your Android video chat application by visually highlighting the currently speaking participant, ensuring a smooth and informative experience for your users during video calls.

Goals

By the End of this Article:

1. Create a VideoSDK account and generate your VideoSDK auth token.
2. Integrate the VideoSDK library and dependencies into your project.
3. Implement core functionalities for video calls using VideoSDK
4. Enable active speaker indication

Getting Started with VideoSDK

To take advantage of the active speaker highlighting functionality, we will need to use the capabilities that the VideoSDK offers. Before we dive into the implementation steps, let's make sure you complete the necessary prerequisites.

Create a VideoSDK Account

Go to your VideoSDK dashboard and sign up if you don't have an account. This account gives you access to the required Video SDK token, which acts as an authentication key that allows your application to interact with VideoSDK functionality.

Generate your Auth Token

Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token plays a crucial role in authorizing your application to use VideoSDK features.

For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.

Prerequisites and Setup

Make sure your development environment meets the following requirements:

• Java Development Kit is supported.
• Android Studio version 3.0 or later.
• Android SDK API level 21 or higher.
• A mobile device with Android 5.0 or later version.

Integrate VideoSDK

Following the account creation and token generation steps, we'll guide you through the process of adding the VideoSDK library and other dependencies to your project. We'll also ensure your app has the required permissions to access features like audio recording, camera usage, and internet connectivity, all crucial for a seamless video experience.

Step (a): Add the repositories to the project's settings.gradle file.

dependencyResolutionManagement {
  repositories {
    // ...
    google()
    mavenCentral()
    maven { url '<https://jitpack.io>' }
    maven { url "<https://maven.aliyun.com/repository/jcenter>" }
  }
}

Step (b): Include the following dependency within your application's build.gradle file:

dependencies {
  implementation 'live.videosdk:rtc-android-sdk:0.1.26'

  // library to perform Network call to generate a meeting id
  implementation 'com.amitshekhar.android:android-networking:1.0.2'

  // Other dependencies specific to your app
}

If your project has set android.useAndroidX=true, then set android.enableJetifier=true in the gradle.properties file to migrate your project to AndroidX and avoid duplicate class conflict.

Step (c): Add permissions to your project

In /app/Manifests/AndroidManifest.xml, add the following permissions after </application>.

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CAMERA" />

These permissions are essential for enabling core functionalities like audio recording, internet connectivity for real-time communication, and camera access for video streams within your video application.

Essential Steps for Building the Video Calling Functionality

We'll now set up the functionalities that make your video application after setting up your project with VideoSDK. This section outlines the essential steps for implementing core functionalities within your app. This section will guide you through four key aspects.

Step 1: Generate a meetingId

Now, we can create the meetingId from the VideoSDK's rooms API. You can refer to this documentation to generate meetingId.

Step 2: Initializing the Meeting

After getting meetingId , the next step involves initializing the meeting for that we need to,

1. Initialize VideoSDK.
2. Configure VideoSDK with a token.
3. Initialize the meeting with required params such as meetingId, participantName, micEnabled, webcamEnabled and more.
4. Add MeetingEventListener for listening events such as Meeting Join/Left and Participant Join/Left.
5. Join the room with meeting.join() a method.

Please copy the .xml file of the MeetingActivity from here.

class MeetingActivity : AppCompatActivity() {
  // declare the variables we will be using to handle the meeting
  private var meeting: Meeting? = null
  private var micEnabled = true
  private var webcamEnabled = true

  override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_meeting)

    val token = "" // Replace with the token you generated from the VideoSDK Dashboard
    val meetingId = "" // Replace with the meetingId you have generated
    val participantName = "John Doe"
    
    // 1. Initialize VideoSDK
    VideoSDK.initialize(applicationContext)

    // 2. Configuration VideoSDK with Token
    VideoSDK.config(token)

    // 3. Initialize VideoSDK Meeting
    meeting = VideoSDK.initMeeting(
      this@MeetingActivity, meetingId, participantName,
      micEnabled, webcamEnabled,null, null, false, null, null)

    // 4. Add event listener for listening upcoming events
    meeting!!.addEventListener(meetingEventListener)

    // 5. Join VideoSDK Meeting
    meeting!!.join()

    (findViewById<View>(R.id.tvMeetingId) as TextView).text = meetingId
  }

  // creating the MeetingEventListener
  private val meetingEventListener: MeetingEventListener = object : MeetingEventListener() {
    override fun onMeetingJoined() {
      Log.d("#meeting", "onMeetingJoined()")
    }

    override fun onMeetingLeft() {
      Log.d("#meeting", "onMeetingLeft()")
      meeting = null
      if (!isDestroyed) finish()
    }

    override fun onParticipantJoined(participant: Participant) {
      Toast.makeText(
        this@MeetingActivity, participant.displayName + " joined",
        Toast.LENGTH_SHORT
      ).show()
    }

    override fun onParticipantLeft(participant: Participant) {
      Toast.makeText(
         this@MeetingActivity, participant.displayName + " left",
         Toast.LENGTH_SHORT
      ).show()
    }
  }
}

Step 3: Handle Local Participant Media

After successfully entering the meeting, it's time to manage the webcam and microphone for the local participant (you).

To enable or disable the webcam, we'll use the Meeting class methods enableWebcam() and disableWebcam(), respectively. Similarly, to mute or unmute the microphone, we'll utilize the methods muteMic() and unmuteMic()

class MeetingActivity : AppCompatActivity() {
  override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_meeting)
    //...Meeting Setup is Here

    // actions
    setActionListeners()
  }

  private fun setActionListeners() {
    // toggle mic
    findViewById<View>(R.id.btnMic).setOnClickListener { view: View? ->
      if (micEnabled) {
        // this will mute the local participant's mic
        meeting!!.muteMic()
        Toast.makeText(this@MeetingActivity, "Mic Muted", Toast.LENGTH_SHORT).show()
      } else {
        // this will unmute the local participant's mic
        meeting!!.unmuteMic()
        Toast.makeText(this@MeetingActivity, "Mic Enabled", Toast.LENGTH_SHORT).show()
      }
        micEnabled=!micEnabled
    }

    // toggle webcam
    findViewById<View>(R.id.btnWebcam).setOnClickListener { view: View? ->
      if (webcamEnabled) {
        // this will disable the local participant webcam
        meeting!!.disableWebcam()
        Toast.makeText(this@MeetingActivity, "Webcam Disabled", Toast.LENGTH_SHORT).show()
      } else {
        // this will enable the local participant webcam
        meeting!!.enableWebcam()
        Toast.makeText(this@MeetingActivity, "Webcam Enabled", Toast.LENGTH_SHORT).show()
      }
       webcamEnabled=!webcamEnabled
    }

    // leave meeting
    findViewById<View>(R.id.btnLeave).setOnClickListener { view: View? ->
      // this will make the local participant leave the meeting
      meeting!!.leave()
    }
  }
}

Step 4: Handling the Participants' View

To display a list of participants in your video UI, we'll utilize a RecyclerView.

(a) This involves creating a new layout for the participant view named item_remote_peer.xml in the res/layout folder. You can copy item_remote_peer.xml file from here.

(b) Create a RecyclerView adapter ParticipantAdapter which will be responsible for displaying the participant list. Within this adapter, define a PeerViewHolder class that extends RecyclerView.ViewHolder.

class ParticipantAdapter(meeting: Meeting) : RecyclerView.Adapter<ParticipantAdapter.PeerViewHolder>() {

  override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): PeerViewHolder {
    return PeerViewHolder(
      LayoutInflater.from(parent.context)
        .inflate(R.layout.item_remote_peer, parent, false)
    )
  }

  override fun onBindViewHolder(holder: PeerViewHolder, position: Int) {
  }

  override fun getItemCount(): Int {
    return 0
  }

  class PeerViewHolder(view: View) : RecyclerView.ViewHolder(view) {
    // 'VideoView' to show Video Stream
    var participantView: VideoView
    var tvName: TextView

    init {
        tvName = view.findViewById(R.id.tvName)
        participantView = view.findViewById(R.id.participantView)
    }
  }
}

(c) Now, we will render a list of Participant for the meeting. We will initialize this list in the constructor of the ParticipantAdapter

class ParticipantAdapter(meeting: Meeting) :
    RecyclerView.Adapter<ParticipantAdapter.PeerViewHolder>() {

  // creating a empty list which will store all participants
  private val participants: MutableList<Participant> = ArrayList()

  init {
    // adding the local participant(You) to the list
    participants.add(meeting.localParticipant)

    // adding Meeting Event listener to get the participant join/leave event in the meeting.
    meeting.addEventListener(object : MeetingEventListener() {
      override fun onParticipantJoined(participant: Participant) {
        // add participant to the list
        participants.add(participant)
        notifyItemInserted(participants.size - 1)
      }

      override fun onParticipantLeft(participant: Participant) {
        var pos = -1
        for (i in participants.indices) {
          if (participants[i].id == participant.id) {
            pos = i
            break
          }
        }
        // remove participant from the list
        participants.remove(participant)
        if (pos >= 0) {
          notifyItemRemoved(pos)
        }
      }
    })
  }

  // replace getItemCount() method with following.
  // this method returns the size of total number of participants
  override fun getItemCount(): Int {
    return participants.size
  }
  //...
}

(d) We have listed our participants. Let's set up the view holder to display a participant video.

class ParticipantAdapter(meeting: Meeting) :
    RecyclerView.Adapter<ParticipantAdapter.PeerViewHolder>() {

  // replace onBindViewHolder() method with following.
  override fun onBindViewHolder(holder: PeerViewHolder, position: Int) {
    val participant = participants[position]

    holder.tvName.text = participant.displayName

    // adding the initial video stream for the participant into the 'VideoView'
    for ((_, stream) in participant.streams) {
      if (stream.kind.equals("video", ignoreCase = true)) {
        holder.participantView.visibility = View.VISIBLE
        val videoTrack = stream.track as VideoTrack
        holder.participantView.addTrack(videoTrack)
        break
      }
    }

    // add Listener to the participant which will update start or stop the video stream of that participant
    participant.addEventListener(object : ParticipantEventListener() {
      override fun onStreamEnabled(stream: Stream) {
        if (stream.kind.equals("video", ignoreCase = true)) {
          holder.participantView.visibility = View.VISIBLE
          val videoTrack = stream.track as VideoTrack
          holder.participantView.addTrack(videoTrack)
       }
      }

      override fun onStreamDisabled(stream: Stream) {
        if (stream.kind.equals("video", ignoreCase = true)) {
          holder.participantView.removeTrack()
          holder.participantView.visibility = View.GONE
        }
      }
    })
  }
}

(e) Now, add this adapter to the MeetingActivity

override fun onCreate(savedInstanceState: Bundle?) {
  // Meeting Setup...
  //...
  val rvParticipants = findViewById<RecyclerView>(R.id.rvParticipants)
  rvParticipants.layoutManager = GridLayoutManager(this, 2)
  rvParticipants.adapter = ParticipantAdapter(meeting!!)
}

Integrate Active Speaker Indication

Once you've successfully installed VideoSDK in your Android app, it's time to integrate the Active Speaker Indication feature. This functionality utilizes real-time audio analysis to point to the participant who's speaking at any given moment. VideoSDK then provides this information to your app, allowing you to integrate visual cues into your user interface. Once you've successfully installed VideoSDK in your Android app, it's time to integrate the Active Speaker Indication feature. This functionality utilizes real-time audio analysis to point to the participant who's speaking at any given moment. Text-to-speech can further enhance this by providing audible cues for participants. VideoSDK then provides this information to your app, allowing you to integrate visual cues into your user interface.

The active speaker highlight feature aims to visually indicate the participant currently speaking in the video call. This functionality is especially valuable in scenarios with a large number of participants, where it can be difficult to identify the source of the incoming audio.

This is how we can integrate this feature with VideoSDK. Every time a participant actively speaks during the meeting, the onSpeakerChanged event is triggered. This event transmits the participant ID of the person who is currently speaking.

By capturing this event and leveraging the participant ID, you can visually highlight the corresponding participant in your application's user interface. This can be achieved by modifying the user interface element that represents the speaking participant, such as changing the background color or adding a visual indicator.

private final MeetingEventListener meetingEventListener = new MeetingEventListener() {
    @Override
    public void onSpeakerChanged(String participantId) {
        Toast.makeText(MainActivity.this, "Active Speaker participantId" + participantId, Toast.LENGTH_SHORT).show();
        super.onSpeakerChanged(participantId);
    }
};

To implement the dynamic speaker detection UI featured in the above image, feel free to check out our GitHub repository.

GitHub - videosdk-live/videosdk-rtc-android-kotlin-sdk-example: WebRTC based video conferencing SDK for Android (Kotlin)

WebRTC based video conferencing SDK for Android (Kotlin) - videosdk-live/videosdk-rtc-android-kotlin-sdk-example

GitHubvideosdk-live

Conclusion

This guide has equipped you with the knowledge to integrate active speaker signals into your Android video chat application using the VideoSDK. With this feature, you enhance the user experience by providing clear visual cues that enhance communication and participation in your video calls.

To unlock the full potential of VideoSDK, we encourage you to sign up for VideoSDK and further explore its features. Start implementing VideoSDK and Active Speaker Indication today to improve your video app's functionality and user engagement.

Sign up with VideoSDK today and Get 10,000 Free Minutes to take your video app to the next level!