How video players work – Step-by-Step Breakdown

Ever wondered what happens when you hit “Play” on a video? Here’s a step-by-step breakdown of how video players work behind the scenes — especially useful for developers using Flutter and packages like flick_video_player.

📌 1. Video Source Input

• The video file or stream is loaded.

• Sources can be:

  • Local file (.mp4, .mkv, etc.)

  • Network stream (HLS, DASH, etc.)

• Metadata is parsed: duration, resolution, codec info.

📌 2. Demuxing (Stream Separation)

• The container format (e.g., .mp4, .webm) holds multiple streams.

• Demuxer splits the streams into:

  • Video stream

  • Audio stream

  • Optional subtitles or metadata

📌 3. Decoding (Decompression)

• Streams are encoded (compressed) to save space.

• Decoder turns them into raw data:

  • Hardware decoding: Uses device chip, more efficient.

  • Software decoding: Uses CPU, more compatible.

📌 4. Synchronisation

• Each audio/video packet includes a timestamp.

• Ensures:

  • Lips match sound

  • Video frames show at correct intervals

• Sync is maintained even after seeking.

📌 5. Rendering

• Video frames sent to GPU/display (SurfaceView, TextureView, etc.)

• Audio samples sent to audio output (headphones, speaker)

• Handles:

  • Aspect ratio

  • Resolution scaling

  • Frame dropping (if needed)

📌 6. Playback Controls

• Basic controls: play, pause, seek, mute, volume

• Advanced options:

  • Playback speed change

  • Subtitles

  • Picture-in-picture

• Connected to player state machine

📌 7. Error Handling & Decoding Fallback

• Common failure: Unsupported codec/profile

• Examples:

  • AVC High Profile not supported by low-end device

  • Hardware decoder (like OMX.MTK.VIDEO.DECODER.AVC) fails to start

• Fallback:

  • Use software decoding (e.g., via FFmpeg)

  • Not built-in to flick_video_player, but possible with native integration

📌 Flutter Context

• flick_video_player is built on top of video_player

• Uses:

  • ExoPlayer on Android

  • AVPlayer on iOS

• You can extend it to support custom rendering or alternate decoding paths (e.g., via external plugins or native code)

✅ Summary

Each video playback session involves:

1. Loading → 2. Demuxing → 3. Decoding → 4. Sync → 5. Rendering → 6. Controls

Understanding this pipeline is key for:

• Fixing device-specific issues

• Supporting more formats

• Building high-quality video apps

Playback issue on some android devices & How fvp fixed it?

The Problem

• We faced video playback failures on several Android 10 devices using MediaTek processors.

• These devices failed when playing videos using the Flick Video Player, which internally depends on Flutter's default video_player plugin.

• Errors seen:

  • MediaCodecVideoRenderer: Decoder init failed: OMX.MTK.VIDEO.DECODER.AVC

  • IllegalArgumentException: start failed

Affected Devices

Below are some devices and chipsets where Flick Video Player failed to play videos:

• Redmi 9 — MediaTek MT6765G, Android 10 ❌

• Redmi 9 — MediaTek MT6769V/CU, Android 10 ❌

• Oppo Reno3 Pro — MediaTek MT6779V, Android 10 ❌

These devices have limited or unstable support for hardware video decoding, especially for H.264/AVC profiles like Main or High.

🔍 Why This Happened

• Flutter’s default video player uses only hardware decoding via MediaCodec.

• On these MediaTek devices, hardware decoding fails due to:

  • Unsupported codec profiles

  • Incompatible bitstream parameters

  • Driver bugs or partial codec support

• There’s no software fallback — so video playback fails completely.

✅ How We Fixed It with fvp

• We added the fvp package — a platform-independent video backend built on libmdk.

• We did not change the player UI or switch from Flick.

We only added the following line during app startup:
fvp.registerWith();

• This overrides the default video_player backend, adding smart fallback logic:

  • Tries hardware decoding first

  • If it fails, automatically switches to software decoding using FFmpeg
    
    

💡 Why Just fvp.registerWith() Was Enough

• This single line tells Flutter to use fvp instead of the default player engine.

• Since Flick Video Player depends on video_player, it also benefits from the backend replacement.

• fvp supports:

  • Software decoding fallback

  • Better codec compatibility

  • Graceful error handling

• Playback now works on previously failing devices without changing any widgets or video formats.

🧪 What We Observed

• All affected MediaTek devices started playing videos correctly.

• No more codec errors or crashes.

• Works even with high-profile AVC videos.

• UI remained unchanged, only backend was swapped.