Article inspired by video https://youtu.be/xmE99sHBgy0

I will not go into details what gstreamer is and how gstreamer works, because I assume you already know it. I assume you also you have a basic knowledge and experience with video streaming and RPi devices.

It’s very hard to measure and achieve low latency or even zero latency, so we need to focus on simplification and removing bottle necks in our gstreamer pipeline to get best performance.

Hardware used in experiment

Software used

Preparation

Now you can try to stream video from your camera via gstreamer and it’s time to focus on latency. If you have already tried experiment with raspivid, netcat and mplayer, then you see how effective such combination can be. Mplayer has lowest latency itself, but mplayer is not a media framework you may want to use and integrate with your other systems. However, here is an experiment you may try to see results:

Where 192.168.0.12 is your Laptop IP.

In my experiment with gstreamer I focused on achieving the same performance as I observed with mplayer. I started reducing bottle necks from network, then by pipeline simplification. Here, Netcat due to it’s simplicity has better performance than gstreamer’s tcpserversink and tcpclientsink. I was surprised, but UDP sinks/clients were even worst. I modified pipeline by adding small queue with size=1 to keep only one frame in the buffer and drop all the rest. Subsequent pipeline elements, like videorate and autovideosync is all we need.

In both examples below two IPs were used: 192.168.0.13 is an Android device, 192.168.0.14 is RPi.

Low latency solution – TCP version

Scenario #1 – Android device is a host

On Android device, open RaspberryPi Camera Viewer and run following pipeline:

On Raspberry Pi:

click play button on the center of preview window in RaspberryPi Camera Viewer

Scenario #2 – Raspberry Pi is a host

On Raspberry Pi:

On Android device, open RaspberryPi Camera Viewer and run following pipeline:

click play button on the center of preview window in RaspberryPi Camera Viewer

Tweaks:

To get even better performance, you can also play with videorate, like:

In theory, setting framerate bigger than input streams’ rate could help, but personally I did not see any difference. However, it’s worth to try and see results.

Network bandwidth is satisfied as well (nmon output):

Low latency solution – UDP version

On Android device, open RaspberryPi Camera Viewer and run following pipeline:

On Raspberry Pi (where 192.168.0.13 is IP of your Android device):

click play button on the center of preview window in RaspberryPi Camera Viewer

Summary

I hope this article help you with effective video streaming with minimal latency. I’m happy to discuss, so feel free to comment or contact me.

What's your reaction?
2Smile0Lol0Wow0Love0Sad0Angry

Leave a comment