Live Streaming 01

My mate Matt and I decided that we want to stream our paired programming sessions to youtube. We've only completed one stream so far, and like all things keeping consistency is key so i'm hopeful for the future.

Since we dont get to see the stream live while we are doing it because of audio feedback etc, we have to take some things on faith, like audio quality etc, and hope for the best.. this post mortem is my attempt at reviewing the footage after the fact and making notes on what we did and how we can make it better next time.

I might even make an edit of the footage cutting out the things I think are shit or boring. try to reduce its length.

The Stream itself.
https://youtu.be/UB91O40-IOU

The Streaming Setup

Because of the way we wanted to make things work, it took us a couple of hours to get started. and once we were started we quickly found limitations in what we were doing, and worked on solving them during the first streaming session.

The idea was that I would stream any video and audio information from my computer directly to Matt's and he would then stream it to youtube. that way I could reboot, stop and start the streams on my side without affecting the final output to youtube.

Three and a half hours or so of wrangling with FFmpeg we managed to get it working in a rudimentary way. and throughout the video we worked on dropping the latency and upping the quality.

Streaming setup at the beginning of the video:

ffmpeg -loglevel error \
-i /dev/video0 \
-f x11grab -video_size 1920x1080 -framerate 30 -i :0.0 \
-c:v libx264 -pix_fmt yuv420p -crf 23 -r 30 -g 60 -b:v 2500k \
-profile:v high -level 4.1 -preset ultrafast -tune zerolatency \
-map 0:0 -map 1:0 \ -f mpegts udp://$MATTS_IP:1234
This takes two input video streams and pushes them both over udp to matt's pc. Who opens them up in VLC for viewing along with his own webcam stream. Matt used OBS Studio to stream from his second monitor to youtube.

We chose this way so that there was a buffer between doing any 'work' on my pc and streaming, the stream would continue even if I rebooted my pc, we could dynamically create new streams and integrate them into the webcast without any starting or stopping of the streaming pc.

It worked well enough for beginnings. but there were some major downsides
  • Latency from my pc to Matt's
  • No audio from my pc

Problems encountered:

  • We couldn't find a way to get vlc to allow you to specify the port number when you use it to connect to udp streams, This makes using multiple network streams a no go since it always wants to bind to the default port of 1234
  • ffplay wont display multiple video channels from the same stream at once
  • vlc's playback latency is pretty bad and is not intuitive on how to reduce it
  • Couldn't figure out how to capture pulseaudio to stream

Successes:

  • ffplay does let you reduce the playback latency significantly with the -fflags nobuffer option
  • ffplay does allow you to specify the port number so multiple network streams works
So we ended up using the original idea, stream each source from my computer into a separate network stream and load them up individually onto Matt's pc.

Final streaming setup after the video:

Samuel's PC

Three separate scripts:

desktop
ffmpeg -loglevel warning \-f x11grab -video_size 1920x1080 -framerate 15 -i :0.0 \-c:v libx264 -pix_fmt yub420p -crf 23 -r 15 -g 60 -b:v 500k \-profile:v high -level 4.1 -preset ultrafast -tune zerolatency \-f mpegts udp://$MATTS_IP:1234
webcam
ffmpeg -loglevel warning \-i /dev/video0 -video_size 424x240 -framerate 30 \-c:v libx264 -pix_fmt yub420p -crf 23 -r 30 -g 60 -b:v 500k \-profile:v high -level 4.1 -preset ultrafast -tune zerolatency \-f mpegts udp://$MATTS_IP:1235
audio
ffmpeg -loglevel warning \-f jack -i udp_audio \-f mpegts udp://$MATTS_IP:1236

Matt's PC

ffplay -fflags nobuffer udp://$SAMS_IP:1234 #desktop 
ffplay -fflags nobuffer udp://$SAMS_IP:1235 #webcam 
ffplay -fflags nobuffer udp://$SAMS_IP:1236 #audio
And obs studio to stream his second desktop to youtube.

Still Problems

Video footage to youtube was still choppy and crap which can be seen by the speed of the games later in the stream. I believe it's probably due to network bandwidth considerations since we are pushing a lot of streams back and forth over the wireless network.

Future Ideas

Having a third pc act as the streaming manager so that either matt or I can dump out of the situation and it will continue to plod along streaming the room.

The Content

While doing the stream and while watching it back, you always see and think of a load of things you liked, and things you would like to change.

Observations:

  • Background of webcam's is boring
  • Lighting in webcams is bad
  • Audio levelling needs work maybe a display
  • Multiple microphones
  • Silences are really deadening
  • Need GoPro or separate external camera(Partially Solved)
  • Need to organise topical material before stream starts.
  • Need to have pre-prepared conversation topics to raise during silences
  • The dynamic nature of the setup is really flexible while we figure things out.
  • Not currently working streams need to display something to indicate failure
  • royalty free music to play in the background
  • should use git to keep versions of streaming files
  • Need to solve jack xruns
  • Need to solve Past duration warnings (Solved)
  • having our webcams next to each other was fun
  • Should automate the scripts further
  • Need to monitor network bandwidth

References

FFmpeg

VLC
http://www.videolan.org/vlc/index.html

OBS Studio
https://obsproject.com/

A very helpful blog post on low latency streaming:
http://fomori.org/blog/?p=1213

JACK Audio Connection Kit
http://jackaudio.org/





For the "Past duration 0.n too large" messages:

A viable solution for quick and dirty movable camera:
https://play.google.com/store/apps/details?id=com.pas.webcam&hl=en