Release v3: Simulcast!

By enabling simulcast on a Chrome publisher (API details here)) your video quality will no longer be dragged down by the worst connection in the room.

Enjoy your automatic spatial and temporal scalability. Also, we are ready for SVC once its available :slight_smile:

As a bonus, we have tons of bugfixes and little improvements for you.

Full release notes here

2 Likes

Really great work!

Regarding the simulcast feature I have a question:
It is only a chrome feature because firefox is not ready yet or there is a Licode missing part to handle also a firefox simulcast?

thanks!

Firefox “kind of” has simulcast but in our tests the behavior has proven to be inconsistent. We’ve decided to delay our Firefox simulcast support for now. The pieces are there in Licode so it shouldn’t be a lot of work once we are happy with how Firefox behaves in simulcast.

Hi

Regarding Simulcast

  1. based on which policy the receiver gets forwarded less or more spatial layers ?

  2. can Licode make the simulcast sender adjust the BW of the lower layer to handle slow participants when they join ?

  3. if a conference with only chrome users is running and a Firefox joins what will happen ? Simulcast is stopped ? Simulcast keeps going but Firefox will get what ?

Thanks

answering first to your second answer:

the publisher always send a very low layer, so once one join, it starts from this one.
so, if you have two layer set on a 640x480 stream you will have:

  • 640x480 ~30fps layer
  • 640x480 ~15fps layer
  • 640x480 ~7 fps layer
  • 320x240 ~30fps layer
  • 320x240 ~15fps layer
  • 320x240 ~7fps layer

when one user join, licode automatically forward the lowest layer.

connecting with your first answer:
once receiving the first layer, each subscriber send periodic rtcp packets to the sender, informing him about his available bandwidth. these packets suggest licode on which layer to switch.

last:
if a firefox join, it can decode other patricipants simulcasts streams, but he cannot publish simulcast right now. he will publish a full resolution stream and the other will subscribe to this as simulcast don’t exist.

Ok that’s an answer.

Still not clear if there is a retrofeeback mechanism to tell the sender(s) to reconfigure the layers like for example change number of layers or only bitrate of the smallest to better cope with certain scenarios .

Moreover I wonder if and how (asked that in the past) one can do a continuous presence layout and have the Licode client library automatically subscribe to more or less (spatial domain) based in the size of the tile (video element) the stream is displayed at.

Also the fact Licode doea not support lastN is also very limiting in certain scenarios .

Happy to explain more about certain use cases and how these are handled by others.

Licode runs the congestion control algorithm to estimate the available bandwidth for each subscriber. The bitrate needed for each layer is calculated in real time. Currently Licode selects the highest possible spatial layer (and the highest temporal layer corresponding to that spatial) that can be sent within the limits of the estimated bandwidth.

There isn’t any, publishers will send the highest quality possible for them, limited by maxVideoBW, of course. You can adjust that parameter dynamically in your application, though. We haven’t played around enough with this in simulcast so there may be some kinks but we’d be happy to fix them if you detect it.

We’re planning on adding options to manually customize this behavior, for instance, you could limit the resolution or the frame rate for a particular subscriber. No ETA for this yet, but it shouldn’t be too far in the future.

Not at this point. We have limited flexibility when it comes to adjusting the bitrate of each layer, most of the options are hardcoded in Chrome (not saying we couldn’t do a better job, though). We do have a fallback mode, licode will detect a subscriber is not able to receive the lower layer and will send a very low-bitrate stream by sporadically sending keyframes (and padding) to keep the communication going.

As @Francesco_Durighetto said, Chrome users will publish simulcast while Firefox will publish non-simulcast streams. Firefox users, as subscribers, will benefit from the advantages of the simulcasted Chrome streams, since after going through they SFU, they are effectively normal RTP/VP8 streams. Firefox–published streams will be vanilla pre-simulcast streams and will adapt to the lowest subscriber.

Feel free to elaborate the cases where you feel Licode is lagging behind, we’re certainly not covering every use case and we always open to new ideas.

Is simulcast is working with the latest updates?

Yep.
It’s in both v3 and in the master branch.

Hello,
I tried simulcast with three layers on chrome 77, but am not able to get all three layers. Highest layer I get is 360P though max b/w is set as 4MB in both client and server configs. Also resolution is set as 720P.
chrome statistics (webrtc internals) says camera is generating 720p frames but what is encoded and sent is 360P.
Any idea why am not getting HD even though my server and client in I Gbps high speed lan.

Hi @boby, welcome and thanks for the report!
I’ve been investigating this and I can reproduce it only when there are no subscribers present. Is that the case in your tests?
Actually it would be better if you submitted this an an issue in the repo and we continue the discussion there github repo Please use the template because that will provide useful info for us

Is it possible to configure subscribers to subscribe to the highest layer at the beginning?
For now, they subscribes to the lowest and it takes about 1 min to rich good quality…

There’s a tradeoff there: how quick you can get to higher quality while providing a good experience for limited connections. If we started sending, let’s say, 1Mbps, we would flood the connection of people with insufficient bandwidth, causing a very bad experience, even if we could react before the websocket fails.
We are not being overly conservative but we obviously prefer people to wait a bit for better quality if that means others with poor connection can actually participate. That said, it is not perfect and we are tweaking it all the time.