New JSON based way to send sdps breaks lots of integrations

Features
#1

Hi guys @Javier @pedro,
I’d like to make a “complaint” about the new way to send the sdps in a JSONized way.

Why this change? sdps are in a way “complicated” in plaintext but at the same time, for those who works with these monsters for a long time, they are also clear and meaningful.
wrapping them with an hand written library could lend to undesired behaviors, and it will be way more complicated to debug issues since sdps generated by the browser and these received by the agent aren’t the same.

Is there a specific reason for this introduction?

There is also a major problem with this changes that probably breaks lots of integrations and will force us to stick to the v3: interoperability.
I know that lots of people using licode in a production environment, are also using a probably hand made native app for android and ios or a open source project such as Zevarito’s excellent work .
There’s no such library that translates sdps in JSON and there’s no way (correct me if i’m wrong) we can spend such lot of time to write one.

I think this highly-customised way to send sdps is a “regression” in my point of view and there has to be a way to exchange sdps in plaintext as before.

I hope you’ll get my point.

Please share your thoughts guys

#2

I understand your concerns and I see how this may seem like an arbitrary decision, let me explain some of the reasoning behind this change.

A big reason to do this is SDP is not consistent across all browsers. You have seen, for instance, the differences in simulcast. There are big and small details that can change with browsers and even with browser versions. Until now, we’ve treated those differences both on the client and in the server side (in SdpInfo). As a result, SdpInfo is full of little workarounds and fixes. In fact, sometimes we’re parsing-unparsing the SDP message more than once for every negotiation. We see plenty of benefits in concentrating the full SDP generation in the client side and exchange the information we need with the server in a different format that we fully control.

We will make extensive use of this with the Single PeerConnection implementation. We won’t have to generate in the server side a full browser-dependant SDP for every stream that we want to add to a connection.

I’ve praised Zevarito’s iOS work plenty of times. However, I don’t think freezing all changes in ErizoClient side and in the negotiation because of it is the way to go. I hope we (meaning both us and the community) can find a way to solve this.

#3

I see your point but only partially. It has totally sense to include this library server side, to benefit in the future, of the advantages of managing the stream in an object oriented way (maybe as you said by using it to replace SDPInfo.cpp).
I partially see the benefit to include this library also in the client side to better manage the munging of sdps (see setMaxBW, it’s way more cleaner and easier written in this object way), but I totally can’t see an advantage to TRANSMIT the sdp in JSON format.
I expect that every capable webrtc device will be able to talk to licode in a “native” way.
I also expect to have to manage server side some sdps change in order to make the new device fully compatible, but is something we will do in the SDPInfo.js in the same way we do in the SDPInfo.cpp, just in a more flexible and easier way.

This is a simple concept of what I think:
PC SCENARIO:
GenerateOffer -> SemanticSdp -> modifySdp in object oriented style -> tostring() — SEND TO ERIZO IN PLAIN TEXT —> processString() -> modify/parse Sdps in object oriented way in ErizoJS

ANY OTHER DEVICE SCENARIO:
GenerateOffer (native webrtc library) -> munge sdp if needed (like old regex style) — SEND TO ERIZO —> processString() -> modify/parse Sdps in object oriented way in ErizoJS

#4

I get your point and I get how the current implementation can raise more questions than answers. It is only the first iteration of a series of improvements in the negotiation.

The idea is that we won’t be exchanging full SDP messages and only use the parts that we need for each step of the negotiation. Only browser-agnostic information in JSON format will be sent from and to the server side. That way we can concentrate the SDP browser-specific work on the client. Again, we now have tweaks for different browsers in both the Client and the server side, making it much harder to maintain. Furthermore, by exchanging only the parts of the SDP that we need at any given time we can speed up and simplify the negotiation.

This will probably involve more changes in Zevarito’s iOS client and any other Licode client implementation that doesn’t use the Licode JS stack but we consider this both a way to improve the negotiation and prepare it for more browsers and more features (single peerconnection, simulcast, ICE restarts, etc.)

The SDP and negotiation changes we’re implementing do not make licode inherently “less interoperable”. Any device that wants to connect to Licode needs to implement the Licode negotiation to join the room, subscribe to the streams, send ICE candidates, mangle SDP the way Erizo likes it, etc. It does, however, break compatibility with current implementations of solutions that do not use our ErizoClient.

We’ve given this some thoughts and there are ways we could make it easier to get the new SDP managing library working in clients such as Zevarito’s. We can package SdpInfo.js and it’s dependencies separately for that purpose. That code could be transpiled or even run in a JS vm natively in iOS as a starting point to keep compatibility.

We are aware that this does involve work and are willing to help with it but we really need to shake up the negotiation if we want to be able to safely introduce new features and support for things to come.

#5

Yes but can’t we just make this library work as a “middleware” between plain sdps and c++ sdp processing?
I don’t get the point to include this library both client and server side.

I see this as something the future c++ rework of sdpinfo will need to be more clear and efficient. but transforming in object-form the sdp client side and sending them as JSON and then retransforming them in text or directly transforming them in object once they arrive to erizoJSController isn’t the same thing?

this way we can avoid the headache to make the thing compatible to every device/language.

I know it’s necessary to lightweight the whole sdp parsing process to be able to introduce new features, but is something that has to do with the server side, not client.
Sdps are the language of webrtc and translating it in “something else” from the source is in my opinion a non-necessary step.

I speak for myself here, but we have our custom Licode connected to native custom-made android app, iOS app and also IE and this change cannot be introduced because is not backward compatible, so there will be rolling update problems and incompatibilities, without considering the effort to translate this library in multiple languages.

I frankly don’t know how to transpile the js into swift and java code but if it’s the only option you see I hope you’ll deliver an easy to use tool to make so.

#6

I agree with Francesco.

If transferring SDP in JSON format is the way we choose, we need the support on the mobile side. My team is now developing custom mobile apps based on pre-v5.1. IMO it’s good to be mobile-friendly to make Licode a better WebRTC communication platform.

#7

We don’t own or maintain any of the Licode-based clients other than the browser-based one that we have in the main repository. We don’t even have a clear sense of how popular they are.
Because of that, it looks like we have miscalculated the effect the new SDP negotiation and any big change in ErizoClient has in the community.

We have decided to ease the transition by:

  1. Going back to SDP in plain text messages for negotiation in v5. We will still use the libraries that have been introduced but the message will be stringified before sending it via the websocket. Hopefully this will help alternative clients. We expect to release v5 mid January so, please, as soon as that PR goes live (next week probably), help us testing it and share your thoughts.

  2. ErizoClient negotiation hasn’t changed in years and it hasn’t aged well. We can’t avoid ErizoClient changes forever to avoid changing parts of projects we have no control of. We’re investigating ways we could help those projects that are affected by ErizoClient changes. We cannot take over the development and/or create new iOS or Android sdks but we could definitely be more useful to those working on them. Expect more news about this in the near future, as we flesh out our ideas about that.

However, we still see the Object-based negotiation a much better solution for Licode. Again, even if we didn’t do that, the negotiation has to change to support Single peerconnection and that would affect the clients also. Our current plan is to introduce both, at least partially, in v6. There is some time and we hope we can find a way together to make the mobile clients work. We can also have some sort of online meeting to discuss it (@Javier is working on that, expect more news).

#8

Great! This news is a big relief for us and we really appreciate your collaboration!

I know that the single peerconnection will need some big modification also to the client side but a this stage of the project, this will be a breaking change, with the risk that lots of people using Licode have to stick to the latest non object-based release until they have time to “reinvent the wheel” on their clients.

This will also affect the project itself since we cannot be “helpful” if we are on an old release.

Said that, obviously we need to go further.
You spent lot of time investigating the single peerconnection feature so you have a wider vision.
The singlepeerconnection will be an optional feature I think, since as we said in a previous post, It cannot always be done if various streams are in various different ErizoJSes.
So maybe, we can think a way “backward compatible” in which if we don’t specify that we want the single peerconnection feature, the system will act as it is today, so only the erizo web clients will be using this, and meanwhile we have the time to adapt our various implementations for the new “optional” feature.

#9

My two cents:

In the future I see the new SdpInfo.js like a black box that holds the SDP inside, and provides methods to easily modify the data inside and export it to JSON/String. Some example of function would be processString, toString, add/remove streams, mute/unmute audio/video, setMaxBW, restartIce. I tried to draft a preliminar idea in two minutes here (I’m sorry for the ugly figures…)

SDP JSON (1)
SDP JSON (1).jpg954×486 40.3 KB

Big advantages for parsing the SDP to JSON:

  • It simplifies how we modify session info in the SDP in JS and in Erizo a lot (no more regex, matches, etc.). Erizo will not need to know about Firefox/Chrome/etc. specific implementations anymore, or it will be reduced a lot.
  • It saves bandwidth and simplify signalling by sending only partial updates of the negotiation. There are some examples: adding or removing a new stream or track to a peer connection, ICE restarts, setting max bandwidth, muting streams in the publisher, etc. All of this could be done also using a string, but it would entail changes in the client side too, and it simplifies it a lot to have a common format for sending such updates. (this one is my favourite)
  • Simplify the development of non-webrtc native clients that doesn’t know about SDP and that could send such info in a JSON format, which could be much easier to develop from scratch. Spine is an example of this.

Big disadvantages:

  • Need for implementing the parser for other languages that each native client is using. We’ve implemented it in JS, but there would need to be versions or helpers for Java/Kotlin, Objective-C/Swift, too.
  • There have been comments about the need for tackling the differences between the SDPs generated by Chrome, Firefox, Safari, Edge, etc. Actually, this also affects the string-based SDP. The additional issue is that by sending the raw SDP we also need to change Erizo side in some cases (simulcast, unified vs plan b, etc.)

As @pedro commented we might have miscalculated the effect in native clients because, to be honest, we didn’t even know that there were so many of them out there. But we don’t want to stop improving Licode and evolving Erizo Client. So, as he suggests we’re considering different ideas and alternatives to help developers who own or use native clients to be able to use the latest versions of Licode and update their code to the latest client changes.

@Francesco_Durighetto we’re also considering to keep support for SDP string in future (6+) releases, but IMO it might be a bad idea because we’re just a few developers and it doubles the number of possible issues/bugs/etc that we would need to check when testing features.

#10

@Francesco_Durighetto we’re also considering to keep support for SDP string in future (6+) releases, but IMO it might be a bad idea because we’re just a few developers and it doubles the number of possible issues/bugs/etc that we would need to check when testing features.

I know, but a seamless transition would be better for anyone using Licode from long time.
I actually don’t know how many people will be affected by this change, maybe only me and I don’t want you to do an extra effort for only me and other few guys. But I think supporting both (at least for a short period of time) is a necessary tradeoff.

Keep us informed please!
Have a nice day!